2024-04-15T15:08:08.348 INFO:root:teuthology version: 0.0.1.dev267+g6c63784 2024-04-15T15:08:08.348 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/quiesce} --archive /home/teuthworker/archive/leonidus-2024-04-15_14:54:08-fs-wip-lusov-quiesce-xlock-distro-default-smithi/7657132 --name leonidus-2024-04-15_14:54:08-fs-wip-lusov-quiesce-xlock-distro-default-smithi --owner scheduled_leonidus@teuthology -- /home/teuthworker/archive/leonidus-2024-04-15_14:54:08-fs-wip-lusov-quiesce-xlock-distro-default-smithi/7657132/orig.config.yaml 2024-04-15T15:08:08.396 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-15T15:08:08.485 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-04-15_14:54:08-fs-wip-lusov-quiesce-xlock-distro-default-smithi/7657132 branch: wip-lusov-quiesce-xlock description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/quiesce} email: null first_in_suite: false job_id: '7657132' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: leonidus-2024-04-15_14:54:08-fs-wip-lusov-quiesce-xlock-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lusov-quiesce-xlock ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow request sha1: 9bf1c4ba967dd24077be6e5b87edda4ca87a6e14 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 9bf1c4ba967dd24077be6e5b87edda4ca87a6e14 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 subvolume_version: 1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-quiesce-xlock sha1: 9bf1c4ba967dd24077be6e5b87edda4ca87a6e14 owner: scheduled_leonidus@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 5929 sha1: 9bf1c4ba967dd24077be6e5b87edda4ca87a6e14 sleep_before_teardown: 0 suite: fs suite_branch: wip-lusov-quiesce-xlock suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 9bf1c4ba967dd24077be6e5b87edda4ca87a6e14 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi105.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGtK/GTXibZ8m3DeYPVGTDDt7HFJ9BpvJbR1yk41DfL7oxO7ofgneKZl4qsIl1MlnWxCCYjI3Wgx1TP31Ne8oRM= smithi154.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEfwTt4MW0X5hlmNE9lb0QI3waz4vvcx9Y/jhtC4t1Ux/qKV/DMJmxst2RT3YbWTceKKt3owaT80t6MOstAUvfs= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_quiesce teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 6c637841c215537a4502385240412f1966e0faab timestamp: 2024-04-15_14:54:08 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-15T15:08:08.485 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/qa; will attempt to use it 2024-04-15T15:08:08.486 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/qa/tasks 2024-04-15T15:08:08.486 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-15T15:08:08.488 INFO:teuthology.task.internal:Checking packages... 2024-04-15T15:08:08.514 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '9bf1c4ba967dd24077be6e5b87edda4ca87a6e14' 2024-04-15T15:08:08.514 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-15T15:08:08.514 INFO:teuthology.packaging:ref: None 2024-04-15T15:08:08.514 INFO:teuthology.packaging:tag: None 2024-04-15T15:08:08.514 INFO:teuthology.packaging:branch: wip-lusov-quiesce-xlock 2024-04-15T15:08:08.514 INFO:teuthology.packaging:sha1: 9bf1c4ba967dd24077be6e5b87edda4ca87a6e14 2024-04-15T15:08:08.514 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lusov-quiesce-xlock 2024-04-15T15:08:08.656 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2861.g9bf1c4ba 2024-04-15T15:08:08.658 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-15T15:08:08.662 INFO:teuthology.task.internal:no buildpackages task found 2024-04-15T15:08:08.662 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-15T15:08:08.668 INFO:teuthology.task.internal:Saving configuration 2024-04-15T15:08:08.685 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-15T15:08:08.690 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-15T15:08:08.717 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi105.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-15_14:54:08-fs-wip-lusov-quiesce-xlock-distro-default-smithi/7657132', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-15 15:02:54.101133', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGtK/GTXibZ8m3DeYPVGTDDt7HFJ9BpvJbR1yk41DfL7oxO7ofgneKZl4qsIl1MlnWxCCYjI3Wgx1TP31Ne8oRM='} 2024-04-15T15:08:08.740 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi154.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-15_14:54:08-fs-wip-lusov-quiesce-xlock-distro-default-smithi/7657132', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-15 15:02:54.103150', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEfwTt4MW0X5hlmNE9lb0QI3waz4vvcx9Y/jhtC4t1Ux/qKV/DMJmxst2RT3YbWTceKKt3owaT80t6MOstAUvfs='} 2024-04-15T15:08:08.740 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-15T15:08:08.746 INFO:teuthology.task.internal:roles: ubuntu@smithi105.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-15T15:08:08.746 INFO:teuthology.task.internal:roles: ubuntu@smithi154.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-15T15:08:08.746 INFO:teuthology.run_tasks:Running task console_log... 2024-04-15T15:08:08.814 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7efeebd880d0>, signals=[15]) 2024-04-15T15:08:08.815 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-15T15:08:08.820 INFO:teuthology.task.internal:Opening connections... 2024-04-15T15:08:08.820 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi105.front.sepia.ceph.com 2024-04-15T15:08:08.822 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-15T15:08:08.913 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi154.front.sepia.ceph.com 2024-04-15T15:08:08.914 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-15T15:08:09.005 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-15T15:08:09.012 DEBUG:teuthology.orchestra.run.smithi105:> uname -m 2024-04-15T15:08:09.029 INFO:teuthology.orchestra.run.smithi105.stdout:x86_64 2024-04-15T15:08:09.029 DEBUG:teuthology.orchestra.run.smithi105:> cat /etc/os-release 2024-04-15T15:08:09.086 INFO:teuthology.orchestra.run.smithi105.stdout:NAME="CentOS Stream" 2024-04-15T15:08:09.086 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION="9" 2024-04-15T15:08:09.087 INFO:teuthology.orchestra.run.smithi105.stdout:ID="centos" 2024-04-15T15:08:09.087 INFO:teuthology.orchestra.run.smithi105.stdout:ID_LIKE="rhel fedora" 2024-04-15T15:08:09.087 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION_ID="9" 2024-04-15T15:08:09.087 INFO:teuthology.orchestra.run.smithi105.stdout:PLATFORM_ID="platform:el9" 2024-04-15T15:08:09.087 INFO:teuthology.orchestra.run.smithi105.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-15T15:08:09.087 INFO:teuthology.orchestra.run.smithi105.stdout:ANSI_COLOR="0;31" 2024-04-15T15:08:09.087 INFO:teuthology.orchestra.run.smithi105.stdout:LOGO="fedora-logo-icon" 2024-04-15T15:08:09.087 INFO:teuthology.orchestra.run.smithi105.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-15T15:08:09.087 INFO:teuthology.orchestra.run.smithi105.stdout:HOME_URL="https://centos.org/" 2024-04-15T15:08:09.087 INFO:teuthology.orchestra.run.smithi105.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-15T15:08:09.087 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-15T15:08:09.088 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-15T15:08:09.089 INFO:teuthology.lock.ops:Updating smithi105.front.sepia.ceph.com on lock server 2024-04-15T15:08:09.125 DEBUG:teuthology.orchestra.run.smithi154:> uname -m 2024-04-15T15:08:09.143 INFO:teuthology.orchestra.run.smithi154.stdout:x86_64 2024-04-15T15:08:09.143 DEBUG:teuthology.orchestra.run.smithi154:> cat /etc/os-release 2024-04-15T15:08:09.202 INFO:teuthology.orchestra.run.smithi154.stdout:NAME="CentOS Stream" 2024-04-15T15:08:09.202 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION="9" 2024-04-15T15:08:09.202 INFO:teuthology.orchestra.run.smithi154.stdout:ID="centos" 2024-04-15T15:08:09.203 INFO:teuthology.orchestra.run.smithi154.stdout:ID_LIKE="rhel fedora" 2024-04-15T15:08:09.203 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION_ID="9" 2024-04-15T15:08:09.203 INFO:teuthology.orchestra.run.smithi154.stdout:PLATFORM_ID="platform:el9" 2024-04-15T15:08:09.203 INFO:teuthology.orchestra.run.smithi154.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-15T15:08:09.203 INFO:teuthology.orchestra.run.smithi154.stdout:ANSI_COLOR="0;31" 2024-04-15T15:08:09.203 INFO:teuthology.orchestra.run.smithi154.stdout:LOGO="fedora-logo-icon" 2024-04-15T15:08:09.203 INFO:teuthology.orchestra.run.smithi154.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-15T15:08:09.203 INFO:teuthology.orchestra.run.smithi154.stdout:HOME_URL="https://centos.org/" 2024-04-15T15:08:09.203 INFO:teuthology.orchestra.run.smithi154.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-15T15:08:09.203 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-15T15:08:09.203 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-15T15:08:09.203 INFO:teuthology.lock.ops:Updating smithi154.front.sepia.ceph.com on lock server 2024-04-15T15:08:09.224 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-15T15:08:09.233 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-15T15:08:09.242 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-15T15:08:09.242 DEBUG:teuthology.orchestra.run.smithi105:> test '!' -e /home/ubuntu/cephtest 2024-04-15T15:08:09.245 DEBUG:teuthology.orchestra.run.smithi154:> test '!' -e /home/ubuntu/cephtest 2024-04-15T15:08:09.261 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-15T15:08:09.266 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-15T15:08:09.266 DEBUG:teuthology.orchestra.run.smithi105:> test -z $(ls -A /var/lib/ceph) 2024-04-15T15:08:09.302 DEBUG:teuthology.orchestra.run.smithi154:> test -z $(ls -A /var/lib/ceph) 2024-04-15T15:08:09.413 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-15T15:08:09.472 INFO:teuthology.run_tasks:Running task kernel... 2024-04-15T15:08:09.487 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-04-15T15:08:09.487 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-04-15T15:08:09.488 DEBUG:teuthology.orchestra.run.smithi105:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-15T15:08:09.488 DEBUG:teuthology.orchestra.run.smithi154:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-15T15:08:09.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-15T15:08:09.504 DEBUG:teuthology.orchestra.run.smithi105:> uname -r 2024-04-15T15:08:09.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-15T15:08:09.505 DEBUG:teuthology.orchestra.run.smithi154:> uname -r 2024-04-15T15:08:09.560 INFO:teuthology.orchestra.run.smithi105.stdout:5.14.0-361.el9.x86_64 2024-04-15T15:08:09.561 INFO:teuthology.task.kernel:Running kernel on smithi105: 5.14.0-361.el9.x86_64 2024-04-15T15:08:09.561 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum install -y kernel 2024-04-15T15:08:09.563 INFO:teuthology.orchestra.run.smithi154.stdout:5.14.0-361.el9.x86_64 2024-04-15T15:08:09.563 INFO:teuthology.task.kernel:Running kernel on smithi154: 5.14.0-361.el9.x86_64 2024-04-15T15:08:09.563 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum install -y kernel 2024-04-15T15:08:12.030 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS-9 - AppStream 8.2 MB/s | 6.7 MB 00:00 2024-04-15T15:08:12.799 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS-9 - AppStream 4.1 MB/s | 6.7 MB 00:01 2024-04-15T15:08:13.655 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS-9 - BaseOS 3.6 MB/s | 2.0 MB 00:00 2024-04-15T15:08:14.524 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS-9 - BaseOS 3.3 MB/s | 2.0 MB 00:00 2024-04-15T15:08:14.554 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.0 MB 00:00 2024-04-15T15:08:15.908 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - BaseOS 7.7 MB/s | 8.0 MB 00:01 2024-04-15T15:08:16.738 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - AppStream 16 MB/s | 19 MB 00:01 2024-04-15T15:08:20.989 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - CRB 8.1 MB/s | 6.2 MB 00:00 2024-04-15T15:08:22.566 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - Extras packages 38 kB/s | 16 kB 00:00 2024-04-15T15:08:26.691 INFO:teuthology.orchestra.run.smithi154.stdout:Extra Packages for Enterprise Linux 5.2 MB/s | 21 MB 00:03 2024-04-15T15:08:31.807 INFO:teuthology.orchestra.run.smithi154.stdout:lab-extras 18 kB/s | 1.7 kB 00:00 2024-04-15T15:08:34.181 INFO:teuthology.orchestra.run.smithi154.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-15T15:08:34.181 INFO:teuthology.orchestra.run.smithi154.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-04-15T15:08:34.271 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2024-04-15T15:08:34.275 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2024-04-15T15:08:34.275 INFO:teuthology.orchestra.run.smithi154.stdout: Package Arch Version Repository Size 2024-04-15T15:08:34.275 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2024-04-15T15:08:34.275 INFO:teuthology.orchestra.run.smithi154.stdout:Installing: 2024-04-15T15:08:34.275 INFO:teuthology.orchestra.run.smithi154.stdout: kernel x86_64 5.14.0-438.el9 CentOS-BaseOS 277 k 2024-04-15T15:08:34.276 INFO:teuthology.orchestra.run.smithi154.stdout:Installing dependencies: 2024-04-15T15:08:34.276 INFO:teuthology.orchestra.run.smithi154.stdout: kernel-core x86_64 5.14.0-438.el9 CentOS-BaseOS 15 M 2024-04-15T15:08:34.276 INFO:teuthology.orchestra.run.smithi154.stdout: kernel-modules x86_64 5.14.0-438.el9 CentOS-BaseOS 34 M 2024-04-15T15:08:34.276 INFO:teuthology.orchestra.run.smithi154.stdout: kernel-modules-core x86_64 5.14.0-438.el9 CentOS-BaseOS 28 M 2024-04-15T15:08:34.276 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-15T15:08:34.276 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction Summary 2024-04-15T15:08:34.276 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2024-04-15T15:08:34.276 INFO:teuthology.orchestra.run.smithi154.stdout:Install 4 Packages 2024-04-15T15:08:34.276 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-15T15:08:34.277 INFO:teuthology.orchestra.run.smithi154.stdout:Total download size: 77 M 2024-04-15T15:08:34.277 INFO:teuthology.orchestra.run.smithi154.stdout:Installed size: 123 M 2024-04-15T15:08:34.277 INFO:teuthology.orchestra.run.smithi154.stdout:Downloading Packages: 2024-04-15T15:08:34.434 INFO:teuthology.orchestra.run.smithi154.stdout:(1/4): kernel-5.14.0-438.el9.x86_64.rpm 1.7 MB/s | 277 kB 00:00 2024-04-15T15:08:34.968 INFO:teuthology.orchestra.run.smithi154.stdout:(2/4): kernel-core-5.14.0-438.el9.x86_64.rpm 22 MB/s | 15 MB 00:00 2024-04-15T15:08:35.709 INFO:teuthology.orchestra.run.smithi154.stdout:(3/4): kernel-modules-core-5.14.0-438.el9.x86_6 22 MB/s | 28 MB 00:01 2024-04-15T15:08:36.084 INFO:teuthology.orchestra.run.smithi154.stdout:(4/4): kernel-modules-5.14.0-438.el9.x86_64.rpm 19 MB/s | 34 MB 00:01 2024-04-15T15:08:36.088 INFO:teuthology.orchestra.run.smithi154.stdout:-------------------------------------------------------------------------------- 2024-04-15T15:08:36.088 INFO:teuthology.orchestra.run.smithi154.stdout:Total 42 MB/s | 77 MB 00:01 2024-04-15T15:08:36.088 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction check 2024-04-15T15:08:36.471 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction check succeeded. 2024-04-15T15:08:36.471 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction test 2024-04-15T15:08:37.296 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction test succeeded. 2024-04-15T15:08:37.296 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction 2024-04-15T15:08:38.763 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - AppStream 889 kB/s | 19 MB 00:21 2024-04-15T15:08:39.612 INFO:teuthology.orchestra.run.smithi154.stdout: Preparing : 1/1 2024-04-15T15:08:40.226 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : kernel-modules-core-5.14.0-438.el9.x86_64 1/4 2024-04-15T15:08:40.319 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : kernel-core-5.14.0-438.el9.x86_64 2/4 2024-04-15T15:08:43.107 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: kernel-core-5.14.0-438.el9.x86_64 2/4 2024-04-15T15:08:43.266 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : kernel-modules-5.14.0-438.el9.x86_64 3/4 2024-04-15T15:08:45.705 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - CRB 1.8 MB/s | 6.2 MB 00:03 2024-04-15T15:08:48.030 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: kernel-modules-5.14.0-438.el9.x86_64 3/4 2024-04-15T15:08:48.081 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : kernel-5.14.0-438.el9.x86_64 4/4 2024-04-15T15:08:48.305 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - Extras packages 11 kB/s | 16 kB 00:01 2024-04-15T15:08:49.341 INFO:teuthology.orchestra.run.smithi105.stdout:Extra Packages for Enterprise Linux 23 MB/s | 21 MB 00:00 2024-04-15T15:08:52.822 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: kernel-modules-core-5.14.0-438.el9.x86_64 4/4 2024-04-15T15:08:54.365 INFO:teuthology.orchestra.run.smithi105.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-04-15T15:08:56.730 INFO:teuthology.orchestra.run.smithi105.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-15T15:08:56.731 INFO:teuthology.orchestra.run.smithi105.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-04-15T15:08:56.828 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2024-04-15T15:08:56.839 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2024-04-15T15:08:56.839 INFO:teuthology.orchestra.run.smithi105.stdout: Package Arch Version Repository Size 2024-04-15T15:08:56.839 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2024-04-15T15:08:56.839 INFO:teuthology.orchestra.run.smithi105.stdout:Installing: 2024-04-15T15:08:56.839 INFO:teuthology.orchestra.run.smithi105.stdout: kernel x86_64 5.14.0-438.el9 CentOS-BaseOS 277 k 2024-04-15T15:08:56.839 INFO:teuthology.orchestra.run.smithi105.stdout:Installing dependencies: 2024-04-15T15:08:56.840 INFO:teuthology.orchestra.run.smithi105.stdout: kernel-core x86_64 5.14.0-438.el9 CentOS-BaseOS 15 M 2024-04-15T15:08:56.840 INFO:teuthology.orchestra.run.smithi105.stdout: kernel-modules x86_64 5.14.0-438.el9 CentOS-BaseOS 34 M 2024-04-15T15:08:56.840 INFO:teuthology.orchestra.run.smithi105.stdout: kernel-modules-core x86_64 5.14.0-438.el9 CentOS-BaseOS 28 M 2024-04-15T15:08:56.840 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:08:56.840 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction Summary 2024-04-15T15:08:56.840 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2024-04-15T15:08:56.840 INFO:teuthology.orchestra.run.smithi105.stdout:Install 4 Packages 2024-04-15T15:08:56.840 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:08:56.841 INFO:teuthology.orchestra.run.smithi105.stdout:Total download size: 77 M 2024-04-15T15:08:56.841 INFO:teuthology.orchestra.run.smithi105.stdout:Installed size: 123 M 2024-04-15T15:08:56.841 INFO:teuthology.orchestra.run.smithi105.stdout:Downloading Packages: 2024-04-15T15:08:57.000 INFO:teuthology.orchestra.run.smithi105.stdout:(1/4): kernel-5.14.0-438.el9.x86_64.rpm 1.7 MB/s | 277 kB 00:00 2024-04-15T15:08:57.642 INFO:teuthology.orchestra.run.smithi105.stdout:(2/4): kernel-core-5.14.0-438.el9.x86_64.rpm 19 MB/s | 15 MB 00:00 2024-04-15T15:08:59.192 INFO:teuthology.orchestra.run.smithi105.stdout:(3/4): kernel-modules-5.14.0-438.el9.x86_64.rpm 14 MB/s | 34 MB 00:02 2024-04-15T15:09:00.009 INFO:teuthology.orchestra.run.smithi105.stdout:(4/4): kernel-modules-core-5.14.0-438.el9.x86_6 9.2 MB/s | 28 MB 00:03 2024-04-15T15:09:00.012 INFO:teuthology.orchestra.run.smithi105.stdout:-------------------------------------------------------------------------------- 2024-04-15T15:09:00.012 INFO:teuthology.orchestra.run.smithi105.stdout:Total 24 MB/s | 77 MB 00:03 2024-04-15T15:09:00.013 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction check 2024-04-15T15:09:00.372 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction check succeeded. 2024-04-15T15:09:00.373 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction test 2024-04-15T15:09:01.202 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction test succeeded. 2024-04-15T15:09:01.203 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction 2024-04-15T15:09:03.503 INFO:teuthology.orchestra.run.smithi105.stdout: Preparing : 1/1 2024-04-15T15:09:04.202 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : kernel-modules-core-5.14.0-438.el9.x86_64 1/4 2024-04-15T15:09:04.292 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : kernel-core-5.14.0-438.el9.x86_64 2/4 2024-04-15T15:09:07.043 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: kernel-core-5.14.0-438.el9.x86_64 2/4 2024-04-15T15:09:07.179 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : kernel-modules-5.14.0-438.el9.x86_64 3/4 2024-04-15T15:09:11.869 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: kernel-modules-5.14.0-438.el9.x86_64 3/4 2024-04-15T15:09:11.911 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : kernel-5.14.0-438.el9.x86_64 4/4 2024-04-15T15:09:16.682 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: kernel-modules-core-5.14.0-438.el9.x86_64 4/4 2024-04-15T15:09:22.174 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: kernel-core-5.14.0-438.el9.x86_64 4/4 2024-04-15T15:09:22.174 INFO:teuthology.orchestra.run.smithi154.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-04-15T15:09:22.174 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-15T15:09:22.217 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: kernel-modules-5.14.0-438.el9.x86_64 4/4 2024-04-15T15:09:22.616 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: kernel-5.14.0-438.el9.x86_64 4/4 2024-04-15T15:09:22.616 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : kernel-5.14.0-438.el9.x86_64 1/4 2024-04-15T15:09:22.616 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : kernel-core-5.14.0-438.el9.x86_64 2/4 2024-04-15T15:09:22.616 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : kernel-modules-5.14.0-438.el9.x86_64 3/4 2024-04-15T15:09:23.068 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : kernel-modules-core-5.14.0-438.el9.x86_64 4/4 2024-04-15T15:09:23.068 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-15T15:09:23.069 INFO:teuthology.orchestra.run.smithi154.stdout:Installed: 2024-04-15T15:09:23.069 INFO:teuthology.orchestra.run.smithi154.stdout: kernel-5.14.0-438.el9.x86_64 2024-04-15T15:09:23.069 INFO:teuthology.orchestra.run.smithi154.stdout: kernel-core-5.14.0-438.el9.x86_64 2024-04-15T15:09:23.069 INFO:teuthology.orchestra.run.smithi154.stdout: kernel-modules-5.14.0-438.el9.x86_64 2024-04-15T15:09:23.069 INFO:teuthology.orchestra.run.smithi154.stdout: kernel-modules-core-5.14.0-438.el9.x86_64 2024-04-15T15:09:23.069 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-15T15:09:23.069 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2024-04-15T15:09:23.397 DEBUG:teuthology.orchestra.run.smithi154:> rpm -q kernel | sort -rV | head -n 1 2024-04-15T15:09:23.501 INFO:teuthology.orchestra.run.smithi154.stdout:kernel-5.14.0-438.el9.x86_64 2024-04-15T15:09:23.501 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-438.el9.x86_64 2024-04-15T15:09:23.501 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-438.el9.x86_64 2024-04-15T15:09:23.501 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-15T15:09:23.501 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-15T15:09:23.502 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-04-15T15:09:23.502 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi154.front.sepia.ceph.com, path=None, version=distro) 2024-04-15T15:09:23.502 DEBUG:teuthology.orchestra.run.smithi154:> rpm -q kernel | sort -rV | head -n 1 2024-04-15T15:09:23.583 INFO:teuthology.orchestra.run.smithi154.stdout:kernel-5.14.0-438.el9.x86_64 2024-04-15T15:09:23.583 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-438.el9.x86_64 2024-04-15T15:09:23.583 DEBUG:teuthology.orchestra.run.smithi154:> sudo rpm -qi grub2-tools 2024-04-15T15:09:23.617 INFO:teuthology.orchestra.run.smithi154.stdout:Name : grub2-tools 2024-04-15T15:09:23.618 INFO:teuthology.orchestra.run.smithi154.stdout:Epoch : 1 2024-04-15T15:09:23.618 INFO:teuthology.orchestra.run.smithi154.stdout:Version : 2.06 2024-04-15T15:09:23.618 INFO:teuthology.orchestra.run.smithi154.stdout:Release : 68.el9 2024-04-15T15:09:23.618 INFO:teuthology.orchestra.run.smithi154.stdout:Architecture: x86_64 2024-04-15T15:09:23.618 INFO:teuthology.orchestra.run.smithi154.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-04-15T15:09:23.618 INFO:teuthology.orchestra.run.smithi154.stdout:Group : Unspecified 2024-04-15T15:09:23.618 INFO:teuthology.orchestra.run.smithi154.stdout:Size : 8264254 2024-04-15T15:09:23.618 INFO:teuthology.orchestra.run.smithi154.stdout:License : GPLv3+ 2024-04-15T15:09:23.618 INFO:teuthology.orchestra.run.smithi154.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-15T15:09:23.618 INFO:teuthology.orchestra.run.smithi154.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-15T15:09:23.618 INFO:teuthology.orchestra.run.smithi154.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-15T15:09:23.618 INFO:teuthology.orchestra.run.smithi154.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-15T15:09:23.619 INFO:teuthology.orchestra.run.smithi154.stdout:Packager : builder@centos.org 2024-04-15T15:09:23.619 INFO:teuthology.orchestra.run.smithi154.stdout:Vendor : CentOS 2024-04-15T15:09:23.619 INFO:teuthology.orchestra.run.smithi154.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-15T15:09:23.619 INFO:teuthology.orchestra.run.smithi154.stdout:Summary : Support tools for GRUB. 2024-04-15T15:09:23.619 INFO:teuthology.orchestra.run.smithi154.stdout:Description : 2024-04-15T15:09:23.619 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-15T15:09:23.619 INFO:teuthology.orchestra.run.smithi154.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-15T15:09:23.619 INFO:teuthology.orchestra.run.smithi154.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-15T15:09:23.619 INFO:teuthology.orchestra.run.smithi154.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-15T15:09:23.619 INFO:teuthology.orchestra.run.smithi154.stdout:hardware devices. 2024-04-15T15:09:23.619 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-15T15:09:23.620 INFO:teuthology.orchestra.run.smithi154.stdout:This subpackage provides tools for support of all platforms. 2024-04-15T15:09:23.621 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-15T15:09:23.621 INFO:teuthology.task.kernel:Updating grub on smithi154 to boot 5.14.0-438.el9.x86_64 2024-04-15T15:09:23.621 DEBUG:teuthology.orchestra.run.smithi154:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-15T15:09:24.224 INFO:teuthology.orchestra.run.smithi154.stderr:Generating grub configuration file ... 2024-04-15T15:09:25.614 INFO:teuthology.orchestra.run.smithi154.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-15T15:09:25.651 INFO:teuthology.orchestra.run.smithi154.stderr:done 2024-04-15T15:09:25.654 DEBUG:teuthology.orchestra.run.smithi154:> mktemp 2024-04-15T15:09:25.669 INFO:teuthology.orchestra.run.smithi154.stdout:/tmp/tmp.4umLZC2Ofk 2024-04-15T15:09:25.669 DEBUG:teuthology.orchestra.run.smithi154:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.4umLZC2Ofk 2024-04-15T15:09:25.737 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 0666 /tmp/tmp.4umLZC2Ofk 2024-04-15T15:09:25.871 DEBUG:teuthology.orchestra.remote:smithi154:/tmp/tmp.4umLZC2Ofk is 6KB 2024-04-15T15:09:25.920 DEBUG:teuthology.orchestra.run.smithi154:> rm -fr /tmp/tmp.4umLZC2Ofk 2024-04-15T15:09:25.934 DEBUG:teuthology.orchestra.run.smithi154:> sudo /bin/ls /boot/loader/entries || true 2024-04-15T15:09:25.998 INFO:teuthology.orchestra.run.smithi154.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-04-15T15:09:25.999 INFO:teuthology.orchestra.run.smithi154.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-04-15T15:09:25.999 INFO:teuthology.orchestra.run.smithi154.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-04-15T15:09:25.999 INFO:teuthology.orchestra.run.smithi154.stdout:15949e3c88704300882482eddfd7baae-5.14.0-438.el9.x86_64.conf 2024-04-15T15:09:26.000 DEBUG:teuthology.orchestra.run.smithi154:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-438.el9.x86_64 2024-04-15T15:09:26.139 DEBUG:teuthology.orchestra.run.smithi154:> sudo shutdown -r now 2024-04-15T15:09:46.369 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: kernel-core-5.14.0-438.el9.x86_64 4/4 2024-04-15T15:09:46.369 INFO:teuthology.orchestra.run.smithi105.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-04-15T15:09:46.369 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:09:46.412 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: kernel-modules-5.14.0-438.el9.x86_64 4/4 2024-04-15T15:09:46.786 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: kernel-5.14.0-438.el9.x86_64 4/4 2024-04-15T15:09:46.786 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : kernel-5.14.0-438.el9.x86_64 1/4 2024-04-15T15:09:46.786 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : kernel-core-5.14.0-438.el9.x86_64 2/4 2024-04-15T15:09:46.787 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : kernel-modules-5.14.0-438.el9.x86_64 3/4 2024-04-15T15:09:47.205 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : kernel-modules-core-5.14.0-438.el9.x86_64 4/4 2024-04-15T15:09:47.205 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:09:47.205 INFO:teuthology.orchestra.run.smithi105.stdout:Installed: 2024-04-15T15:09:47.205 INFO:teuthology.orchestra.run.smithi105.stdout: kernel-5.14.0-438.el9.x86_64 2024-04-15T15:09:47.205 INFO:teuthology.orchestra.run.smithi105.stdout: kernel-core-5.14.0-438.el9.x86_64 2024-04-15T15:09:47.205 INFO:teuthology.orchestra.run.smithi105.stdout: kernel-modules-5.14.0-438.el9.x86_64 2024-04-15T15:09:47.205 INFO:teuthology.orchestra.run.smithi105.stdout: kernel-modules-core-5.14.0-438.el9.x86_64 2024-04-15T15:09:47.205 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:09:47.205 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2024-04-15T15:09:47.557 DEBUG:teuthology.orchestra.run.smithi105:> rpm -q kernel | sort -rV | head -n 1 2024-04-15T15:09:47.656 INFO:teuthology.orchestra.run.smithi105.stdout:kernel-5.14.0-438.el9.x86_64 2024-04-15T15:09:47.656 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-438.el9.x86_64 2024-04-15T15:09:47.656 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-438.el9.x86_64 2024-04-15T15:09:47.656 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-15T15:09:47.656 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-15T15:09:47.657 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-04-15T15:09:47.657 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi105.front.sepia.ceph.com, path=None, version=distro) 2024-04-15T15:09:47.657 DEBUG:teuthology.orchestra.run.smithi105:> rpm -q kernel | sort -rV | head -n 1 2024-04-15T15:09:47.737 INFO:teuthology.orchestra.run.smithi105.stdout:kernel-5.14.0-438.el9.x86_64 2024-04-15T15:09:47.738 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-438.el9.x86_64 2024-04-15T15:09:47.738 DEBUG:teuthology.orchestra.run.smithi105:> sudo rpm -qi grub2-tools 2024-04-15T15:09:47.780 INFO:teuthology.orchestra.run.smithi105.stdout:Name : grub2-tools 2024-04-15T15:09:47.780 INFO:teuthology.orchestra.run.smithi105.stdout:Epoch : 1 2024-04-15T15:09:47.780 INFO:teuthology.orchestra.run.smithi105.stdout:Version : 2.06 2024-04-15T15:09:47.780 INFO:teuthology.orchestra.run.smithi105.stdout:Release : 68.el9 2024-04-15T15:09:47.780 INFO:teuthology.orchestra.run.smithi105.stdout:Architecture: x86_64 2024-04-15T15:09:47.781 INFO:teuthology.orchestra.run.smithi105.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-04-15T15:09:47.781 INFO:teuthology.orchestra.run.smithi105.stdout:Group : Unspecified 2024-04-15T15:09:47.781 INFO:teuthology.orchestra.run.smithi105.stdout:Size : 8264254 2024-04-15T15:09:47.781 INFO:teuthology.orchestra.run.smithi105.stdout:License : GPLv3+ 2024-04-15T15:09:47.781 INFO:teuthology.orchestra.run.smithi105.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-15T15:09:47.781 INFO:teuthology.orchestra.run.smithi105.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-15T15:09:47.781 INFO:teuthology.orchestra.run.smithi105.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-15T15:09:47.781 INFO:teuthology.orchestra.run.smithi105.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-15T15:09:47.781 INFO:teuthology.orchestra.run.smithi105.stdout:Packager : builder@centos.org 2024-04-15T15:09:47.781 INFO:teuthology.orchestra.run.smithi105.stdout:Vendor : CentOS 2024-04-15T15:09:47.782 INFO:teuthology.orchestra.run.smithi105.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-15T15:09:47.782 INFO:teuthology.orchestra.run.smithi105.stdout:Summary : Support tools for GRUB. 2024-04-15T15:09:47.782 INFO:teuthology.orchestra.run.smithi105.stdout:Description : 2024-04-15T15:09:47.782 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:09:47.782 INFO:teuthology.orchestra.run.smithi105.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-15T15:09:47.782 INFO:teuthology.orchestra.run.smithi105.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-15T15:09:47.782 INFO:teuthology.orchestra.run.smithi105.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-15T15:09:47.782 INFO:teuthology.orchestra.run.smithi105.stdout:hardware devices. 2024-04-15T15:09:47.782 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:09:47.782 INFO:teuthology.orchestra.run.smithi105.stdout:This subpackage provides tools for support of all platforms. 2024-04-15T15:09:47.783 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-15T15:09:47.783 INFO:teuthology.task.kernel:Updating grub on smithi105 to boot 5.14.0-438.el9.x86_64 2024-04-15T15:09:47.784 DEBUG:teuthology.orchestra.run.smithi105:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-15T15:09:48.438 INFO:teuthology.orchestra.run.smithi105.stderr:Generating grub configuration file ... 2024-04-15T15:09:49.791 INFO:teuthology.orchestra.run.smithi105.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-15T15:09:49.821 INFO:teuthology.orchestra.run.smithi105.stderr:done 2024-04-15T15:09:49.823 DEBUG:teuthology.orchestra.run.smithi105:> mktemp 2024-04-15T15:09:49.838 INFO:teuthology.orchestra.run.smithi105.stdout:/tmp/tmp.rzTu1kh6of 2024-04-15T15:09:49.838 DEBUG:teuthology.orchestra.run.smithi105:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.rzTu1kh6of 2024-04-15T15:09:49.907 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 0666 /tmp/tmp.rzTu1kh6of 2024-04-15T15:09:50.048 DEBUG:teuthology.orchestra.remote:smithi105:/tmp/tmp.rzTu1kh6of is 6KB 2024-04-15T15:09:50.096 DEBUG:teuthology.orchestra.run.smithi105:> rm -fr /tmp/tmp.rzTu1kh6of 2024-04-15T15:09:50.110 DEBUG:teuthology.orchestra.run.smithi105:> sudo /bin/ls /boot/loader/entries || true 2024-04-15T15:09:50.175 INFO:teuthology.orchestra.run.smithi105.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-04-15T15:09:50.175 INFO:teuthology.orchestra.run.smithi105.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-04-15T15:09:50.175 INFO:teuthology.orchestra.run.smithi105.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-04-15T15:09:50.175 INFO:teuthology.orchestra.run.smithi105.stdout:15949e3c88704300882482eddfd7baae-5.14.0-438.el9.x86_64.conf 2024-04-15T15:09:50.176 DEBUG:teuthology.orchestra.run.smithi105:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-438.el9.x86_64 2024-04-15T15:09:50.335 DEBUG:teuthology.orchestra.run.smithi105:> sudo shutdown -r now 2024-04-15T15:09:56.147 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-04-15T15:09:56.147 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2024-04-15T15:09:56.148 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-15T15:10:14.563 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.154 2024-04-15T15:10:20.342 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-04-15T15:10:20.342 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-04-15T15:10:20.343 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-15T15:10:23.567 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2024-04-15T15:10:23.568 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-15T15:10:26.622 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.154 2024-04-15T15:10:38.635 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2024-04-15T15:10:38.636 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-15T15:10:38.882 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.105 2024-04-15T15:10:47.890 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-04-15T15:10:47.891 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-15T15:10:50.942 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.105 2024-04-15T15:11:02.954 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-04-15T15:11:02.955 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-15T15:11:34.774 DEBUG:teuthology.orchestra.run.smithi105:> true 2024-04-15T15:11:35.266 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-04-15T15:11:35.267 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-438.el9.x86_64"... 2024-04-15T15:11:35.267 DEBUG:teuthology.orchestra.run.smithi105:> uname -r 2024-04-15T15:11:35.292 INFO:teuthology.orchestra.run.smithi105.stdout:5.14.0-438.el9.x86_64 2024-04-15T15:11:35.292 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-438.el9.x86_64 vs 5.14.0-438.el9.x86_64 2024-04-15T15:11:35.292 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-15T15:11:35.292 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-15T15:11:36.293 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-15T15:11:36.293 DEBUG:teuthology.orchestra.run.smithi105:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-15T15:11:36.360 INFO:teuthology.orchestra.run.smithi105.stdout:ttyS1 2024-04-15T15:11:36.399 DEBUG:teuthology.parallel:result is None 2024-04-15T15:11:38.638 DEBUG:teuthology.orchestra.remote:timed out 2024-04-15T15:11:53.640 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2024-04-15T15:11:53.641 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-15T15:11:54.110 DEBUG:teuthology.orchestra.run.smithi154:> true 2024-04-15T15:11:54.488 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi154.front.sepia.ceph.com' 2024-04-15T15:11:54.488 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-438.el9.x86_64"... 2024-04-15T15:11:54.488 DEBUG:teuthology.orchestra.run.smithi154:> uname -r 2024-04-15T15:11:54.504 INFO:teuthology.orchestra.run.smithi154.stdout:5.14.0-438.el9.x86_64 2024-04-15T15:11:54.505 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-438.el9.x86_64 vs 5.14.0-438.el9.x86_64 2024-04-15T15:11:54.505 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-15T15:11:54.505 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-15T15:11:55.506 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-15T15:11:55.506 DEBUG:teuthology.orchestra.run.smithi154:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-15T15:11:55.594 INFO:teuthology.orchestra.run.smithi154.stdout:ttyS1 2024-04-15T15:11:55.622 DEBUG:teuthology.parallel:result is None 2024-04-15T15:11:55.622 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-15T15:11:55.631 INFO:teuthology.task.internal:Creating test directory... 2024-04-15T15:11:55.631 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-15T15:11:55.634 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-15T15:11:55.651 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-15T15:11:55.658 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-15T15:11:55.666 INFO:teuthology.task.internal:Creating archive directory... 2024-04-15T15:11:55.667 DEBUG:teuthology.orchestra.run.smithi105:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-15T15:11:55.693 DEBUG:teuthology.orchestra.run.smithi154:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-15T15:11:55.741 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-15T15:11:55.748 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-15T15:11:55.748 DEBUG:teuthology.orchestra.run.smithi105:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-15T15:11:55.750 DEBUG:teuthology.orchestra.run.smithi154:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-15T15:11:55.782 INFO:teuthology.orchestra.run.smithi105.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-15T15:11:55.797 INFO:teuthology.orchestra.run.smithi105.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-15T15:11:55.800 INFO:teuthology.orchestra.run.smithi154.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-15T15:11:55.814 INFO:teuthology.orchestra.run.smithi154.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-15T15:11:55.816 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-15T15:11:55.823 INFO:teuthology.task.internal:Configuring sudo... 2024-04-15T15:11:55.823 DEBUG:teuthology.orchestra.run.smithi105:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-15T15:11:55.841 DEBUG:teuthology.orchestra.run.smithi154:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-15T15:11:55.887 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-15T15:11:55.897 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-15T15:11:55.898 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-15T15:11:55.911 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-15T15:11:55.944 DEBUG:teuthology.orchestra.run.smithi105:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-15T15:11:56.008 DEBUG:teuthology.orchestra.run.smithi105:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-15T15:11:56.056 DEBUG:teuthology.orchestra.run.smithi105:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-15T15:11:56.143 DEBUG:teuthology.orchestra.run.smithi105:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-15T15:11:56.183 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-04-15T15:11:56.183 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-15T15:11:56.252 DEBUG:teuthology.orchestra.run.smithi154:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-15T15:11:56.296 DEBUG:teuthology.orchestra.run.smithi154:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-15T15:11:56.357 DEBUG:teuthology.orchestra.run.smithi154:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-15T15:11:56.399 DEBUG:teuthology.orchestra.run.smithi154:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-15T15:11:56.444 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-15T15:11:56.444 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-15T15:11:56.513 DEBUG:teuthology.orchestra.run.smithi105:> sudo service rsyslog restart 2024-04-15T15:11:56.515 DEBUG:teuthology.orchestra.run.smithi154:> sudo service rsyslog restart 2024-04-15T15:11:56.563 INFO:teuthology.orchestra.run.smithi105.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-15T15:11:56.597 INFO:teuthology.orchestra.run.smithi154.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-15T15:11:56.840 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-15T15:11:56.848 INFO:teuthology.task.internal:Starting timer... 2024-04-15T15:11:56.848 INFO:teuthology.run_tasks:Running task pcp... 2024-04-15T15:11:56.861 INFO:teuthology.run_tasks:Running task selinux... 2024-04-15T15:11:56.869 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-04-15T15:11:56.870 DEBUG:teuthology.orchestra.run.smithi105:> sudo service auditd rotate 2024-04-15T15:11:56.917 INFO:teuthology.orchestra.run.smithi105.stdout:Rotating logs: 2024-04-15T15:11:56.919 DEBUG:teuthology.orchestra.run.smithi154:> sudo service auditd rotate 2024-04-15T15:11:56.985 INFO:teuthology.orchestra.run.smithi154.stdout:Rotating logs: 2024-04-15T15:11:56.986 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-15T15:11:56.987 DEBUG:teuthology.orchestra.run.smithi105:> /usr/sbin/getenforce 2024-04-15T15:11:57.016 INFO:teuthology.orchestra.run.smithi105.stdout:Permissive 2024-04-15T15:11:57.017 DEBUG:teuthology.orchestra.run.smithi154:> /usr/sbin/getenforce 2024-04-15T15:11:57.051 INFO:teuthology.orchestra.run.smithi154.stdout:Permissive 2024-04-15T15:11:57.052 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi105.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi154.front.sepia.ceph.com': 'permissive'} 2024-04-15T15:11:57.052 DEBUG:teuthology.orchestra.run.smithi105:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-15T15:11:57.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-15T15:11:57.085 DEBUG:teuthology.orchestra.run.smithi154:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-15T15:11:57.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-15T15:11:57.122 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-15T15:11:57.122 DEBUG:teuthology.orchestra.run.smithi105:> sudo /usr/sbin/setenforce permissive 2024-04-15T15:11:57.161 DEBUG:teuthology.orchestra.run.smithi154:> sudo /usr/sbin/setenforce permissive 2024-04-15T15:11:57.196 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-15T15:11:57.209 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-15T15:11:57.226 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-15T15:11:57.227 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi105.front.sepia.ceph.com,smithi154.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-15T15:16:14.709 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi105.front.sepia.ceph.com'), Remote(name='ubuntu@smithi154.front.sepia.ceph.com')] 2024-04-15T15:16:14.711 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-04-15T15:16:14.712 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-15T15:16:14.803 DEBUG:teuthology.orchestra.run.smithi105:> true 2024-04-15T15:16:14.889 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-04-15T15:16:14.890 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2024-04-15T15:16:14.890 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-15T15:16:14.981 DEBUG:teuthology.orchestra.run.smithi154:> true 2024-04-15T15:16:15.062 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi154.front.sepia.ceph.com' 2024-04-15T15:16:15.063 INFO:teuthology.run_tasks:Running task clock... 2024-04-15T15:16:15.075 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-15T15:16:15.075 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-15T15:16:15.076 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-15T15:16:15.078 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-15T15:16:15.079 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-15T15:16:15.109 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-15T15:16:15.126 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-15T15:16:15.136 INFO:teuthology.orchestra.run.smithi154.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-15T15:16:15.154 INFO:teuthology.orchestra.run.smithi154.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-15T15:16:15.157 INFO:teuthology.orchestra.run.smithi105.stderr:sudo: ntpd: command not found 2024-04-15T15:16:15.172 INFO:teuthology.orchestra.run.smithi105.stdout:506 Cannot talk to daemon 2024-04-15T15:16:15.184 INFO:teuthology.orchestra.run.smithi154.stderr:sudo: ntpd: command not found 2024-04-15T15:16:15.190 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-15T15:16:15.198 INFO:teuthology.orchestra.run.smithi154.stdout:506 Cannot talk to daemon 2024-04-15T15:16:15.206 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-15T15:16:15.216 INFO:teuthology.orchestra.run.smithi154.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-15T15:16:15.233 INFO:teuthology.orchestra.run.smithi154.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-15T15:16:15.266 INFO:teuthology.orchestra.run.smithi105.stderr:bash: line 1: ntpq: command not found 2024-04-15T15:16:15.269 INFO:teuthology.orchestra.run.smithi105.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-15T15:16:15.269 INFO:teuthology.orchestra.run.smithi105.stdout:=============================================================================== 2024-04-15T15:16:15.269 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-15T15:16:15.269 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-15T15:16:15.269 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-15T15:16:15.269 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-15T15:16:15.287 INFO:teuthology.orchestra.run.smithi154.stderr:bash: line 1: ntpq: command not found 2024-04-15T15:16:15.290 INFO:teuthology.orchestra.run.smithi154.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-15T15:16:15.290 INFO:teuthology.orchestra.run.smithi154.stdout:=============================================================================== 2024-04-15T15:16:15.291 INFO:teuthology.orchestra.run.smithi154.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-15T15:16:15.291 INFO:teuthology.orchestra.run.smithi154.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-15T15:16:15.291 INFO:teuthology.orchestra.run.smithi154.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-15T15:16:15.291 INFO:teuthology.orchestra.run.smithi154.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-15T15:16:15.292 INFO:teuthology.run_tasks:Running task install... 2024-04-15T15:16:15.301 DEBUG:teuthology.task.install:project ceph 2024-04-15T15:16:15.301 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '9bf1c4ba967dd24077be6e5b87edda4ca87a6e14'}} 2024-04-15T15:16:15.301 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '9bf1c4ba967dd24077be6e5b87edda4ca87a6e14'} 2024-04-15T15:16:15.301 INFO:teuthology.task.install:Using flavor: default 2024-04-15T15:16:15.307 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-15T15:16:15.307 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-15T15:16:15.307 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9bf1c4ba967dd24077be6e5b87edda4ca87a6e14', 'tag': None, 'wait_for_package': False} 2024-04-15T15:16:15.308 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9bf1c4ba967dd24077be6e5b87edda4ca87a6e14 2024-04-15T15:16:15.311 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9bf1c4ba967dd24077be6e5b87edda4ca87a6e14', 'tag': None, 'wait_for_package': False} 2024-04-15T15:16:15.311 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9bf1c4ba967dd24077be6e5b87edda4ca87a6e14 2024-04-15T15:16:15.459 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce-xlock/9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/centos/9/flavors/default/ 2024-04-15T15:16:15.460 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2861.g9bf1c4ba 2024-04-15T15:16:15.565 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce-xlock/9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/centos/9/flavors/default/ 2024-04-15T15:16:15.565 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2861.g9bf1c4ba 2024-04-15T15:16:15.646 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce-xlock/9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce-xlock/9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce-xlock/9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-15T15:16:15.646 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-15T15:16:15.646 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-15T15:16:15.676 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-15T15:16:15.676 DEBUG:teuthology.orchestra.run.smithi154:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-15T15:16:15.679 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce-xlock/9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce-xlock/9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce-xlock/9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-15T15:16:15.679 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-04-15T15:16:15.679 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-15T15:16:15.708 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-15T15:16:15.708 DEBUG:teuthology.orchestra.run.smithi105:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-15T15:16:15.745 DEBUG:teuthology.orchestra.run.smithi154:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-15T15:16:15.777 DEBUG:teuthology.orchestra.run.smithi105:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-15T15:16:15.828 DEBUG:teuthology.orchestra.run.smithi154:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-15T15:16:15.859 DEBUG:teuthology.orchestra.run.smithi105:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-15T15:16:15.896 INFO:teuthology.orchestra.run.smithi154.stdout:check_obsoletes = 1 2024-04-15T15:16:15.899 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum clean all 2024-04-15T15:16:15.927 INFO:teuthology.orchestra.run.smithi105.stdout:check_obsoletes = 1 2024-04-15T15:16:15.929 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum clean all 2024-04-15T15:16:16.138 INFO:teuthology.orchestra.run.smithi154.stdout:74 files removed 2024-04-15T15:16:16.166 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-15T15:16:16.175 INFO:teuthology.orchestra.run.smithi105.stdout:74 files removed 2024-04-15T15:16:16.201 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-15T15:16:16.878 INFO:teuthology.orchestra.run.smithi105.stdout:ceph packages for x86_64 223 kB/s | 83 kB 00:00 2024-04-15T15:16:16.921 INFO:teuthology.orchestra.run.smithi154.stdout:ceph packages for x86_64 186 kB/s | 83 kB 00:00 2024-04-15T15:16:17.220 INFO:teuthology.orchestra.run.smithi105.stdout:ceph noarch packages 39 kB/s | 11 kB 00:00 2024-04-15T15:16:17.281 INFO:teuthology.orchestra.run.smithi154.stdout:ceph noarch packages 37 kB/s | 11 kB 00:00 2024-04-15T15:16:17.607 INFO:teuthology.orchestra.run.smithi154.stdout:ceph source packages 6.6 kB/s | 1.9 kB 00:00 2024-04-15T15:16:17.646 INFO:teuthology.orchestra.run.smithi105.stdout:ceph source packages 4.9 kB/s | 1.9 kB 00:00 2024-04-15T15:16:18.373 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS-9 - AppStream 9.7 MB/s | 6.7 MB 00:00 2024-04-15T15:16:18.392 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS-9 - AppStream 9.0 MB/s | 6.7 MB 00:00 2024-04-15T15:16:19.882 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS-9 - BaseOS 5.1 MB/s | 2.0 MB 00:00 2024-04-15T15:16:29.155 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - BaseOS 920 kB/s | 8.0 MB 00:08 2024-04-15T15:16:30.934 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS-9 - BaseOS 180 kB/s | 2.0 MB 00:11 2024-04-15T15:16:31.442 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - AppStream 16 MB/s | 19 MB 00:01 2024-04-15T15:16:32.915 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - BaseOS 4.9 MB/s | 8.0 MB 00:01 2024-04-15T15:16:36.716 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - CRB 3.5 MB/s | 6.2 MB 00:01 2024-04-15T15:16:38.508 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 16 kB 00:00 2024-04-15T15:16:40.196 INFO:teuthology.orchestra.run.smithi105.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-04-15T15:16:45.123 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - AppStream 1.7 MB/s | 19 MB 00:11 2024-04-15T15:16:45.193 INFO:teuthology.orchestra.run.smithi105.stdout:lab-extras 16 kB/s | 1.7 kB 00:00 2024-04-15T15:16:47.871 INFO:teuthology.orchestra.run.smithi105.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-15T15:16:47.872 INFO:teuthology.orchestra.run.smithi105.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-15T15:16:47.884 INFO:teuthology.orchestra.run.smithi105.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-04-15T15:16:47.885 INFO:teuthology.orchestra.run.smithi105.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-15T15:16:47.886 INFO:teuthology.orchestra.run.smithi105.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-04-15T15:16:47.887 INFO:teuthology.orchestra.run.smithi105.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-04-15T15:16:47.888 INFO:teuthology.orchestra.run.smithi105.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-15T15:16:47.891 INFO:teuthology.orchestra.run.smithi105.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-04-15T15:16:47.891 INFO:teuthology.orchestra.run.smithi105.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-04-15T15:16:47.970 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2024-04-15T15:16:47.998 INFO:teuthology.orchestra.run.smithi105.stdout:============================================================================================ 2024-04-15T15:16:47.998 INFO:teuthology.orchestra.run.smithi105.stdout: Package Arch Version Repository Size 2024-04-15T15:16:47.998 INFO:teuthology.orchestra.run.smithi105.stdout:============================================================================================ 2024-04-15T15:16:47.998 INFO:teuthology.orchestra.run.smithi105.stdout:Installing: 2024-04-15T15:16:47.998 INFO:teuthology.orchestra.run.smithi105.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-04-15T15:16:47.998 INFO:teuthology.orchestra.run.smithi105.stdout: ceph x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 6.6 k 2024-04-15T15:16:47.998 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-base x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 5.4 M 2024-04-15T15:16:47.998 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-fuse x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 890 k 2024-04-15T15:16:47.998 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 145 k 2024-04-15T15:16:47.998 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 1.5 M 2024-04-15T15:16:47.999 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2861.g9bf1c4ba.el8 ceph-noarch 142 k 2024-04-15T15:16:47.999 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2861.g9bf1c4ba.el8 ceph-noarch 3.5 M 2024-04-15T15:16:47.999 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2861.g9bf1c4ba.el8 ceph-noarch 7.4 M 2024-04-15T15:16:47.999 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-rook noarch 2:19.0.0-2861.g9bf1c4ba.el8 ceph-noarch 49 k 2024-04-15T15:16:47.999 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-radosgw x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 10 M 2024-04-15T15:16:47.999 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-test x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 49 M 2024-04-15T15:16:47.999 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-volume noarch 2:19.0.0-2861.g9bf1c4ba.el8 ceph-noarch 264 k 2024-04-15T15:16:47.999 INFO:teuthology.orchestra.run.smithi105.stdout: cephadm noarch 2:19.0.0-2861.g9bf1c4ba.el8 ceph-noarch 762 k 2024-04-15T15:16:47.999 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-mirror x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 224 k 2024-04-15T15:16:47.999 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-shell x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 27 k 2024-04-15T15:16:47.999 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-top noarch 2:19.0.0-2861.g9bf1c4ba.el8 ceph-noarch 18 k 2024-04-15T15:16:47.999 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-15T15:16:47.999 INFO:teuthology.orchestra.run.smithi105.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-04-15T15:16:47.999 INFO:teuthology.orchestra.run.smithi105.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-04-15T15:16:48.000 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs-devel x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 32 k 2024-04-15T15:16:48.000 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs2 x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 747 k 2024-04-15T15:16:48.000 INFO:teuthology.orchestra.run.smithi105.stdout: librados-devel x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 127 k 2024-04-15T15:16:48.000 INFO:teuthology.orchestra.run.smithi105.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-04-15T15:16:48.000 INFO:teuthology.orchestra.run.smithi105.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-15T15:16:48.000 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-15T15:16:48.000 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql x86_64 13.14-1.el9 CentOS-AppStream 1.5 M 2024-04-15T15:16:48.000 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-contrib x86_64 13.14-1.el9 CentOS-AppStream 817 k 2024-04-15T15:16:48.000 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-server x86_64 13.14-1.el9 CentOS-AppStream 5.7 M 2024-04-15T15:16:48.000 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cephfs x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 162 k 2024-04-15T15:16:48.000 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rados x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 320 k 2024-04-15T15:16:48.000 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rbd x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 297 k 2024-04-15T15:16:48.001 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rgw x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 99 k 2024-04-15T15:16:48.001 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-fuse x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 85 k 2024-04-15T15:16:48.001 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-mirror x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 3.1 M 2024-04-15T15:16:48.001 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-nbd x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 171 k 2024-04-15T15:16:48.001 INFO:teuthology.orchestra.run.smithi105.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-15T15:16:48.001 INFO:teuthology.orchestra.run.smithi105.stdout:Upgrading: 2024-04-15T15:16:48.001 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager x86_64 1:1.47.4-1.el9 CentOS-BaseOS 2.3 M 2024-04-15T15:16:48.002 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-libnm x86_64 1:1.47.4-1.el9 CentOS-BaseOS 1.8 M 2024-04-15T15:16:48.002 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-team x86_64 1:1.47.4-1.el9 CentOS-BaseOS 41 k 2024-04-15T15:16:48.002 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-tui x86_64 1:1.47.4-1.el9 CentOS-BaseOS 248 k 2024-04-15T15:16:48.002 INFO:teuthology.orchestra.run.smithi105.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-04-15T15:16:48.002 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-04-15T15:16:48.002 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-04-15T15:16:48.004 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-04-15T15:16:48.004 INFO:teuthology.orchestra.run.smithi105.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-04-15T15:16:48.004 INFO:teuthology.orchestra.run.smithi105.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-04-15T15:16:48.004 INFO:teuthology.orchestra.run.smithi105.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-04-15T15:16:48.004 INFO:teuthology.orchestra.run.smithi105.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-04-15T15:16:48.004 INFO:teuthology.orchestra.run.smithi105.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-04-15T15:16:48.004 INFO:teuthology.orchestra.run.smithi105.stdout: librados2 x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 3.4 M 2024-04-15T15:16:48.004 INFO:teuthology.orchestra.run.smithi105.stdout: librbd1 x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 3.1 M 2024-04-15T15:16:48.004 INFO:teuthology.orchestra.run.smithi105.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-04-15T15:16:48.004 INFO:teuthology.orchestra.run.smithi105.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-04-15T15:16:48.004 INFO:teuthology.orchestra.run.smithi105.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-04-15T15:16:48.004 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-04-15T15:16:48.004 INFO:teuthology.orchestra.run.smithi105.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-04-15T15:16:48.005 INFO:teuthology.orchestra.run.smithi105.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-04-15T15:16:48.005 INFO:teuthology.orchestra.run.smithi105.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-04-15T15:16:48.005 INFO:teuthology.orchestra.run.smithi105.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-04-15T15:16:48.005 INFO:teuthology.orchestra.run.smithi105.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-04-15T15:16:48.005 INFO:teuthology.orchestra.run.smithi105.stdout:Installing dependencies: 2024-04-15T15:16:48.005 INFO:teuthology.orchestra.run.smithi105.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-15T15:16:48.005 INFO:teuthology.orchestra.run.smithi105.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-04-15T15:16:48.005 INFO:teuthology.orchestra.run.smithi105.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-04-15T15:16:48.005 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-common x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 21 M 2024-04-15T15:16:48.005 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2861.g9bf1c4ba.el8 ceph-noarch 34 k 2024-04-15T15:16:48.005 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mds x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 2.4 M 2024-04-15T15:16:48.005 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2861.g9bf1c4ba.el8 ceph-noarch 249 k 2024-04-15T15:16:48.005 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mon x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 4.7 M 2024-04-15T15:16:48.006 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-osd x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 17 M 2024-04-15T15:16:48.006 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2861.g9bf1c4ba.el8 ceph-noarch 16 k 2024-04-15T15:16:48.006 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-selinux x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 25 k 2024-04-15T15:16:48.006 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-04-15T15:16:48.006 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-04-15T15:16:48.006 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-04-15T15:16:48.007 INFO:teuthology.orchestra.run.smithi105.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-04-15T15:16:48.007 INFO:teuthology.orchestra.run.smithi105.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-15T15:16:48.007 INFO:teuthology.orchestra.run.smithi105.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-15T15:16:48.007 INFO:teuthology.orchestra.run.smithi105.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-15T15:16:48.007 INFO:teuthology.orchestra.run.smithi105.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-15T15:16:48.007 INFO:teuthology.orchestra.run.smithi105.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-15T15:16:48.007 INFO:teuthology.orchestra.run.smithi105.stdout: libcephsqlite x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 164 k 2024-04-15T15:16:48.007 INFO:teuthology.orchestra.run.smithi105.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-15T15:16:48.007 INFO:teuthology.orchestra.run.smithi105.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-15T15:16:48.007 INFO:teuthology.orchestra.run.smithi105.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-04-15T15:16:48.008 INFO:teuthology.orchestra.run.smithi105.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-15T15:16:48.008 INFO:teuthology.orchestra.run.smithi105.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-04-15T15:16:48.008 INFO:teuthology.orchestra.run.smithi105.stdout: libradosstriper1 x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 501 k 2024-04-15T15:16:48.008 INFO:teuthology.orchestra.run.smithi105.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-04-15T15:16:48.008 INFO:teuthology.orchestra.run.smithi105.stdout: librgw2 x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 5.2 M 2024-04-15T15:16:48.008 INFO:teuthology.orchestra.run.smithi105.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-15T15:16:48.008 INFO:teuthology.orchestra.run.smithi105.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-04-15T15:16:48.008 INFO:teuthology.orchestra.run.smithi105.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-04-15T15:16:48.008 INFO:teuthology.orchestra.run.smithi105.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-04-15T15:16:48.008 INFO:teuthology.orchestra.run.smithi105.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-15T15:16:48.008 INFO:teuthology.orchestra.run.smithi105.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-15T15:16:48.008 INFO:teuthology.orchestra.run.smithi105.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-04-15T15:16:48.008 INFO:teuthology.orchestra.run.smithi105.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-04-15T15:16:48.009 INFO:teuthology.orchestra.run.smithi105.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-04-15T15:16:48.009 INFO:teuthology.orchestra.run.smithi105.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-15T15:16:48.009 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-private-libs x86_64 13.14-1.el9 CentOS-AppStream 132 k 2024-04-15T15:16:48.009 INFO:teuthology.orchestra.run.smithi105.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-04-15T15:16:48.009 INFO:teuthology.orchestra.run.smithi105.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-15T15:16:48.009 INFO:teuthology.orchestra.run.smithi105.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-15T15:16:48.009 INFO:teuthology.orchestra.run.smithi105.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-04-15T15:16:48.009 INFO:teuthology.orchestra.run.smithi105.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-04-15T15:16:48.009 INFO:teuthology.orchestra.run.smithi105.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-15T15:16:48.009 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-15T15:16:48.009 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-argparse x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 45 k 2024-04-15T15:16:48.009 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-common x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 135 k 2024-04-15T15:16:48.009 INFO:teuthology.orchestra.run.smithi105.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-15T15:16:48.009 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-04-15T15:16:48.010 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-15T15:16:48.010 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-15T15:16:48.010 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-15T15:16:48.010 INFO:teuthology.orchestra.run.smithi105.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-15T15:16:48.010 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-04-15T15:16:48.010 INFO:teuthology.orchestra.run.smithi105.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-15T15:16:48.010 INFO:teuthology.orchestra.run.smithi105.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-15T15:16:48.010 INFO:teuthology.orchestra.run.smithi105.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-15T15:16:48.010 INFO:teuthology.orchestra.run.smithi105.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-15T15:16:48.011 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-15T15:16:48.011 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-15T15:16:48.011 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-15T15:16:48.011 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-15T15:16:48.011 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-15T15:16:48.011 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-15T15:16:48.011 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-04-15T15:16:48.011 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-04-15T15:16:48.011 INFO:teuthology.orchestra.run.smithi105.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-15T15:16:48.011 INFO:teuthology.orchestra.run.smithi105.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-15T15:16:48.011 INFO:teuthology.orchestra.run.smithi105.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-04-15T15:16:48.011 INFO:teuthology.orchestra.run.smithi105.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-04-15T15:16:48.011 INFO:teuthology.orchestra.run.smithi105.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-15T15:16:48.012 INFO:teuthology.orchestra.run.smithi105.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-15T15:16:48.012 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-04-15T15:16:48.012 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-04-15T15:16:48.012 INFO:teuthology.orchestra.run.smithi105.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-04-15T15:16:48.012 INFO:teuthology.orchestra.run.smithi105.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-04-15T15:16:48.012 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-15T15:16:48.012 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-04-15T15:16:48.012 INFO:teuthology.orchestra.run.smithi105.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-15T15:16:48.012 INFO:teuthology.orchestra.run.smithi105.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-04-15T15:16:48.012 INFO:teuthology.orchestra.run.smithi105.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-04-15T15:16:48.013 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-15T15:16:48.013 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-04-15T15:16:48.013 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-04-15T15:16:48.013 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-04-15T15:16:48.014 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-15T15:16:48.014 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-04-15T15:16:48.014 INFO:teuthology.orchestra.run.smithi105.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-15T15:16:48.014 INFO:teuthology.orchestra.run.smithi105.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-15T15:16:48.014 INFO:teuthology.orchestra.run.smithi105.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-04-15T15:16:48.014 INFO:teuthology.orchestra.run.smithi105.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-15T15:16:48.014 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-15T15:16:48.014 INFO:teuthology.orchestra.run.smithi105.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-04-15T15:16:48.015 INFO:teuthology.orchestra.run.smithi105.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-15T15:16:48.015 INFO:teuthology.orchestra.run.smithi105.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-04-15T15:16:48.015 INFO:teuthology.orchestra.run.smithi105.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-15T15:16:48.015 INFO:teuthology.orchestra.run.smithi105.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-04-15T15:16:48.015 INFO:teuthology.orchestra.run.smithi105.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-04-15T15:16:48.015 INFO:teuthology.orchestra.run.smithi105.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-15T15:16:48.015 INFO:teuthology.orchestra.run.smithi105.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-15T15:16:48.015 INFO:teuthology.orchestra.run.smithi105.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-15T15:16:48.015 INFO:teuthology.orchestra.run.smithi105.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-15T15:16:48.015 INFO:teuthology.orchestra.run.smithi105.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-04-15T15:16:48.015 INFO:teuthology.orchestra.run.smithi105.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-04-15T15:16:48.015 INFO:teuthology.orchestra.run.smithi105.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-15T15:16:48.015 INFO:teuthology.orchestra.run.smithi105.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-04-15T15:16:48.015 INFO:teuthology.orchestra.run.smithi105.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-15T15:16:48.015 INFO:teuthology.orchestra.run.smithi105.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-04-15T15:16:48.016 INFO:teuthology.orchestra.run.smithi105.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-04-15T15:16:48.016 INFO:teuthology.orchestra.run.smithi105.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-04-15T15:16:48.016 INFO:teuthology.orchestra.run.smithi105.stdout:Installing weak dependencies: 2024-04-15T15:16:48.016 INFO:teuthology.orchestra.run.smithi105.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-04-15T15:16:48.016 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:16:48.016 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction Summary 2024-04-15T15:16:48.016 INFO:teuthology.orchestra.run.smithi105.stdout:============================================================================================ 2024-04-15T15:16:48.016 INFO:teuthology.orchestra.run.smithi105.stdout:Install 150 Packages 2024-04-15T15:16:48.016 INFO:teuthology.orchestra.run.smithi105.stdout:Upgrade 24 Packages 2024-04-15T15:16:48.016 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:16:48.017 INFO:teuthology.orchestra.run.smithi105.stdout:Total download size: 228 M 2024-04-15T15:16:48.017 INFO:teuthology.orchestra.run.smithi105.stdout:Downloading Packages: 2024-04-15T15:16:48.769 INFO:teuthology.orchestra.run.smithi105.stdout:(1/174): ceph-19.0.0-2861.g9bf1c4ba.el8.x86_64. 35 kB/s | 6.6 kB 00:00 2024-04-15T15:16:48.978 INFO:teuthology.orchestra.run.smithi105.stdout:(2/174): ceph-fuse-19.0.0-2861.g9bf1c4ba.el8.x8 4.2 MB/s | 890 kB 00:00 2024-04-15T15:16:49.028 INFO:teuthology.orchestra.run.smithi105.stdout:(3/174): ceph-immutable-object-cache-19.0.0-286 2.8 MB/s | 145 kB 00:00 2024-04-15T15:16:49.286 INFO:teuthology.orchestra.run.smithi105.stdout:(4/174): ceph-mds-19.0.0-2861.g9bf1c4ba.el8.x86 9.1 MB/s | 2.4 MB 00:00 2024-04-15T15:16:49.445 INFO:teuthology.orchestra.run.smithi105.stdout:(5/174): ceph-base-19.0.0-2861.g9bf1c4ba.el8.x8 6.2 MB/s | 5.4 MB 00:00 2024-04-15T15:16:49.487 INFO:teuthology.orchestra.run.smithi105.stdout:(6/174): ceph-mgr-19.0.0-2861.g9bf1c4ba.el8.x86 7.5 MB/s | 1.5 MB 00:00 2024-04-15T15:16:49.946 INFO:teuthology.orchestra.run.smithi105.stdout:(7/174): ceph-mon-19.0.0-2861.g9bf1c4ba.el8.x86 9.4 MB/s | 4.7 MB 00:00 2024-04-15T15:16:50.223 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - CRB 3.9 MB/s | 6.2 MB 00:01 2024-04-15T15:16:51.087 INFO:teuthology.orchestra.run.smithi105.stdout:(8/174): ceph-osd-19.0.0-2861.g9bf1c4ba.el8.x86 11 MB/s | 17 MB 00:01 2024-04-15T15:16:51.196 INFO:teuthology.orchestra.run.smithi105.stdout:(9/174): ceph-radosgw-19.0.0-2861.g9bf1c4ba.el8 8.0 MB/s | 10 MB 00:01 2024-04-15T15:16:51.221 INFO:teuthology.orchestra.run.smithi105.stdout:(10/174): ceph-selinux-19.0.0-2861.g9bf1c4ba.el 188 kB/s | 25 kB 00:00 2024-04-15T15:16:51.272 INFO:teuthology.orchestra.run.smithi105.stdout:(11/174): cephfs-mirror-19.0.0-2861.g9bf1c4ba.e 4.4 MB/s | 224 kB 00:00 2024-04-15T15:16:51.330 INFO:teuthology.orchestra.run.smithi105.stdout:(12/174): cephfs-shell-19.0.0-2861.g9bf1c4ba.el 463 kB/s | 27 kB 00:00 2024-04-15T15:16:51.389 INFO:teuthology.orchestra.run.smithi105.stdout:(13/174): libcephfs-devel-19.0.0-2861.g9bf1c4ba 541 kB/s | 32 kB 00:00 2024-04-15T15:16:51.481 INFO:teuthology.orchestra.run.smithi105.stdout:(14/174): libcephfs2-19.0.0-2861.g9bf1c4ba.el8. 8.0 MB/s | 747 kB 00:00 2024-04-15T15:16:51.548 INFO:teuthology.orchestra.run.smithi105.stdout:(15/174): libcephsqlite-19.0.0-2861.g9bf1c4ba.e 2.4 MB/s | 164 kB 00:00 2024-04-15T15:16:51.765 INFO:teuthology.orchestra.run.smithi105.stdout:(16/174): ceph-common-19.0.0-2861.g9bf1c4ba.el8 6.5 MB/s | 21 MB 00:03 2024-04-15T15:16:51.790 INFO:teuthology.orchestra.run.smithi105.stdout:(17/174): librados-devel-19.0.0-2861.g9bf1c4ba. 524 kB/s | 127 kB 00:00 2024-04-15T15:16:51.957 INFO:teuthology.orchestra.run.smithi105.stdout:(18/174): libradosstriper1-19.0.0-2861.g9bf1c4b 2.5 MB/s | 501 kB 00:00 2024-04-15T15:16:52.074 INFO:teuthology.orchestra.run.smithi105.stdout:(19/174): python3-ceph-argparse-19.0.0-2861.g9b 387 kB/s | 45 kB 00:00 2024-04-15T15:16:52.166 INFO:teuthology.orchestra.run.smithi105.stdout:(20/174): librgw2-19.0.0-2861.g9bf1c4ba.el8.x86 14 MB/s | 5.2 MB 00:00 2024-04-15T15:16:52.191 INFO:teuthology.orchestra.run.smithi105.stdout:(21/174): python3-ceph-common-19.0.0-2861.g9bf1 1.1 MB/s | 135 kB 00:00 2024-04-15T15:16:52.225 INFO:teuthology.orchestra.run.smithi105.stdout:(22/174): python3-cephfs-19.0.0-2861.g9bf1c4ba. 2.7 MB/s | 162 kB 00:00 2024-04-15T15:16:52.275 INFO:teuthology.orchestra.run.smithi105.stdout:(23/174): python3-rados-19.0.0-2861.g9bf1c4ba.e 3.7 MB/s | 320 kB 00:00 2024-04-15T15:16:52.317 INFO:teuthology.orchestra.run.smithi105.stdout:(24/174): python3-rbd-19.0.0-2861.g9bf1c4ba.el8 3.2 MB/s | 297 kB 00:00 2024-04-15T15:16:52.343 INFO:teuthology.orchestra.run.smithi105.stdout:(25/174): python3-rgw-19.0.0-2861.g9bf1c4ba.el8 1.4 MB/s | 99 kB 00:00 2024-04-15T15:16:52.368 INFO:teuthology.orchestra.run.smithi105.stdout:(26/174): rbd-fuse-19.0.0-2861.g9bf1c4ba.el8.x8 1.7 MB/s | 85 kB 00:00 2024-04-15T15:16:52.446 INFO:teuthology.orchestra.run.smithi105.stdout:(27/174): rbd-nbd-19.0.0-2861.g9bf1c4ba.el8.x86 2.2 MB/s | 171 kB 00:00 2024-04-15T15:16:52.507 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - Extras packages 15 kB/s | 16 kB 00:01 2024-04-15T15:16:52.510 INFO:teuthology.orchestra.run.smithi105.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2861.g 513 kB/s | 34 kB 00:00 2024-04-15T15:16:52.585 INFO:teuthology.orchestra.run.smithi105.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2861.g9bf1c4b 1.9 MB/s | 142 kB 00:00 2024-04-15T15:16:52.744 INFO:teuthology.orchestra.run.smithi105.stdout:(30/174): rbd-mirror-19.0.0-2861.g9bf1c4ba.el8. 7.7 MB/s | 3.1 MB 00:00 2024-04-15T15:16:52.936 INFO:teuthology.orchestra.run.smithi105.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2861.g9bf1c 10 MB/s | 3.5 MB 00:00 2024-04-15T15:16:53.053 INFO:teuthology.orchestra.run.smithi105.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2861.g9b 2.1 MB/s | 249 kB 00:00 2024-04-15T15:16:53.111 INFO:teuthology.orchestra.run.smithi105.stdout:(33/174): ceph-mgr-rook-19.0.0-2861.g9bf1c4ba.e 847 kB/s | 49 kB 00:00 2024-04-15T15:16:53.178 INFO:teuthology.orchestra.run.smithi105.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2861.g9 246 kB/s | 16 kB 00:00 2024-04-15T15:16:53.279 INFO:teuthology.orchestra.run.smithi105.stdout:(35/174): ceph-volume-19.0.0-2861.g9bf1c4ba.el8 2.6 MB/s | 264 kB 00:00 2024-04-15T15:16:53.487 INFO:teuthology.orchestra.run.smithi105.stdout:(36/174): cephadm-19.0.0-2861.g9bf1c4ba.el8.noa 3.6 MB/s | 762 kB 00:00 2024-04-15T15:16:53.554 INFO:teuthology.orchestra.run.smithi105.stdout:(37/174): cephfs-top-19.0.0-2861.g9bf1c4ba.el8. 273 kB/s | 18 kB 00:00 2024-04-15T15:16:53.671 INFO:teuthology.orchestra.run.smithi105.stdout:(38/174): ceph-mgr-diskprediction-local-19.0.0- 8.0 MB/s | 7.4 MB 00:00 2024-04-15T15:16:53.771 INFO:teuthology.orchestra.run.smithi105.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 3.0 MB/s | 662 kB 00:00 2024-04-15T15:16:53.813 INFO:teuthology.orchestra.run.smithi105.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 6.4 MB/s | 920 kB 00:00 2024-04-15T15:16:53.839 INFO:teuthology.orchestra.run.smithi105.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 106 kB 00:00 2024-04-15T15:16:53.864 INFO:teuthology.orchestra.run.smithi105.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 6.0 MB/s | 307 kB 00:00 2024-04-15T15:16:53.881 INFO:teuthology.orchestra.run.smithi105.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 763 kB/s | 32 kB 00:00 2024-04-15T15:16:53.956 INFO:teuthology.orchestra.run.smithi105.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 32 MB/s | 3.0 MB 00:00 2024-04-15T15:16:53.973 INFO:teuthology.orchestra.run.smithi105.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 188 kB/s | 17 kB 00:00 2024-04-15T15:16:53.998 INFO:teuthology.orchestra.run.smithi105.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 241 kB/s | 10 kB 00:00 2024-04-15T15:16:54.024 INFO:teuthology.orchestra.run.smithi105.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 159 kB 00:00 2024-04-15T15:16:54.049 INFO:teuthology.orchestra.run.smithi105.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 882 kB/s | 44 kB 00:00 2024-04-15T15:16:54.074 INFO:teuthology.orchestra.run.smithi105.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 661 kB 00:00 2024-04-15T15:16:54.100 INFO:teuthology.orchestra.run.smithi105.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 11 MB/s | 578 kB 00:00 2024-04-15T15:16:54.125 INFO:teuthology.orchestra.run.smithi105.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 936 kB/s | 46 kB 00:00 2024-04-15T15:16:54.150 INFO:teuthology.orchestra.run.smithi105.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2024-04-15T15:16:54.176 INFO:teuthology.orchestra.run.smithi105.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-04-15T15:16:54.201 INFO:teuthology.orchestra.run.smithi105.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.2 MB/s | 217 kB 00:00 2024-04-15T15:16:54.226 INFO:teuthology.orchestra.run.smithi105.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 684 kB/s | 34 kB 00:00 2024-04-15T15:16:54.318 INFO:teuthology.orchestra.run.smithi105.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 41 MB/s | 4.7 MB 00:00 2024-04-15T15:16:54.352 INFO:teuthology.orchestra.run.smithi105.stdout:(57/174): postgresql-13.14-1.el9.x86_64.rpm 12 MB/s | 1.5 MB 00:00 2024-04-15T15:16:54.385 INFO:teuthology.orchestra.run.smithi105.stdout:(58/174): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 817 kB 00:00 2024-04-15T15:16:54.410 INFO:teuthology.orchestra.run.smithi105.stdout:(59/174): postgresql-private-libs-13.14-1.el9.x 2.3 MB/s | 132 kB 00:00 2024-04-15T15:16:54.536 INFO:teuthology.orchestra.run.smithi105.stdout:(60/174): postgresql-server-13.14-1.el9.x86_64. 38 MB/s | 5.7 MB 00:00 2024-04-15T15:16:54.569 INFO:teuthology.orchestra.run.smithi105.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 6.5 MB/s | 1.0 MB 00:00 2024-04-15T15:16:54.595 INFO:teuthology.orchestra.run.smithi105.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 78 kB 00:00 2024-04-15T15:16:55.037 INFO:teuthology.orchestra.run.smithi105.stdout:(63/174): ceph-test-19.0.0-2861.g9bf1c4ba.el8.x 13 MB/s | 49 MB 00:03 2024-04-15T15:16:55.121 INFO:teuthology.orchestra.run.smithi105.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 10 MB/s | 5.8 MB 00:00 2024-04-15T15:16:55.146 INFO:teuthology.orchestra.run.smithi105.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 44 kB/s | 24 kB 00:00 2024-04-15T15:16:55.171 INFO:teuthology.orchestra.run.smithi105.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 138 kB/s | 18 kB 00:00 2024-04-15T15:16:55.196 INFO:teuthology.orchestra.run.smithi105.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 636 kB/s | 32 kB 00:00 2024-04-15T15:16:55.221 INFO:teuthology.orchestra.run.smithi105.stdout:(68/174): python3-mako-1.1.4-6.el9.noarch.rpm 1.9 MB/s | 144 kB 00:00 2024-04-15T15:16:55.305 INFO:teuthology.orchestra.run.smithi105.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 36 MB/s | 4.8 MB 00:00 2024-04-15T15:16:55.330 INFO:teuthology.orchestra.run.smithi105.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.6 MB/s | 217 kB 00:00 2024-04-15T15:16:55.356 INFO:teuthology.orchestra.run.smithi105.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.3 MB/s | 174 kB 00:00 2024-04-15T15:16:55.381 INFO:teuthology.orchestra.run.smithi105.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 918 kB/s | 69 kB 00:00 2024-04-15T15:16:55.406 INFO:teuthology.orchestra.run.smithi105.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 825 kB/s | 41 kB 00:00 2024-04-15T15:16:55.431 INFO:teuthology.orchestra.run.smithi105.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 238 kB 00:00 2024-04-15T15:16:55.456 INFO:teuthology.orchestra.run.smithi105.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-04-15T15:16:55.482 INFO:teuthology.orchestra.run.smithi105.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 951 kB/s | 47 kB 00:00 2024-04-15T15:16:55.507 INFO:teuthology.orchestra.run.smithi105.stdout:(77/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 211 kB 00:00 2024-04-15T15:16:55.532 INFO:teuthology.orchestra.run.smithi105.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 569 kB/s | 43 kB 00:00 2024-04-15T15:16:55.824 INFO:teuthology.orchestra.run.smithi105.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 46 MB/s | 16 MB 00:00 2024-04-15T15:16:55.841 INFO:teuthology.orchestra.run.smithi105.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 134 kB/s | 41 kB 00:00 2024-04-15T15:16:55.858 INFO:teuthology.orchestra.run.smithi105.stdout:(81/174): python3-toml-0.10.2-6.el9.noarch.rpm 120 kB/s | 39 kB 00:00 2024-04-15T15:16:55.883 INFO:teuthology.orchestra.run.smithi105.stdout:(82/174): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 220 kB 00:00 2024-04-15T15:16:55.900 INFO:teuthology.orchestra.run.smithi105.stdout:(83/174): qatlib-service-24.02.0-1.el9.x86_64.r 596 kB/s | 35 kB 00:00 2024-04-15T15:16:55.917 INFO:teuthology.orchestra.run.smithi105.stdout:(84/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.4 MB/s | 46 kB 00:00 2024-04-15T15:16:55.951 INFO:teuthology.orchestra.run.smithi105.stdout:(85/174): socat-1.7.4.1-5.el9.x86_64.rpm 4.4 MB/s | 300 kB 00:00 2024-04-15T15:16:55.967 INFO:teuthology.orchestra.run.smithi105.stdout:(86/174): uuid-1.6.2-55.el9.x86_64.rpm 842 kB/s | 56 kB 00:00 2024-04-15T15:16:55.984 INFO:teuthology.orchestra.run.smithi105.stdout:(87/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-04-15T15:16:56.001 INFO:teuthology.orchestra.run.smithi105.stdout:(88/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-04-15T15:16:56.027 INFO:teuthology.orchestra.run.smithi105.stdout:(89/174): fuse-2.9.9-15.el9.x86_64.rpm 1.3 MB/s | 79 kB 00:00 2024-04-15T15:16:56.052 INFO:teuthology.orchestra.run.smithi105.stdout:(90/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.5 MB/s | 241 kB 00:00 2024-04-15T15:16:56.077 INFO:teuthology.orchestra.run.smithi105.stdout:(91/174): python3-ply-3.11-14.el9.noarch.rpm 2.0 MB/s | 103 kB 00:00 2024-04-15T15:16:56.119 INFO:teuthology.orchestra.run.smithi105.stdout:(92/174): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.1 MB 00:00 2024-04-15T15:16:56.136 INFO:teuthology.orchestra.run.smithi105.stdout:(93/174): python3-pycparser-2.20-6.el9.noarch.r 1.5 MB/s | 124 kB 00:00 2024-04-15T15:16:56.161 INFO:teuthology.orchestra.run.smithi105.stdout:(94/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.2 MB/s | 187 kB 00:00 2024-04-15T15:16:56.386 INFO:teuthology.orchestra.run.smithi105.stdout:(95/174): jq-1.6-16.el9.x86_64.rpm 749 kB/s | 187 kB 00:00 2024-04-15T15:16:56.436 INFO:teuthology.orchestra.run.smithi105.stdout:(96/174): libgfortran-11.4.1-3.el9.x86_64.rpm 2.6 MB/s | 803 kB 00:00 2024-04-15T15:16:56.459 INFO:teuthology.orchestra.run.smithi154.stdout:Extra Packages for Enterprise Linux 5.5 MB/s | 21 MB 00:03 2024-04-15T15:16:56.470 INFO:teuthology.orchestra.run.smithi105.stdout:(97/174): libquadmath-11.4.1-3.el9.x86_64.rpm 632 kB/s | 195 kB 00:00 2024-04-15T15:16:56.496 INFO:teuthology.orchestra.run.smithi105.stdout:(98/174): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-04-15T15:16:57.062 INFO:teuthology.orchestra.run.smithi105.stdout:(99/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 43 kB/s | 25 kB 00:00 2024-04-15T15:16:57.096 INFO:teuthology.orchestra.run.smithi105.stdout:(100/174): elfutils-libelf-devel-0.190-2.el9.x8 33 kB/s | 21 kB 00:00 2024-04-15T15:16:58.021 INFO:teuthology.orchestra.run.smithi105.stdout:(101/174): python3-devel-3.9.17-2.el9.x86_64.rp 256 kB/s | 245 kB 00:00 2024-04-15T15:16:58.055 INFO:teuthology.orchestra.run.smithi105.stdout:(102/174): python3-jinja2-2.11.3-5.el9.noarch.r 259 kB/s | 248 kB 00:00 2024-04-15T15:16:58.130 INFO:teuthology.orchestra.run.smithi105.stdout:(103/174): lua-devel-5.4.4-4.el9.x86_64.rpm 298 kB/s | 22 kB 00:00 2024-04-15T15:16:58.263 INFO:teuthology.orchestra.run.smithi105.stdout:(104/174): protobuf-compiler-3.14.0-13.el9.x86_ 6.3 MB/s | 863 kB 00:00 2024-04-15T15:16:58.731 INFO:teuthology.orchestra.run.smithi105.stdout:(105/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.1 MB/s | 548 kB 00:00 2024-04-15T15:16:58.806 INFO:teuthology.orchestra.run.smithi105.stdout:(106/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 119 kB/s | 94 kB 00:00 2024-04-15T15:16:58.831 INFO:teuthology.orchestra.run.smithi105.stdout:(107/174): gperftools-libs-2.9.1-2.el9.x86_64.r 3.0 MB/s | 309 kB 00:00 2024-04-15T15:16:59.056 INFO:teuthology.orchestra.run.smithi105.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 78 kB/s | 19 kB 00:00 2024-04-15T15:16:59.123 INFO:teuthology.orchestra.run.smithi105.stdout:(109/174): libarrow-9.0.0-10.el9.x86_64.rpm 15 MB/s | 4.4 MB 00:00 2024-04-15T15:16:59.157 INFO:teuthology.orchestra.run.smithi105.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 259 kB/s | 26 kB 00:00 2024-04-15T15:16:59.199 INFO:teuthology.orchestra.run.smithi105.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 647 kB/s | 49 kB 00:00 2024-04-15T15:16:59.291 INFO:teuthology.orchestra.run.smithi105.stdout:(112/174): luarocks-3.9.2-1.el9.noarch.rpm 1.6 MB/s | 151 kB 00:00 2024-04-15T15:16:59.317 INFO:teuthology.orchestra.run.smithi105.stdout:(113/174): libunwind-1.6.2-1.el9.x86_64.rpm 424 kB/s | 67 kB 00:00 2024-04-15T15:16:59.375 INFO:teuthology.orchestra.run.smithi105.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 9.8 MB/s | 838 kB 00:00 2024-04-15T15:16:59.450 INFO:teuthology.orchestra.run.smithi105.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 580 kB/s | 43 kB 00:00 2024-04-15T15:16:59.525 INFO:teuthology.orchestra.run.smithi105.stdout:(116/174): python3-cachetools-4.2.4-1.el9.noarc 429 kB/s | 32 kB 00:00 2024-04-15T15:16:59.550 INFO:teuthology.orchestra.run.smithi105.stdout:(117/174): python3-asyncssh-2.13.2-5.el9.noarch 2.3 MB/s | 548 kB 00:00 2024-04-15T15:16:59.601 INFO:teuthology.orchestra.run.smithi105.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 199 kB/s | 15 kB 00:00 2024-04-15T15:16:59.634 INFO:teuthology.orchestra.run.smithi105.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2024-04-15T15:16:59.676 INFO:teuthology.orchestra.run.smithi105.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 4.7 MB/s | 358 kB 00:00 2024-04-15T15:16:59.727 INFO:teuthology.orchestra.run.smithi105.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-04-15T15:16:59.752 INFO:teuthology.orchestra.run.smithi105.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 469 kB/s | 35 kB 00:00 2024-04-15T15:16:59.811 INFO:teuthology.orchestra.run.smithi105.stdout:(123/174): python3-google-auth-2.28.2-1.el9.noa 2.4 MB/s | 203 kB 00:00 2024-04-15T15:16:59.869 INFO:teuthology.orchestra.run.smithi105.stdout:(124/174): python3-grpcio-1.46.7-10.el9.x86_64. 17 MB/s | 2.0 MB 00:00 2024-04-15T15:16:59.894 INFO:teuthology.orchestra.run.smithi105.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 1.7 MB/s | 144 kB 00:00 2024-04-15T15:16:59.945 INFO:teuthology.orchestra.run.smithi105.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 142 kB/s | 11 kB 00:00 2024-04-15T15:16:59.962 INFO:teuthology.orchestra.run.smithi105.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 264 kB/s | 18 kB 00:00 2024-04-15T15:17:00.020 INFO:teuthology.orchestra.run.smithi105.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 308 kB/s | 23 kB 00:00 2024-04-15T15:17:00.046 INFO:teuthology.orchestra.run.smithi105.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 232 kB/s | 19 kB 00:00 2024-04-15T15:17:00.096 INFO:teuthology.orchestra.run.smithi105.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 261 kB/s | 20 kB 00:00 2024-04-15T15:17:00.171 INFO:teuthology.orchestra.run.smithi105.stdout:(131/174): python3-logutils-0.3.5-21.el9.noarch 620 kB/s | 46 kB 00:00 2024-04-15T15:17:00.205 INFO:teuthology.orchestra.run.smithi105.stdout:(132/174): python3-kubernetes-26.1.0-2.el9.noar 6.4 MB/s | 1.0 MB 00:00 2024-04-15T15:17:00.238 INFO:teuthology.orchestra.run.smithi105.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 1.2 MB/s | 79 kB 00:00 2024-04-15T15:17:00.280 INFO:teuthology.orchestra.run.smithi105.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 764 kB/s | 58 kB 00:00 2024-04-15T15:17:00.314 INFO:teuthology.orchestra.run.smithi105.stdout:(135/174): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-04-15T15:17:00.356 INFO:teuthology.orchestra.run.smithi105.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 218 kB/s | 16 kB 00:00 2024-04-15T15:17:00.390 INFO:teuthology.orchestra.run.smithi105.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-04-15T15:17:00.432 INFO:teuthology.orchestra.run.smithi105.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 375 kB/s | 28 kB 00:00 2024-04-15T15:17:00.465 INFO:teuthology.orchestra.run.smithi105.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 409 kB/s | 31 kB 00:00 2024-04-15T15:17:00.507 INFO:teuthology.orchestra.run.smithi105.stdout:(140/174): python3-routes-2.5.1-5.el9.noarch.rp 2.5 MB/s | 188 kB 00:00 2024-04-15T15:17:00.541 INFO:teuthology.orchestra.run.smithi105.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 785 kB/s | 59 kB 00:00 2024-04-15T15:17:00.583 INFO:teuthology.orchestra.run.smithi105.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 475 kB/s | 36 kB 00:00 2024-04-15T15:17:00.616 INFO:teuthology.orchestra.run.smithi105.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 683 kB/s | 51 kB 00:00 2024-04-15T15:17:00.658 INFO:teuthology.orchestra.run.smithi105.stdout:(144/174): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-04-15T15:17:00.692 INFO:teuthology.orchestra.run.smithi105.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-04-15T15:17:00.742 INFO:teuthology.orchestra.run.smithi105.stdout:(146/174): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-04-15T15:17:00.767 INFO:teuthology.orchestra.run.smithi105.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 265 kB/s | 20 kB 00:00 2024-04-15T15:17:00.818 INFO:teuthology.orchestra.run.smithi105.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-04-15T15:17:00.868 INFO:teuthology.orchestra.run.smithi105.stdout:(149/174): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-04-15T15:17:01.118 INFO:teuthology.orchestra.run.smithi105.stdout:(150/174): librados2-19.0.0-2861.g9bf1c4ba.el8. 11 MB/s | 3.4 MB 00:00 2024-04-15T15:17:01.285 INFO:teuthology.orchestra.run.smithi105.stdout:(151/174): NetworkManager-1.47.4-1.el9.x86_64.r 14 MB/s | 2.3 MB 00:00 2024-04-15T15:17:01.352 INFO:teuthology.orchestra.run.smithi105.stdout:(152/174): NetworkManager-libnm-1.47.4-1.el9.x8 27 MB/s | 1.8 MB 00:00 2024-04-15T15:17:01.394 INFO:teuthology.orchestra.run.smithi105.stdout:(153/174): NetworkManager-team-1.47.4-1.el9.x86 984 kB/s | 41 kB 00:00 2024-04-15T15:17:01.444 INFO:teuthology.orchestra.run.smithi105.stdout:(154/174): NetworkManager-tui-1.47.4-1.el9.x86_ 4.8 MB/s | 248 kB 00:00 2024-04-15T15:17:01.486 INFO:teuthology.orchestra.run.smithi105.stdout:(155/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.6 MB/s | 321 kB 00:00 2024-04-15T15:17:01.492 INFO:teuthology.orchestra.run.smithi154.stdout:lab-extras 41 kB/s | 1.7 kB 00:00 2024-04-15T15:17:01.545 INFO:teuthology.orchestra.run.smithi105.stdout:(156/174): librbd1-19.0.0-2861.g9bf1c4ba.el8.x8 4.6 MB/s | 3.1 MB 00:00 2024-04-15T15:17:01.703 INFO:teuthology.orchestra.run.smithi105.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 329 kB/s | 71 kB 00:00 2024-04-15T15:17:01.762 INFO:teuthology.orchestra.run.smithi105.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 168 kB/s | 36 kB 00:00 2024-04-15T15:17:01.854 INFO:teuthology.orchestra.run.smithi105.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 1.3 MB/s | 192 kB 00:00 2024-04-15T15:17:01.946 INFO:teuthology.orchestra.run.smithi105.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 251 kB/s | 23 kB 00:00 2024-04-15T15:17:01.971 INFO:teuthology.orchestra.run.smithi105.stdout:(161/174): elfutils-libs-0.190-2.el9.x86_64.rpm 1.2 MB/s | 256 kB 00:00 2024-04-15T15:17:02.030 INFO:teuthology.orchestra.run.smithi105.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.3 MB/s | 108 kB 00:00 2024-04-15T15:17:02.175 INFO:teuthology.orchestra.run.smithi105.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 759 kB/s | 155 kB 00:00 2024-04-15T15:17:02.222 INFO:teuthology.orchestra.run.smithi105.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 705 kB/s | 136 kB 00:00 2024-04-15T15:17:02.307 INFO:teuthology.orchestra.run.smithi105.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 490 kB/s | 64 kB 00:00 2024-04-15T15:17:02.365 INFO:teuthology.orchestra.run.smithi105.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 202 kB/s | 29 kB 00:00 2024-04-15T15:17:02.565 INFO:teuthology.orchestra.run.smithi105.stdout:(167/174): openssl-3.0.7-27.el9.x86_64.rpm 4.7 MB/s | 1.2 MB 00:00 2024-04-15T15:17:02.732 INFO:teuthology.orchestra.run.smithi105.stdout:(168/174): openssl-devel-3.0.7-27.el9.x86_64.rp 678 kB/s | 4.1 MB 00:06 2024-04-15T15:17:02.774 INFO:teuthology.orchestra.run.smithi105.stdout:(169/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 5.3 MB/s | 2.2 MB 00:00 2024-04-15T15:17:02.824 INFO:teuthology.orchestra.run.smithi105.stdout:(170/174): util-linux-2.37.4-18.el9.x86_64.rpm 8.8 MB/s | 2.3 MB 00:00 2024-04-15T15:17:02.866 INFO:teuthology.orchestra.run.smithi105.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 339 kB/s | 31 kB 00:00 2024-04-15T15:17:02.891 INFO:teuthology.orchestra.run.smithi105.stdout:(172/174): util-linux-core-2.37.4-18.el9.x86_64 2.9 MB/s | 465 kB 00:00 2024-04-15T15:17:02.958 INFO:teuthology.orchestra.run.smithi105.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 8.0 MB/s | 1.1 MB 00:00 2024-04-15T15:17:03.575 INFO:teuthology.orchestra.run.smithi105.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 105 kB/s | 74 kB 00:00 2024-04-15T15:17:03.584 INFO:teuthology.orchestra.run.smithi105.stdout:-------------------------------------------------------------------------------- 2024-04-15T15:17:03.584 INFO:teuthology.orchestra.run.smithi105.stdout:Total 15 MB/s | 228 MB 00:15 2024-04-15T15:17:03.956 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction check 2024-04-15T15:17:04.205 INFO:teuthology.orchestra.run.smithi154.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-15T15:17:04.205 INFO:teuthology.orchestra.run.smithi154.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-15T15:17:04.220 INFO:teuthology.orchestra.run.smithi154.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-04-15T15:17:04.221 INFO:teuthology.orchestra.run.smithi154.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-15T15:17:04.222 INFO:teuthology.orchestra.run.smithi154.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-04-15T15:17:04.223 INFO:teuthology.orchestra.run.smithi154.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-04-15T15:17:04.226 INFO:teuthology.orchestra.run.smithi154.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-15T15:17:04.228 INFO:teuthology.orchestra.run.smithi154.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-04-15T15:17:04.229 INFO:teuthology.orchestra.run.smithi154.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-04-15T15:17:04.231 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction check succeeded. 2024-04-15T15:17:04.231 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction test 2024-04-15T15:17:04.310 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2024-04-15T15:17:04.322 INFO:teuthology.orchestra.run.smithi154.stdout:============================================================================================ 2024-04-15T15:17:04.322 INFO:teuthology.orchestra.run.smithi154.stdout: Package Arch Version Repository Size 2024-04-15T15:17:04.322 INFO:teuthology.orchestra.run.smithi154.stdout:============================================================================================ 2024-04-15T15:17:04.322 INFO:teuthology.orchestra.run.smithi154.stdout:Installing: 2024-04-15T15:17:04.322 INFO:teuthology.orchestra.run.smithi154.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-04-15T15:17:04.322 INFO:teuthology.orchestra.run.smithi154.stdout: ceph x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 6.6 k 2024-04-15T15:17:04.322 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-base x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 5.4 M 2024-04-15T15:17:04.322 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-fuse x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 890 k 2024-04-15T15:17:04.323 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 145 k 2024-04-15T15:17:04.323 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 1.5 M 2024-04-15T15:17:04.323 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2861.g9bf1c4ba.el8 ceph-noarch 142 k 2024-04-15T15:17:04.323 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2861.g9bf1c4ba.el8 ceph-noarch 3.5 M 2024-04-15T15:17:04.323 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2861.g9bf1c4ba.el8 ceph-noarch 7.4 M 2024-04-15T15:17:04.323 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-rook noarch 2:19.0.0-2861.g9bf1c4ba.el8 ceph-noarch 49 k 2024-04-15T15:17:04.323 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-radosgw x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 10 M 2024-04-15T15:17:04.323 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-test x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 49 M 2024-04-15T15:17:04.323 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-volume noarch 2:19.0.0-2861.g9bf1c4ba.el8 ceph-noarch 264 k 2024-04-15T15:17:04.323 INFO:teuthology.orchestra.run.smithi154.stdout: cephadm noarch 2:19.0.0-2861.g9bf1c4ba.el8 ceph-noarch 762 k 2024-04-15T15:17:04.323 INFO:teuthology.orchestra.run.smithi154.stdout: cephfs-mirror x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 224 k 2024-04-15T15:17:04.323 INFO:teuthology.orchestra.run.smithi154.stdout: cephfs-shell x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 27 k 2024-04-15T15:17:04.323 INFO:teuthology.orchestra.run.smithi154.stdout: cephfs-top noarch 2:19.0.0-2861.g9bf1c4ba.el8 ceph-noarch 18 k 2024-04-15T15:17:04.323 INFO:teuthology.orchestra.run.smithi154.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-15T15:17:04.324 INFO:teuthology.orchestra.run.smithi154.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-04-15T15:17:04.324 INFO:teuthology.orchestra.run.smithi154.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-04-15T15:17:04.324 INFO:teuthology.orchestra.run.smithi154.stdout: libcephfs-devel x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 32 k 2024-04-15T15:17:04.324 INFO:teuthology.orchestra.run.smithi154.stdout: libcephfs2 x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 747 k 2024-04-15T15:17:04.324 INFO:teuthology.orchestra.run.smithi154.stdout: librados-devel x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 127 k 2024-04-15T15:17:04.324 INFO:teuthology.orchestra.run.smithi154.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-04-15T15:17:04.324 INFO:teuthology.orchestra.run.smithi154.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-15T15:17:04.324 INFO:teuthology.orchestra.run.smithi154.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-15T15:17:04.324 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql x86_64 13.14-1.el9 CentOS-AppStream 1.5 M 2024-04-15T15:17:04.324 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-contrib x86_64 13.14-1.el9 CentOS-AppStream 817 k 2024-04-15T15:17:04.324 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-server x86_64 13.14-1.el9 CentOS-AppStream 5.7 M 2024-04-15T15:17:04.324 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cephfs x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 162 k 2024-04-15T15:17:04.324 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rados x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 320 k 2024-04-15T15:17:04.325 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rbd x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 297 k 2024-04-15T15:17:04.325 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rgw x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 99 k 2024-04-15T15:17:04.325 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-fuse x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 85 k 2024-04-15T15:17:04.325 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-mirror x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 3.1 M 2024-04-15T15:17:04.325 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-nbd x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 171 k 2024-04-15T15:17:04.325 INFO:teuthology.orchestra.run.smithi154.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-15T15:17:04.325 INFO:teuthology.orchestra.run.smithi154.stdout:Upgrading: 2024-04-15T15:17:04.325 INFO:teuthology.orchestra.run.smithi154.stdout: NetworkManager x86_64 1:1.47.4-1.el9 CentOS-BaseOS 2.3 M 2024-04-15T15:17:04.325 INFO:teuthology.orchestra.run.smithi154.stdout: NetworkManager-libnm x86_64 1:1.47.4-1.el9 CentOS-BaseOS 1.8 M 2024-04-15T15:17:04.325 INFO:teuthology.orchestra.run.smithi154.stdout: NetworkManager-team x86_64 1:1.47.4-1.el9 CentOS-BaseOS 41 k 2024-04-15T15:17:04.325 INFO:teuthology.orchestra.run.smithi154.stdout: NetworkManager-tui x86_64 1:1.47.4-1.el9 CentOS-BaseOS 248 k 2024-04-15T15:17:04.325 INFO:teuthology.orchestra.run.smithi154.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-04-15T15:17:04.325 INFO:teuthology.orchestra.run.smithi154.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-04-15T15:17:04.326 INFO:teuthology.orchestra.run.smithi154.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-04-15T15:17:04.326 INFO:teuthology.orchestra.run.smithi154.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-04-15T15:17:04.326 INFO:teuthology.orchestra.run.smithi154.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-04-15T15:17:04.326 INFO:teuthology.orchestra.run.smithi154.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-04-15T15:17:04.326 INFO:teuthology.orchestra.run.smithi154.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-04-15T15:17:04.326 INFO:teuthology.orchestra.run.smithi154.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-04-15T15:17:04.326 INFO:teuthology.orchestra.run.smithi154.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-04-15T15:17:04.327 INFO:teuthology.orchestra.run.smithi154.stdout: librados2 x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 3.4 M 2024-04-15T15:17:04.327 INFO:teuthology.orchestra.run.smithi154.stdout: librbd1 x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 3.1 M 2024-04-15T15:17:04.327 INFO:teuthology.orchestra.run.smithi154.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-04-15T15:17:04.327 INFO:teuthology.orchestra.run.smithi154.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-04-15T15:17:04.327 INFO:teuthology.orchestra.run.smithi154.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-04-15T15:17:04.327 INFO:teuthology.orchestra.run.smithi154.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-04-15T15:17:04.329 INFO:teuthology.orchestra.run.smithi154.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-04-15T15:17:04.329 INFO:teuthology.orchestra.run.smithi154.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-04-15T15:17:04.329 INFO:teuthology.orchestra.run.smithi154.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-04-15T15:17:04.329 INFO:teuthology.orchestra.run.smithi154.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-04-15T15:17:04.330 INFO:teuthology.orchestra.run.smithi154.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-04-15T15:17:04.330 INFO:teuthology.orchestra.run.smithi154.stdout:Installing dependencies: 2024-04-15T15:17:04.330 INFO:teuthology.orchestra.run.smithi154.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-15T15:17:04.330 INFO:teuthology.orchestra.run.smithi154.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-04-15T15:17:04.330 INFO:teuthology.orchestra.run.smithi154.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-04-15T15:17:04.330 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-common x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 21 M 2024-04-15T15:17:04.330 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2861.g9bf1c4ba.el8 ceph-noarch 34 k 2024-04-15T15:17:04.330 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mds x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 2.4 M 2024-04-15T15:17:04.330 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2861.g9bf1c4ba.el8 ceph-noarch 249 k 2024-04-15T15:17:04.330 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mon x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 4.7 M 2024-04-15T15:17:04.330 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-osd x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 17 M 2024-04-15T15:17:04.330 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2861.g9bf1c4ba.el8 ceph-noarch 16 k 2024-04-15T15:17:04.331 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-selinux x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 25 k 2024-04-15T15:17:04.331 INFO:teuthology.orchestra.run.smithi154.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-04-15T15:17:04.331 INFO:teuthology.orchestra.run.smithi154.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-04-15T15:17:04.331 INFO:teuthology.orchestra.run.smithi154.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-04-15T15:17:04.331 INFO:teuthology.orchestra.run.smithi154.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-04-15T15:17:04.331 INFO:teuthology.orchestra.run.smithi154.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-15T15:17:04.331 INFO:teuthology.orchestra.run.smithi154.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-15T15:17:04.331 INFO:teuthology.orchestra.run.smithi154.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-15T15:17:04.331 INFO:teuthology.orchestra.run.smithi154.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-15T15:17:04.331 INFO:teuthology.orchestra.run.smithi154.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-15T15:17:04.331 INFO:teuthology.orchestra.run.smithi154.stdout: libcephsqlite x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 164 k 2024-04-15T15:17:04.331 INFO:teuthology.orchestra.run.smithi154.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-15T15:17:04.331 INFO:teuthology.orchestra.run.smithi154.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-15T15:17:04.332 INFO:teuthology.orchestra.run.smithi154.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-04-15T15:17:04.332 INFO:teuthology.orchestra.run.smithi154.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-15T15:17:04.332 INFO:teuthology.orchestra.run.smithi154.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-04-15T15:17:04.332 INFO:teuthology.orchestra.run.smithi154.stdout: libradosstriper1 x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 501 k 2024-04-15T15:17:04.332 INFO:teuthology.orchestra.run.smithi154.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-04-15T15:17:04.332 INFO:teuthology.orchestra.run.smithi154.stdout: librgw2 x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 5.2 M 2024-04-15T15:17:04.332 INFO:teuthology.orchestra.run.smithi154.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-15T15:17:04.332 INFO:teuthology.orchestra.run.smithi154.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-04-15T15:17:04.332 INFO:teuthology.orchestra.run.smithi154.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-04-15T15:17:04.332 INFO:teuthology.orchestra.run.smithi154.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-04-15T15:17:04.332 INFO:teuthology.orchestra.run.smithi154.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-15T15:17:04.332 INFO:teuthology.orchestra.run.smithi154.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-15T15:17:04.332 INFO:teuthology.orchestra.run.smithi154.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-04-15T15:17:04.333 INFO:teuthology.orchestra.run.smithi154.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-04-15T15:17:04.333 INFO:teuthology.orchestra.run.smithi154.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-04-15T15:17:04.333 INFO:teuthology.orchestra.run.smithi154.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-15T15:17:04.333 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-private-libs x86_64 13.14-1.el9 CentOS-AppStream 132 k 2024-04-15T15:17:04.333 INFO:teuthology.orchestra.run.smithi154.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-04-15T15:17:04.333 INFO:teuthology.orchestra.run.smithi154.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-15T15:17:04.333 INFO:teuthology.orchestra.run.smithi154.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-15T15:17:04.333 INFO:teuthology.orchestra.run.smithi154.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-04-15T15:17:04.333 INFO:teuthology.orchestra.run.smithi154.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-04-15T15:17:04.333 INFO:teuthology.orchestra.run.smithi154.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-15T15:17:04.333 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-15T15:17:04.333 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ceph-argparse x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 45 k 2024-04-15T15:17:04.334 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ceph-common x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 135 k 2024-04-15T15:17:04.334 INFO:teuthology.orchestra.run.smithi154.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-15T15:17:04.334 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-04-15T15:17:04.334 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-15T15:17:04.334 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-15T15:17:04.334 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-15T15:17:04.334 INFO:teuthology.orchestra.run.smithi154.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-15T15:17:04.334 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-04-15T15:17:04.334 INFO:teuthology.orchestra.run.smithi154.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-15T15:17:04.334 INFO:teuthology.orchestra.run.smithi154.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-15T15:17:04.334 INFO:teuthology.orchestra.run.smithi154.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-15T15:17:04.334 INFO:teuthology.orchestra.run.smithi154.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-15T15:17:04.334 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-15T15:17:04.335 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-15T15:17:04.335 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-15T15:17:04.335 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-15T15:17:04.335 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-15T15:17:04.335 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-15T15:17:04.335 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-04-15T15:17:04.335 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-04-15T15:17:04.335 INFO:teuthology.orchestra.run.smithi154.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-15T15:17:04.335 INFO:teuthology.orchestra.run.smithi154.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-15T15:17:04.335 INFO:teuthology.orchestra.run.smithi154.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-04-15T15:17:04.335 INFO:teuthology.orchestra.run.smithi154.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-04-15T15:17:04.335 INFO:teuthology.orchestra.run.smithi154.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-15T15:17:04.335 INFO:teuthology.orchestra.run.smithi154.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-15T15:17:04.336 INFO:teuthology.orchestra.run.smithi154.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-04-15T15:17:04.336 INFO:teuthology.orchestra.run.smithi154.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-04-15T15:17:04.336 INFO:teuthology.orchestra.run.smithi154.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-04-15T15:17:04.336 INFO:teuthology.orchestra.run.smithi154.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-04-15T15:17:04.336 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-15T15:17:04.336 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-04-15T15:17:04.336 INFO:teuthology.orchestra.run.smithi154.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-15T15:17:04.336 INFO:teuthology.orchestra.run.smithi154.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-04-15T15:17:04.336 INFO:teuthology.orchestra.run.smithi154.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-04-15T15:17:04.336 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-15T15:17:04.337 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-04-15T15:17:04.337 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-04-15T15:17:04.337 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-04-15T15:17:04.337 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-15T15:17:04.337 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-04-15T15:17:04.337 INFO:teuthology.orchestra.run.smithi154.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-15T15:17:04.337 INFO:teuthology.orchestra.run.smithi154.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-15T15:17:04.337 INFO:teuthology.orchestra.run.smithi154.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-04-15T15:17:04.337 INFO:teuthology.orchestra.run.smithi154.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-15T15:17:04.337 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-15T15:17:04.337 INFO:teuthology.orchestra.run.smithi154.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-04-15T15:17:04.338 INFO:teuthology.orchestra.run.smithi154.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-15T15:17:04.338 INFO:teuthology.orchestra.run.smithi154.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-04-15T15:17:04.338 INFO:teuthology.orchestra.run.smithi154.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-15T15:17:04.338 INFO:teuthology.orchestra.run.smithi154.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-04-15T15:17:04.338 INFO:teuthology.orchestra.run.smithi154.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-04-15T15:17:04.338 INFO:teuthology.orchestra.run.smithi154.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-15T15:17:04.338 INFO:teuthology.orchestra.run.smithi154.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-15T15:17:04.338 INFO:teuthology.orchestra.run.smithi154.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-15T15:17:04.338 INFO:teuthology.orchestra.run.smithi154.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-15T15:17:04.338 INFO:teuthology.orchestra.run.smithi154.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-04-15T15:17:04.339 INFO:teuthology.orchestra.run.smithi154.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-04-15T15:17:04.339 INFO:teuthology.orchestra.run.smithi154.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-15T15:17:04.339 INFO:teuthology.orchestra.run.smithi154.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-04-15T15:17:04.339 INFO:teuthology.orchestra.run.smithi154.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-15T15:17:04.339 INFO:teuthology.orchestra.run.smithi154.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-04-15T15:17:04.339 INFO:teuthology.orchestra.run.smithi154.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-04-15T15:17:04.339 INFO:teuthology.orchestra.run.smithi154.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-04-15T15:17:04.339 INFO:teuthology.orchestra.run.smithi154.stdout:Installing weak dependencies: 2024-04-15T15:17:04.339 INFO:teuthology.orchestra.run.smithi154.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-04-15T15:17:04.339 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-15T15:17:04.339 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction Summary 2024-04-15T15:17:04.339 INFO:teuthology.orchestra.run.smithi154.stdout:============================================================================================ 2024-04-15T15:17:04.339 INFO:teuthology.orchestra.run.smithi154.stdout:Install 150 Packages 2024-04-15T15:17:04.340 INFO:teuthology.orchestra.run.smithi154.stdout:Upgrade 24 Packages 2024-04-15T15:17:04.340 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-15T15:17:04.340 INFO:teuthology.orchestra.run.smithi154.stdout:Total download size: 228 M 2024-04-15T15:17:04.340 INFO:teuthology.orchestra.run.smithi154.stdout:Downloading Packages: 2024-04-15T15:17:05.051 INFO:teuthology.orchestra.run.smithi154.stdout:(1/174): ceph-19.0.0-2861.g9bf1c4ba.el8.x86_64. 39 kB/s | 6.6 kB 00:00 2024-04-15T15:17:05.243 INFO:teuthology.orchestra.run.smithi154.stdout:(2/174): ceph-fuse-19.0.0-2861.g9bf1c4ba.el8.x8 4.5 MB/s | 890 kB 00:00 2024-04-15T15:17:05.302 INFO:teuthology.orchestra.run.smithi154.stdout:(3/174): ceph-immutable-object-cache-19.0.0-286 2.4 MB/s | 145 kB 00:00 2024-04-15T15:17:05.538 INFO:teuthology.orchestra.run.smithi154.stdout:(4/174): ceph-base-19.0.0-2861.g9bf1c4ba.el8.x8 8.2 MB/s | 5.4 MB 00:00 2024-04-15T15:17:05.586 INFO:teuthology.orchestra.run.smithi154.stdout:(5/174): ceph-mds-19.0.0-2861.g9bf1c4ba.el8.x86 8.3 MB/s | 2.4 MB 00:00 2024-04-15T15:17:05.661 INFO:teuthology.orchestra.run.smithi154.stdout:(6/174): ceph-mgr-19.0.0-2861.g9bf1c4ba.el8.x86 12 MB/s | 1.5 MB 00:00 2024-04-15T15:17:06.053 INFO:teuthology.orchestra.run.smithi154.stdout:(7/174): ceph-mon-19.0.0-2861.g9bf1c4ba.el8.x86 10 MB/s | 4.7 MB 00:00 2024-04-15T15:17:06.262 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction test succeeded. 2024-04-15T15:17:06.263 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction 2024-04-15T15:17:06.995 INFO:teuthology.orchestra.run.smithi154.stdout:(8/174): ceph-osd-19.0.0-2861.g9bf1c4ba.el8.x86 13 MB/s | 17 MB 00:01 2024-04-15T15:17:07.112 INFO:teuthology.orchestra.run.smithi154.stdout:(9/174): ceph-radosgw-19.0.0-2861.g9bf1c4ba.el8 9.4 MB/s | 10 MB 00:01 2024-04-15T15:17:07.187 INFO:teuthology.orchestra.run.smithi154.stdout:(10/174): ceph-selinux-19.0.0-2861.g9bf1c4ba.el 131 kB/s | 25 kB 00:00 2024-04-15T15:17:07.262 INFO:teuthology.orchestra.run.smithi154.stdout:(11/174): cephfs-mirror-19.0.0-2861.g9bf1c4ba.e 2.9 MB/s | 224 kB 00:00 2024-04-15T15:17:07.338 INFO:teuthology.orchestra.run.smithi154.stdout:(12/174): cephfs-shell-19.0.0-2861.g9bf1c4ba.el 359 kB/s | 27 kB 00:00 2024-04-15T15:17:07.405 INFO:teuthology.orchestra.run.smithi154.stdout:(13/174): libcephfs-devel-19.0.0-2861.g9bf1c4ba 476 kB/s | 32 kB 00:00 2024-04-15T15:17:07.497 INFO:teuthology.orchestra.run.smithi154.stdout:(14/174): libcephfs2-19.0.0-2861.g9bf1c4ba.el8. 8.0 MB/s | 747 kB 00:00 2024-04-15T15:17:07.572 INFO:teuthology.orchestra.run.smithi154.stdout:(15/174): libcephsqlite-19.0.0-2861.g9bf1c4ba.e 2.1 MB/s | 164 kB 00:00 2024-04-15T15:17:07.647 INFO:teuthology.orchestra.run.smithi154.stdout:(16/174): librados-devel-19.0.0-2861.g9bf1c4ba. 1.7 MB/s | 127 kB 00:00 2024-04-15T15:17:07.748 INFO:teuthology.orchestra.run.smithi154.stdout:(17/174): libradosstriper1-19.0.0-2861.g9bf1c4b 4.9 MB/s | 501 kB 00:00 2024-04-15T15:17:08.365 INFO:teuthology.orchestra.run.smithi154.stdout:(18/174): librgw2-19.0.0-2861.g9bf1c4ba.el8.x86 8.5 MB/s | 5.2 MB 00:00 2024-04-15T15:17:08.423 INFO:teuthology.orchestra.run.smithi154.stdout:(19/174): python3-ceph-argparse-19.0.0-2861.g9b 776 kB/s | 45 kB 00:00 2024-04-15T15:17:08.632 INFO:teuthology.orchestra.run.smithi154.stdout:(20/174): ceph-common-19.0.0-2861.g9bf1c4ba.el8 5.6 MB/s | 21 MB 00:03 2024-04-15T15:17:08.666 INFO:teuthology.orchestra.run.smithi154.stdout:(21/174): python3-ceph-common-19.0.0-2861.g9bf1 557 kB/s | 135 kB 00:00 2024-04-15T15:17:08.699 INFO:teuthology.orchestra.run.smithi154.stdout:(22/174): python3-cephfs-19.0.0-2861.g9bf1c4ba. 2.4 MB/s | 162 kB 00:00 2024-04-15T15:17:08.741 INFO:teuthology.orchestra.run.smithi154.stdout:(23/174): python3-rados-19.0.0-2861.g9bf1c4ba.e 4.1 MB/s | 320 kB 00:00 2024-04-15T15:17:08.783 INFO:teuthology.orchestra.run.smithi154.stdout:(24/174): python3-rbd-19.0.0-2861.g9bf1c4ba.el8 3.5 MB/s | 297 kB 00:00 2024-04-15T15:17:08.808 INFO:teuthology.orchestra.run.smithi154.stdout:(25/174): python3-rgw-19.0.0-2861.g9bf1c4ba.el8 1.4 MB/s | 99 kB 00:00 2024-04-15T15:17:08.859 INFO:teuthology.orchestra.run.smithi154.stdout:(26/174): rbd-fuse-19.0.0-2861.g9bf1c4ba.el8.x8 1.1 MB/s | 85 kB 00:00 2024-04-15T15:17:08.959 INFO:teuthology.orchestra.run.smithi154.stdout:(27/174): rbd-nbd-19.0.0-2861.g9bf1c4ba.el8.x86 1.7 MB/s | 171 kB 00:00 2024-04-15T15:17:09.035 INFO:teuthology.orchestra.run.smithi154.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2861.g 456 kB/s | 34 kB 00:00 2024-04-15T15:17:09.093 INFO:teuthology.orchestra.run.smithi154.stdout:(29/174): rbd-mirror-19.0.0-2861.g9bf1c4ba.el8. 11 MB/s | 3.1 MB 00:00 2024-04-15T15:17:09.118 INFO:teuthology.orchestra.run.smithi154.stdout:(30/174): ceph-mgr-cephadm-19.0.0-2861.g9bf1c4b 1.7 MB/s | 142 kB 00:00 2024-04-15T15:17:09.502 INFO:teuthology.orchestra.run.smithi154.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2861.g9bf1c 8.7 MB/s | 3.5 MB 00:00 2024-04-15T15:17:09.586 INFO:teuthology.orchestra.run.smithi154.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2861.g9b 2.9 MB/s | 249 kB 00:00 2024-04-15T15:17:09.661 INFO:teuthology.orchestra.run.smithi154.stdout:(33/174): ceph-mgr-rook-19.0.0-2861.g9bf1c4ba.e 658 kB/s | 49 kB 00:00 2024-04-15T15:17:09.728 INFO:teuthology.orchestra.run.smithi154.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2861.g9 247 kB/s | 16 kB 00:00 2024-04-15T15:17:09.811 INFO:teuthology.orchestra.run.smithi154.stdout:(35/174): ceph-mgr-diskprediction-local-19.0.0- 11 MB/s | 7.4 MB 00:00 2024-04-15T15:17:09.837 INFO:teuthology.orchestra.run.smithi154.stdout:(36/174): ceph-volume-19.0.0-2861.g9bf1c4ba.el8 2.4 MB/s | 264 kB 00:00 2024-04-15T15:17:09.921 INFO:teuthology.orchestra.run.smithi154.stdout:(37/174): cephadm-19.0.0-2861.g9bf1c4ba.el8.noa 6.9 MB/s | 762 kB 00:00 2024-04-15T15:17:09.946 INFO:teuthology.orchestra.run.smithi154.stdout:(38/174): cephfs-top-19.0.0-2861.g9bf1c4ba.el8. 167 kB/s | 18 kB 00:00 2024-04-15T15:17:10.105 INFO:teuthology.orchestra.run.smithi154.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 3.5 MB/s | 662 kB 00:00 2024-04-15T15:17:10.155 INFO:teuthology.orchestra.run.smithi154.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 4.3 MB/s | 920 kB 00:00 2024-04-15T15:17:10.180 INFO:teuthology.orchestra.run.smithi154.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.4 MB/s | 106 kB 00:00 2024-04-15T15:17:10.206 INFO:teuthology.orchestra.run.smithi154.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 6.0 MB/s | 307 kB 00:00 2024-04-15T15:17:10.231 INFO:teuthology.orchestra.run.smithi154.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 638 kB/s | 32 kB 00:00 2024-04-15T15:17:10.323 INFO:teuthology.orchestra.run.smithi154.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 26 MB/s | 3.0 MB 00:00 2024-04-15T15:17:10.348 INFO:teuthology.orchestra.run.smithi154.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 148 kB/s | 17 kB 00:00 2024-04-15T15:17:10.365 INFO:teuthology.orchestra.run.smithi154.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 240 kB/s | 10 kB 00:00 2024-04-15T15:17:10.391 INFO:teuthology.orchestra.run.smithi154.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.7 MB/s | 159 kB 00:00 2024-04-15T15:17:10.407 INFO:teuthology.orchestra.run.smithi154.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.0 MB/s | 44 kB 00:00 2024-04-15T15:17:10.441 INFO:teuthology.orchestra.run.smithi154.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 661 kB 00:00 2024-04-15T15:17:10.466 INFO:teuthology.orchestra.run.smithi154.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 9.8 MB/s | 578 kB 00:00 2024-04-15T15:17:10.483 INFO:teuthology.orchestra.run.smithi154.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 46 kB 00:00 2024-04-15T15:17:10.509 INFO:teuthology.orchestra.run.smithi154.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.6 MB/s | 282 kB 00:00 2024-04-15T15:17:10.542 INFO:teuthology.orchestra.run.smithi154.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 187 kB 00:00 2024-04-15T15:17:10.576 INFO:teuthology.orchestra.run.smithi154.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.2 MB/s | 217 kB 00:00 2024-04-15T15:17:10.593 INFO:teuthology.orchestra.run.smithi154.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 687 kB/s | 34 kB 00:00 2024-04-15T15:17:10.727 INFO:teuthology.orchestra.run.smithi154.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 32 MB/s | 4.7 MB 00:00 2024-04-15T15:17:10.769 INFO:teuthology.orchestra.run.smithi154.stdout:(57/174): postgresql-13.14-1.el9.x86_64.rpm 8.7 MB/s | 1.5 MB 00:00 2024-04-15T15:17:10.794 INFO:teuthology.orchestra.run.smithi154.stdout:(58/174): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 817 kB 00:00 2024-04-15T15:17:10.819 INFO:teuthology.orchestra.run.smithi154.stdout:(59/174): postgresql-private-libs-13.14-1.el9.x 2.6 MB/s | 132 kB 00:00 2024-04-15T15:17:10.936 INFO:teuthology.orchestra.run.smithi154.stdout:(60/174): postgresql-server-13.14-1.el9.x86_64. 40 MB/s | 5.7 MB 00:00 2024-04-15T15:17:10.970 INFO:teuthology.orchestra.run.smithi154.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 6.8 MB/s | 1.0 MB 00:00 2024-04-15T15:17:10.995 INFO:teuthology.orchestra.run.smithi154.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 78 kB 00:00 2024-04-15T15:17:11.429 INFO:teuthology.orchestra.run.smithi154.stdout:(63/174): ceph-test-19.0.0-2861.g9bf1c4ba.el8.x 11 MB/s | 49 MB 00:04 2024-04-15T15:17:11.513 INFO:teuthology.orchestra.run.smithi154.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 11 MB/s | 5.8 MB 00:00 2024-04-15T15:17:11.538 INFO:teuthology.orchestra.run.smithi154.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 45 kB/s | 24 kB 00:00 2024-04-15T15:17:11.563 INFO:teuthology.orchestra.run.smithi154.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 138 kB/s | 18 kB 00:00 2024-04-15T15:17:11.588 INFO:teuthology.orchestra.run.smithi154.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.8 MB/s | 144 kB 00:00 2024-04-15T15:17:11.613 INFO:teuthology.orchestra.run.smithi154.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 421 kB/s | 32 kB 00:00 2024-04-15T15:17:11.706 INFO:teuthology.orchestra.run.smithi154.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 34 MB/s | 4.8 MB 00:00 2024-04-15T15:17:11.739 INFO:teuthology.orchestra.run.smithi154.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.7 MB/s | 217 kB 00:00 2024-04-15T15:17:11.765 INFO:teuthology.orchestra.run.smithi154.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.2 MB/s | 174 kB 00:00 2024-04-15T15:17:11.790 INFO:teuthology.orchestra.run.smithi154.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 831 kB/s | 69 kB 00:00 2024-04-15T15:17:11.807 INFO:teuthology.orchestra.run.smithi154.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 990 kB/s | 41 kB 00:00 2024-04-15T15:17:11.832 INFO:teuthology.orchestra.run.smithi154.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 238 kB 00:00 2024-04-15T15:17:11.857 INFO:teuthology.orchestra.run.smithi154.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 132 kB 00:00 2024-04-15T15:17:11.874 INFO:teuthology.orchestra.run.smithi154.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 1.1 MB/s | 47 kB 00:00 2024-04-15T15:17:11.901 INFO:teuthology.orchestra.run.smithi154.stdout:(77/174): python3-pyasn1-modules-0.4.8-6.el9.no 3.1 MB/s | 211 kB 00:00 2024-04-15T15:17:11.924 INFO:teuthology.orchestra.run.smithi154.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 641 kB/s | 43 kB 00:00 2024-04-15T15:17:12.275 INFO:teuthology.orchestra.run.smithi154.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 39 MB/s | 16 MB 00:00 2024-04-15T15:17:12.300 INFO:teuthology.orchestra.run.smithi154.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 110 kB/s | 41 kB 00:00 2024-04-15T15:17:12.325 INFO:teuthology.orchestra.run.smithi154.stdout:(81/174): python3-toml-0.10.2-6.el9.noarch.rpm 97 kB/s | 39 kB 00:00 2024-04-15T15:17:12.350 INFO:teuthology.orchestra.run.smithi154.stdout:(82/174): qatlib-service-24.02.0-1.el9.x86_64.r 701 kB/s | 35 kB 00:00 2024-04-15T15:17:12.376 INFO:teuthology.orchestra.run.smithi154.stdout:(83/174): qatlib-24.02.0-1.el9.x86_64.rpm 2.9 MB/s | 220 kB 00:00 2024-04-15T15:17:12.393 INFO:teuthology.orchestra.run.smithi154.stdout:(84/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 692 kB/s | 46 kB 00:00 2024-04-15T15:17:12.418 INFO:teuthology.orchestra.run.smithi154.stdout:(85/174): socat-1.7.4.1-5.el9.x86_64.rpm 4.4 MB/s | 300 kB 00:00 2024-04-15T15:17:12.443 INFO:teuthology.orchestra.run.smithi154.stdout:(86/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-04-15T15:17:12.468 INFO:teuthology.orchestra.run.smithi154.stdout:(87/174): uuid-1.6.2-55.el9.x86_64.rpm 753 kB/s | 56 kB 00:00 2024-04-15T15:17:12.485 INFO:teuthology.orchestra.run.smithi154.stdout:(88/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 659 kB/s | 44 kB 00:00 2024-04-15T15:17:12.511 INFO:teuthology.orchestra.run.smithi154.stdout:(89/174): fuse-2.9.9-15.el9.x86_64.rpm 1.2 MB/s | 79 kB 00:00 2024-04-15T15:17:12.536 INFO:teuthology.orchestra.run.smithi154.stdout:(90/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-04-15T15:17:12.570 INFO:teuthology.orchestra.run.smithi154.stdout:(91/174): python3-cryptography-36.0.1-4.el9.x86 14 MB/s | 1.1 MB 00:00 2024-04-15T15:17:12.595 INFO:teuthology.orchestra.run.smithi154.stdout:(92/174): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 103 kB 00:00 2024-04-15T15:17:12.620 INFO:teuthology.orchestra.run.smithi154.stdout:(93/174): python3-pycparser-2.20-6.el9.noarch.r 1.5 MB/s | 124 kB 00:00 2024-04-15T15:17:12.645 INFO:teuthology.orchestra.run.smithi154.stdout:(94/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.7 MB/s | 187 kB 00:00 2024-04-15T15:17:12.829 INFO:teuthology.orchestra.run.smithi154.stdout:(95/174): jq-1.6-16.el9.x86_64.rpm 802 kB/s | 187 kB 00:00 2024-04-15T15:17:12.905 INFO:teuthology.orchestra.run.smithi154.stdout:(96/174): libquadmath-11.4.1-3.el9.x86_64.rpm 755 kB/s | 195 kB 00:00 2024-04-15T15:17:12.930 INFO:teuthology.orchestra.run.smithi154.stdout:(97/174): python3-requests-2.25.1-8.el9.noarch. 1.2 MB/s | 125 kB 00:00 2024-04-15T15:17:12.972 INFO:teuthology.orchestra.run.smithi154.stdout:(98/174): libgfortran-11.4.1-3.el9.x86_64.rpm 2.2 MB/s | 803 kB 00:00 2024-04-15T15:17:13.423 INFO:teuthology.orchestra.run.smithi105.stdout: Preparing : 1/1 2024-04-15T15:17:13.480 INFO:teuthology.orchestra.run.smithi154.stdout:(99/174): elfutils-libelf-devel-0.190-2.el9.x86 37 kB/s | 21 kB 00:00 2024-04-15T15:17:13.514 INFO:teuthology.orchestra.run.smithi154.stdout:(100/174): libuuid-devel-2.37.4-18.el9.x86_64.r 44 kB/s | 25 kB 00:00 2024-04-15T15:17:13.521 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-04-15T15:17:13.556 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-04-15T15:17:13.586 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-15T15:17:13.595 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-15T15:17:13.661 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-04-15T15:17:13.714 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-04-15T15:17:13.756 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-04-15T15:17:13.790 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-04-15T15:17:13.834 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-04-15T15:17:14.158 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-04-15T15:17:14.220 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : librados2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 10/198 2024-04-15T15:17:14.333 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librados2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 10/198 2024-04-15T15:17:14.363 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libcephfs2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 11/198 2024-04-15T15:17:14.414 INFO:teuthology.orchestra.run.smithi154.stdout:(101/174): python3-devel-3.9.17-2.el9.x86_64.rp 263 kB/s | 245 kB 00:00 2024-04-15T15:17:14.440 INFO:teuthology.orchestra.run.smithi154.stdout:(102/174): python3-jinja2-2.11.3-5.el9.noarch.r 268 kB/s | 248 kB 00:00 2024-04-15T15:17:14.440 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libcephfs2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 11/198 2024-04-15T15:17:14.468 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rados-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 12/198 2024-04-15T15:17:14.508 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-04-15T15:17:14.706 INFO:teuthology.orchestra.run.smithi154.stdout:(103/174): lua-devel-5.4.4-4.el9.x86_64.rpm 84 kB/s | 22 kB 00:00 2024-04-15T15:17:14.723 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-04-15T15:17:14.775 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-04-15T15:17:14.802 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libcephsqlite-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 16/198 2024-04-15T15:17:14.907 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 16/198 2024-04-15T15:17:14.928 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libradosstriper1-2:19.0.0-2861.g9bf1c4ba.el8.x86 17/198 2024-04-15T15:17:14.985 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2861.g9bf1c4ba.el8.x86 17/198 2024-04-15T15:17:15.015 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-04-15T15:17:15.046 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-15T15:17:15.062 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-15T15:17:15.149 INFO:teuthology.orchestra.run.smithi154.stdout:(104/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 128 kB/s | 94 kB 00:00 2024-04-15T15:17:15.232 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-15T15:17:15.279 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-15T15:17:15.309 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-04-15T15:17:15.476 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-04-15T15:17:15.490 INFO:teuthology.orchestra.run.smithi154.stdout:(105/174): protobuf-compiler-3.14.0-13.el9.x86_ 1.1 MB/s | 863 kB 00:00 2024-04-15T15:17:15.629 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-15T15:17:15.680 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-15T15:17:15.795 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-15T15:17:15.848 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-15T15:17:15.849 INFO:teuthology.orchestra.run.smithi154.stdout:(106/174): gperftools-libs-2.9.1-2.el9.x86_64.r 862 kB/s | 309 kB 00:00 2024-04-15T15:17:15.882 INFO:teuthology.orchestra.run.smithi154.stdout:(107/174): abseil-cpp-20211102.0-3.el9.x86_64.r 747 kB/s | 548 kB 00:00 2024-04-15T15:17:15.908 INFO:teuthology.orchestra.run.smithi154.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 332 kB/s | 19 kB 00:00 2024-04-15T15:17:15.967 INFO:teuthology.orchestra.run.smithi154.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 444 kB/s | 26 kB 00:00 2024-04-15T15:17:16.025 INFO:teuthology.orchestra.run.smithi154.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 835 kB/s | 49 kB 00:00 2024-04-15T15:17:16.065 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-15T15:17:16.092 INFO:teuthology.orchestra.run.smithi154.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-04-15T15:17:16.176 INFO:teuthology.orchestra.run.smithi154.stdout:(112/174): luarocks-3.9.2-1.el9.noarch.rpm 1.8 MB/s | 151 kB 00:00 2024-04-15T15:17:16.177 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-04-15T15:17:16.272 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-04-15T15:17:16.303 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-04-15T15:17:16.318 INFO:teuthology.orchestra.run.smithi154.stdout:(113/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 5.8 MB/s | 838 kB 00:00 2024-04-15T15:17:16.330 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-04-15T15:17:16.365 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-04-15T15:17:16.410 INFO:teuthology.orchestra.run.smithi154.stdout:(114/174): python3-asyncssh-2.13.2-5.el9.noarch 5.9 MB/s | 548 kB 00:00 2024-04-15T15:17:16.440 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-04-15T15:17:16.477 INFO:teuthology.orchestra.run.smithi154.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 651 kB/s | 43 kB 00:00 2024-04-15T15:17:16.493 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-04-15T15:17:16.532 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-04-15T15:17:16.544 INFO:teuthology.orchestra.run.smithi154.stdout:(116/174): python3-cachetools-4.2.4-1.el9.noarc 483 kB/s | 32 kB 00:00 2024-04-15T15:17:16.595 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-04-15T15:17:16.602 INFO:teuthology.orchestra.run.smithi154.stdout:(117/174): python3-certifi-2021.10.8-2.el9.noar 257 kB/s | 15 kB 00:00 2024-04-15T15:17:16.631 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-04-15T15:17:16.669 INFO:teuthology.orchestra.run.smithi154.stdout:(118/174): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-04-15T15:17:16.674 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-04-15T15:17:16.707 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-04-15T15:17:16.735 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-04-15T15:17:16.778 INFO:teuthology.orchestra.run.smithi154.stdout:(119/174): python3-cherrypy-18.6.1-2.el9.noarch 3.2 MB/s | 358 kB 00:00 2024-04-15T15:17:16.813 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-04-15T15:17:16.853 INFO:teuthology.orchestra.run.smithi154.stdout:(120/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-04-15T15:17:16.856 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-ceph-common-2:19.0.0-2861.g9bf1c4ba.el8. 39/198 2024-04-15T15:17:16.907 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-ceph-argparse-2:19.0.0-2861.g9bf1c4ba.el 40/198 2024-04-15T15:17:16.912 INFO:teuthology.orchestra.run.smithi154.stdout:(121/174): python3-colorama-0.4.4-12.el9.noarch 603 kB/s | 35 kB 00:00 2024-04-15T15:17:16.936 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cephfs-2:19.0.0-2861.g9bf1c4ba.el8.x86_6 41/198 2024-04-15T15:17:16.979 INFO:teuthology.orchestra.run.smithi154.stdout:(122/174): python3-google-auth-2.28.2-1.el9.noa 3.0 MB/s | 203 kB 00:00 2024-04-15T15:17:17.004 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-04-15T15:17:17.046 INFO:teuthology.orchestra.run.smithi154.stdout:(123/174): libarrow-9.0.0-10.el9.x86_64.rpm 3.8 MB/s | 4.4 MB 00:01 2024-04-15T15:17:17.053 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-04-15T15:17:17.126 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-04-15T15:17:17.155 INFO:teuthology.orchestra.run.smithi154.stdout:(124/174): python3-grpcio-1.46.7-10.el9.x86_64. 12 MB/s | 2.0 MB 00:00 2024-04-15T15:17:17.180 INFO:teuthology.orchestra.run.smithi154.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-04-15T15:17:17.222 INFO:teuthology.orchestra.run.smithi154.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 160 kB/s | 11 kB 00:00 2024-04-15T15:17:17.248 INFO:teuthology.orchestra.run.smithi154.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 264 kB/s | 18 kB 00:00 2024-04-15T15:17:17.262 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-04-15T15:17:17.289 INFO:teuthology.orchestra.run.smithi154.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 346 kB/s | 23 kB 00:00 2024-04-15T15:17:17.313 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-04-15T15:17:17.314 INFO:teuthology.orchestra.run.smithi154.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 292 kB/s | 19 kB 00:00 2024-04-15T15:17:17.356 INFO:teuthology.orchestra.run.smithi154.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 295 kB/s | 20 kB 00:00 2024-04-15T15:17:17.423 INFO:teuthology.orchestra.run.smithi154.stdout:(131/174): python3-logutils-0.3.5-21.el9.noarch 695 kB/s | 46 kB 00:00 2024-04-15T15:17:17.490 INFO:teuthology.orchestra.run.smithi154.stdout:(132/174): python3-more-itertools-8.12.0-2.el9. 1.2 MB/s | 79 kB 00:00 2024-04-15T15:17:17.532 INFO:teuthology.orchestra.run.smithi154.stdout:(133/174): python3-kubernetes-26.1.0-2.el9.noar 4.7 MB/s | 1.0 MB 00:00 2024-04-15T15:17:17.558 INFO:teuthology.orchestra.run.smithi154.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 860 kB/s | 58 kB 00:00 2024-04-15T15:17:17.626 INFO:teuthology.orchestra.run.smithi154.stdout:(135/174): python3-portend-3.1.0-2.el9.noarch.r 247 kB/s | 16 kB 00:00 2024-04-15T15:17:17.650 INFO:teuthology.orchestra.run.smithi154.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2024-04-15T15:17:17.692 INFO:teuthology.orchestra.run.smithi154.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-04-15T15:17:17.717 INFO:teuthology.orchestra.run.smithi154.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 423 kB/s | 28 kB 00:00 2024-04-15T15:17:17.759 INFO:teuthology.orchestra.run.smithi154.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 460 kB/s | 31 kB 00:00 2024-04-15T15:17:17.810 INFO:teuthology.orchestra.run.smithi154.stdout:(140/174): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-04-15T15:17:17.829 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-04-15T15:17:17.835 INFO:teuthology.orchestra.run.smithi154.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 786 kB/s | 59 kB 00:00 2024-04-15T15:17:17.877 INFO:teuthology.orchestra.run.smithi154.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 536 kB/s | 36 kB 00:00 2024-04-15T15:17:17.902 INFO:teuthology.orchestra.run.smithi154.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 766 kB/s | 51 kB 00:00 2024-04-15T15:17:17.910 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-04-15T15:17:17.969 INFO:teuthology.orchestra.run.smithi154.stdout:(144/174): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-04-15T15:17:17.986 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-04-15T15:17:18.003 INFO:teuthology.orchestra.run.smithi154.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-04-15T15:17:18.038 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-15T15:17:18.044 INFO:teuthology.orchestra.run.smithi154.stdout:(146/174): python3-werkzeug-2.0.3-3.el9.1.noarc 5.6 MB/s | 427 kB 00:00 2024-04-15T15:17:18.046 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-15T15:17:18.070 INFO:teuthology.orchestra.run.smithi154.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 299 kB/s | 20 kB 00:00 2024-04-15T15:17:18.078 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-04-15T15:17:18.112 INFO:teuthology.orchestra.run.smithi154.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-04-15T15:17:18.118 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-04-15T15:17:18.156 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librados-devel-2:19.0.0-2861.g9bf1c4ba.el8.x86_6 53/198 2024-04-15T15:17:18.186 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-04-15T15:17:18.313 INFO:teuthology.orchestra.run.smithi154.stdout:(149/174): thrift-0.15.0-2.el9.x86_64.rpm 6.5 MB/s | 1.6 MB 00:00 2024-04-15T15:17:18.429 INFO:teuthology.orchestra.run.smithi154.stdout:(150/174): librados2-19.0.0-2861.g9bf1c4ba.el8. 11 MB/s | 3.4 MB 00:00 2024-04-15T15:17:18.441 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-04-15T15:17:18.505 INFO:teuthology.orchestra.run.smithi154.stdout:(151/174): NetworkManager-1.47.4-1.el9.x86_64.r 31 MB/s | 2.3 MB 00:00 2024-04-15T15:17:18.580 INFO:teuthology.orchestra.run.smithi154.stdout:(152/174): NetworkManager-libnm-1.47.4-1.el9.x8 24 MB/s | 1.8 MB 00:00 2024-04-15T15:17:18.614 INFO:teuthology.orchestra.run.smithi154.stdout:(153/174): NetworkManager-team-1.47.4-1.el9.x86 1.2 MB/s | 41 kB 00:00 2024-04-15T15:17:18.654 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-04-15T15:17:18.655 INFO:teuthology.orchestra.run.smithi154.stdout:(154/174): NetworkManager-tui-1.47.4-1.el9.x86_ 5.8 MB/s | 248 kB 00:00 2024-04-15T15:17:18.706 INFO:teuthology.orchestra.run.smithi154.stdout:(155/174): librbd1-19.0.0-2861.g9bf1c4ba.el8.x8 8.0 MB/s | 3.1 MB 00:00 2024-04-15T15:17:18.731 INFO:teuthology.orchestra.run.smithi154.stdout:(156/174): xfsdump-3.1.12-4.el9.x86_64.rpm 4.2 MB/s | 321 kB 00:00 2024-04-15T15:17:18.746 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-04-15T15:17:18.791 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-04-15T15:17:18.832 INFO:teuthology.orchestra.run.smithi154.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 364 kB/s | 36 kB 00:00 2024-04-15T15:17:18.858 INFO:teuthology.orchestra.run.smithi154.stdout:(158/174): acl-2.3.1-4.el9.x86_64.rpm 475 kB/s | 71 kB 00:00 2024-04-15T15:17:18.930 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-04-15T15:17:18.941 INFO:teuthology.orchestra.run.smithi154.stdout:(159/174): openssl-devel-3.0.7-27.el9.x86_64.rp 709 kB/s | 4.1 MB 00:05 2024-04-15T15:17:18.966 INFO:teuthology.orchestra.run.smithi154.stdout:(160/174): elfutils-libelf-0.190-2.el9.x86_64.r 1.4 MB/s | 192 kB 00:00 2024-04-15T15:17:18.977 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-04-15T15:17:18.991 INFO:teuthology.orchestra.run.smithi154.stdout:(161/174): elfutils-libs-0.190-2.el9.x86_64.rpm 1.9 MB/s | 256 kB 00:00 2024-04-15T15:17:19.025 INFO:teuthology.orchestra.run.smithi154.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.8 MB/s | 108 kB 00:00 2024-04-15T15:17:19.042 INFO:teuthology.orchestra.run.smithi154.stdout:(163/174): libacl-2.3.1-4.el9.x86_64.rpm 230 kB/s | 23 kB 00:00 2024-04-15T15:17:19.067 INFO:teuthology.orchestra.run.smithi154.stdout:(164/174): libfdisk-2.37.4-18.el9.x86_64.rpm 2.0 MB/s | 155 kB 00:00 2024-04-15T15:17:19.092 INFO:teuthology.orchestra.run.smithi154.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 2.0 MB/s | 136 kB 00:00 2024-04-15T15:17:19.118 INFO:teuthology.orchestra.run.smithi154.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 571 kB/s | 29 kB 00:00 2024-04-15T15:17:19.134 INFO:teuthology.orchestra.run.smithi154.stdout:(167/174): libsmartcols-2.37.4-18.el9.x86_64.rp 689 kB/s | 64 kB 00:00 2024-04-15T15:17:19.278 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-04-15T15:17:19.335 INFO:teuthology.orchestra.run.smithi154.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 5.0 MB/s | 1.2 MB 00:00 2024-04-15T15:17:19.348 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-04-15T15:17:19.386 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-04-15T15:17:19.423 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-04-15T15:17:19.444 INFO:teuthology.orchestra.run.smithi154.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 4.2 MB/s | 465 kB 00:00 2024-04-15T15:17:19.470 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-04-15T15:17:19.494 INFO:teuthology.orchestra.run.smithi154.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 624 kB/s | 31 kB 00:00 2024-04-15T15:17:19.506 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-04-15T15:17:19.544 INFO:teuthology.orchestra.run.smithi154.stdout:(171/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 5.1 MB/s | 2.2 MB 00:00 2024-04-15T15:17:19.549 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-04-15T15:17:19.591 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-04-15T15:17:19.686 INFO:teuthology.orchestra.run.smithi154.stdout:(172/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 5.6 MB/s | 1.1 MB 00:00 2024-04-15T15:17:19.724 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-04-15T15:17:19.770 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-04-15T15:17:19.801 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-04-15T15:17:19.842 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-04-15T15:17:19.844 INFO:teuthology.orchestra.run.smithi154.stdout:(173/174): util-linux-2.37.4-18.el9.x86_64.rpm 3.2 MB/s | 2.3 MB 00:00 2024-04-15T15:17:19.936 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-04-15T15:17:19.985 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-04-15T15:17:20.033 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-04-15T15:17:20.058 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-04-15T15:17:20.083 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-04-15T15:17:20.176 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-04-15T15:17:20.222 INFO:teuthology.orchestra.run.smithi154.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 110 kB/s | 74 kB 00:00 2024-04-15T15:17:20.229 INFO:teuthology.orchestra.run.smithi154.stdout:-------------------------------------------------------------------------------- 2024-04-15T15:17:20.229 INFO:teuthology.orchestra.run.smithi154.stdout:Total 14 MB/s | 228 MB 00:15 2024-04-15T15:17:20.300 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-04-15T15:17:20.407 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-04-15T15:17:20.450 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-04-15T15:17:20.498 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-04-15T15:17:20.554 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-04-15T15:17:20.629 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction check 2024-04-15T15:17:20.735 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-04-15T15:17:20.787 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-04-15T15:17:20.908 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction check succeeded. 2024-04-15T15:17:20.909 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction test 2024-04-15T15:17:20.914 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-04-15T15:17:20.967 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-04-15T15:17:21.001 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-04-15T15:17:21.031 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-15T15:17:21.068 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-04-15T15:17:21.103 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-04-15T15:17:21.141 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-04-15T15:17:21.198 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-04-15T15:17:21.228 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-04-15T15:17:21.303 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-04-15T15:17:21.339 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-04-15T15:17:21.372 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-04-15T15:17:21.413 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-04-15T15:17:21.448 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-04-15T15:17:21.537 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-04-15T15:17:21.625 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-04-15T15:17:21.670 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-04-15T15:17:21.710 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-04-15T15:17:21.757 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-04-15T15:17:21.853 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-04-15T15:17:21.911 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-04-15T15:17:22.624 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-04-15T15:17:22.667 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-04-15T15:17:22.701 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-04-15T15:17:22.749 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-04-15T15:17:22.827 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-04-15T15:17:22.862 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-04-15T15:17:22.973 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction test succeeded. 2024-04-15T15:17:22.973 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction 2024-04-15T15:17:23.029 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-04-15T15:17:23.051 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-04-15T15:17:23.128 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-04-15T15:17:23.768 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-04-15T15:17:23.880 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-04-15T15:17:25.087 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-04-15T15:17:25.185 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-04-15T15:17:25.216 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-04-15T15:17:25.256 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-04-15T15:17:25.283 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-04-15T15:17:25.596 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-04-15T15:17:25.630 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : librbd1-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 124/198 2024-04-15T15:17:25.704 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librbd1-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 124/198 2024-04-15T15:17:25.735 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rbd-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 125/198 2024-04-15T15:17:25.774 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-04-15T15:17:26.239 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-04-15T15:17:26.267 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librgw2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 128/198 2024-04-15T15:17:26.337 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librgw2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 128/198 2024-04-15T15:17:26.360 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rgw-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 129/198 2024-04-15T15:17:28.606 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-common-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 130/198 2024-04-15T15:17:28.665 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-common-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 130/198 2024-04-15T15:17:29.208 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-common-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 130/198 2024-04-15T15:17:29.251 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-base-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 131/198 2024-04-15T15:17:29.338 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-base-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 131/198 2024-04-15T15:17:29.338 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-15T15:17:29.338 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-15T15:17:29.338 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:17:29.369 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-selinux-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 132/198 2024-04-15T15:17:29.976 INFO:teuthology.orchestra.run.smithi154.stdout: Preparing : 1/1 2024-04-15T15:17:30.085 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-04-15T15:17:30.128 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-04-15T15:17:30.175 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-15T15:17:30.185 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-15T15:17:30.238 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-04-15T15:17:30.296 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-04-15T15:17:30.328 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-04-15T15:17:30.354 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-04-15T15:17:30.406 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-04-15T15:17:30.736 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-04-15T15:17:30.805 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : librados2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 10/198 2024-04-15T15:17:30.927 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librados2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 10/198 2024-04-15T15:17:30.956 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libcephfs2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 11/198 2024-04-15T15:17:31.030 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: libcephfs2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 11/198 2024-04-15T15:17:31.069 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-rados-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 12/198 2024-04-15T15:17:31.109 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-04-15T15:17:31.300 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-04-15T15:17:31.362 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-04-15T15:17:31.390 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libcephsqlite-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 16/198 2024-04-15T15:17:31.485 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 16/198 2024-04-15T15:17:31.508 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libradosstriper1-2:19.0.0-2861.g9bf1c4ba.el8.x86 17/198 2024-04-15T15:17:31.573 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2861.g9bf1c4ba.el8.x86 17/198 2024-04-15T15:17:31.629 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-04-15T15:17:31.651 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-15T15:17:31.668 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-15T15:17:31.861 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-15T15:17:31.892 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-15T15:17:31.931 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-04-15T15:17:32.135 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-04-15T15:17:32.260 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-15T15:17:32.320 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-15T15:17:32.420 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-15T15:17:32.471 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-15T15:17:32.672 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-15T15:17:32.776 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-04-15T15:17:32.879 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-04-15T15:17:32.910 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-04-15T15:17:32.937 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-04-15T15:17:32.980 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-04-15T15:17:33.055 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-04-15T15:17:33.116 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-04-15T15:17:33.155 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-04-15T15:17:33.220 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-04-15T15:17:33.254 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-04-15T15:17:33.306 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-04-15T15:17:33.339 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-04-15T15:17:33.366 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-04-15T15:17:33.436 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-04-15T15:17:33.471 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-ceph-common-2:19.0.0-2861.g9bf1c4ba.el8. 39/198 2024-04-15T15:17:33.523 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-ceph-argparse-2:19.0.0-2861.g9bf1c4ba.el 40/198 2024-04-15T15:17:33.560 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cephfs-2:19.0.0-2861.g9bf1c4ba.el8.x86_6 41/198 2024-04-15T15:17:33.645 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-04-15T15:17:33.691 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-04-15T15:17:33.766 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-04-15T15:17:33.900 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-04-15T15:17:33.953 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-04-15T15:17:34.473 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-04-15T15:17:34.557 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-04-15T15:17:34.634 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-04-15T15:17:34.684 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-15T15:17:34.694 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-15T15:17:34.725 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-04-15T15:17:34.766 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-04-15T15:17:34.811 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : librados-devel-2:19.0.0-2861.g9bf1c4ba.el8.x86_6 53/198 2024-04-15T15:17:34.850 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-04-15T15:17:35.141 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-04-15T15:17:35.514 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-04-15T15:17:35.586 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-04-15T15:17:35.632 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-04-15T15:17:35.777 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-04-15T15:17:35.862 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-04-15T15:17:36.029 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-04-15T15:17:36.286 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-04-15T15:17:36.332 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-04-15T15:17:36.369 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-04-15T15:17:36.416 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-04-15T15:17:36.452 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-04-15T15:17:36.495 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-04-15T15:17:36.537 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-04-15T15:17:36.673 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-04-15T15:17:36.726 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-04-15T15:17:36.756 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-04-15T15:17:36.797 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-04-15T15:17:36.901 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-04-15T15:17:36.948 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-04-15T15:17:36.999 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-04-15T15:17:37.038 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-04-15T15:17:37.063 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-04-15T15:17:37.155 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-04-15T15:17:37.282 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-04-15T15:17:37.381 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-04-15T15:17:37.421 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-04-15T15:17:37.487 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-04-15T15:17:37.551 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-04-15T15:17:37.746 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-04-15T15:17:37.792 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-04-15T15:17:37.930 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-04-15T15:17:37.974 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-04-15T15:17:38.006 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-04-15T15:17:38.036 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-15T15:17:38.072 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-04-15T15:17:38.107 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-04-15T15:17:38.146 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-04-15T15:17:38.178 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-04-15T15:17:38.208 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-04-15T15:17:38.284 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-04-15T15:17:38.318 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-04-15T15:17:38.351 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-04-15T15:17:38.393 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-04-15T15:17:38.436 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-04-15T15:17:38.535 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-04-15T15:17:38.632 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-04-15T15:17:38.675 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-04-15T15:17:38.714 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-04-15T15:17:38.761 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-04-15T15:17:38.860 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-04-15T15:17:38.916 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-04-15T15:17:39.655 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-04-15T15:17:39.705 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-04-15T15:17:39.739 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-04-15T15:17:39.787 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-04-15T15:17:39.876 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-04-15T15:17:39.948 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-04-15T15:17:40.126 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-04-15T15:17:40.162 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-04-15T15:17:40.240 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-04-15T15:17:40.917 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-04-15T15:17:41.036 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-04-15T15:17:41.232 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 132/198 2024-04-15T15:17:41.232 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /sys 2024-04-15T15:17:41.232 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /proc 2024-04-15T15:17:41.232 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /mnt 2024-04-15T15:17:41.232 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /var/tmp 2024-04-15T15:17:41.232 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /home 2024-04-15T15:17:41.232 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /root 2024-04-15T15:17:41.232 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /tmp 2024-04-15T15:17:41.233 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:17:41.400 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-volume-2:19.0.0-2861.g9bf1c4ba.el8.noarch 133/198 2024-04-15T15:17:41.416 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-volume-2:19.0.0-2861.g9bf1c4ba.el8.noarch 133/198 2024-04-15T15:17:41.416 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-15T15:17:41.416 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-15T15:17:41.416 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:17:42.277 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-04-15T15:17:42.354 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-04-15T15:17:42.385 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-04-15T15:17:42.416 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-04-15T15:17:42.444 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-04-15T15:17:42.756 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-04-15T15:17:42.782 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : librbd1-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 124/198 2024-04-15T15:17:42.881 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librbd1-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 124/198 2024-04-15T15:17:42.903 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-osd-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 134/198 2024-04-15T15:17:42.912 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-rbd-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 125/198 2024-04-15T15:17:42.930 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-osd-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 134/198 2024-04-15T15:17:42.930 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-15T15:17:42.930 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-15T15:17:42.930 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-15T15:17:42.931 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-15T15:17:42.931 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:17:42.958 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-04-15T15:17:43.163 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mds-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 135/198 2024-04-15T15:17:43.185 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mds-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 135/198 2024-04-15T15:17:43.186 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-15T15:17:43.186 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-15T15:17:43.186 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-15T15:17:43.186 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-15T15:17:43.186 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:17:43.421 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-04-15T15:17:43.460 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : librgw2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 128/198 2024-04-15T15:17:43.523 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librgw2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 128/198 2024-04-15T15:17:43.553 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-rgw-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 129/198 2024-04-15T15:17:43.630 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mon-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 136/198 2024-04-15T15:17:43.653 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mon-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 136/198 2024-04-15T15:17:43.653 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-15T15:17:43.654 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-15T15:17:43.654 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-15T15:17:43.654 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-15T15:17:43.654 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:17:43.740 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-04-15T15:17:44.202 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: cephadm-2:19.0.0-2861.g9bf1c4ba.el8.noarch 138/198 2024-04-15T15:17:44.230 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : cephadm-2:19.0.0-2861.g9bf1c4ba.el8.noarch 138/198 2024-04-15T15:17:44.262 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2861.g9bf1c4ba.e 139/198 2024-04-15T15:17:44.324 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2861.g9bf1c4ba. 140/198 2024-04-15T15:17:44.370 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2861.g9bf1c4ba.el8.noa 141/198 2024-04-15T15:17:45.406 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2861.g9bf1c4ba.el8.noa 141/198 2024-04-15T15:17:45.447 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2861.g9bf1c4ba.el8.n 142/198 2024-04-15T15:17:45.838 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-common-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 130/198 2024-04-15T15:17:45.885 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-common-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 130/198 2024-04-15T15:17:46.418 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2861.g9bf1c4ba.el8.n 142/198 2024-04-15T15:17:46.434 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-common-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 130/198 2024-04-15T15:17:46.447 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2861.g9bf 143/198 2024-04-15T15:17:46.488 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-base-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 131/198 2024-04-15T15:17:46.548 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2861.g9bf 143/198 2024-04-15T15:17:46.570 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-base-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 131/198 2024-04-15T15:17:46.570 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-15T15:17:46.570 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-15T15:17:46.571 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-15T15:17:46.606 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-selinux-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 132/198 2024-04-15T15:17:46.710 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2861.g9bf1c4ba.el 144/198 2024-04-15T15:17:46.741 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 145/198 2024-04-15T15:17:46.765 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 145/198 2024-04-15T15:17:46.765 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-15T15:17:46.765 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-15T15:17:46.765 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-15T15:17:46.766 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-15T15:17:46.766 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:17:46.816 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-rook-2:19.0.0-2861.g9bf1c4ba.el8.noarch 146/198 2024-04-15T15:17:46.828 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2861.g9bf1c4ba.el8.noarch 146/198 2024-04-15T15:17:46.876 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 147/198 2024-04-15T15:17:47.749 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-04-15T15:17:47.777 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-radosgw-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 149/198 2024-04-15T15:17:47.800 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 149/198 2024-04-15T15:17:47.800 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-15T15:17:47.800 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-15T15:17:47.801 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-15T15:17:47.801 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-15T15:17:47.801 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:17:47.844 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2861.g9bf1c 150/198 2024-04-15T15:17:47.867 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2861.g9bf1c 150/198 2024-04-15T15:17:47.867 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-15T15:17:47.867 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-15T15:17:47.867 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:17:47.919 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : cephfs-mirror-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 151/198 2024-04-15T15:17:47.943 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 151/198 2024-04-15T15:17:47.943 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-15T15:17:47.943 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-15T15:17:47.943 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-15T15:17:47.943 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-15T15:17:47.943 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:17:48.245 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : rbd-mirror-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 152/198 2024-04-15T15:17:48.269 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 152/198 2024-04-15T15:17:48.269 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-15T15:17:48.269 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-15T15:17:48.269 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-15T15:17:48.269 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-15T15:17:48.269 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:17:52.875 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-test-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 153/198 2024-04-15T15:17:52.917 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : rbd-fuse-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 154/198 2024-04-15T15:17:52.947 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : rbd-nbd-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 155/198 2024-04-15T15:17:52.974 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-04-15T15:17:53.156 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : cephfs-shell-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 157/198 2024-04-15T15:17:53.273 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-04-15T15:17:53.313 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-fuse-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 159/198 2024-04-15T15:17:53.380 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-04-15T15:17:53.430 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-04-15T15:17:53.473 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-04-15T15:17:53.506 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libcephfs-devel-2:19.0.0-2861.g9bf1c4ba.el8.x86_ 163/198 2024-04-15T15:17:53.530 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-04-15T15:17:53.560 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : NetworkManager-team-1:1.47.4-1.el9.x86_64 165/198 2024-04-15T15:17:53.625 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : NetworkManager-tui-1:1.47.4-1.el9.x86_64 166/198 2024-04-15T15:17:53.686 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-04-15T15:17:53.725 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-04-15T15:17:54.270 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-15T15:17:54.309 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-15T15:17:54.329 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-15T15:17:55.142 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : cephfs-top-2:19.0.0-2861.g9bf1c4ba.el8.noarch 170/198 2024-04-15T15:17:55.220 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-04-15T15:17:55.261 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-04-15T15:17:55.342 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-04-15T15:17:55.374 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-04-15T15:17:55.407 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-04-15T15:17:55.432 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-04-15T15:17:55.459 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-04-15T15:17:55.459 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-15T15:17:55.555 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-15T15:17:55.585 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-04-15T15:17:55.585 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-15T15:17:55.686 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-15T15:17:55.721 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-04-15T15:17:55.745 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-04-15T15:17:55.771 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-04-15T15:17:55.797 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-04-15T15:17:55.832 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-04-15T15:17:55.869 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-04-15T15:17:55.900 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-04-15T15:17:55.926 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-04-15T15:17:55.951 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-04-15T15:17:55.977 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-04-15T15:17:56.002 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-04-15T15:17:56.008 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-15T15:17:56.009 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-15T15:17:56.188 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-15T15:17:56.188 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-15T15:17:56.254 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-15T15:17:56.286 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-04-15T15:17:56.313 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-04-15T15:17:56.338 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-04-15T15:17:56.372 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-04-15T15:17:56.438 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-15T15:17:58.301 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 132/198 2024-04-15T15:17:58.301 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /sys 2024-04-15T15:17:58.301 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /proc 2024-04-15T15:17:58.301 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /mnt 2024-04-15T15:17:58.301 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /var/tmp 2024-04-15T15:17:58.301 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /home 2024-04-15T15:17:58.301 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /root 2024-04-15T15:17:58.301 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /tmp 2024-04-15T15:17:58.301 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-15T15:17:58.475 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-volume-2:19.0.0-2861.g9bf1c4ba.el8.noarch 133/198 2024-04-15T15:17:58.491 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-volume-2:19.0.0-2861.g9bf1c4ba.el8.noarch 133/198 2024-04-15T15:17:58.491 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-15T15:17:58.492 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-15T15:17:58.492 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-15T15:18:00.028 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-osd-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 134/198 2024-04-15T15:18:00.055 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-osd-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 134/198 2024-04-15T15:18:00.055 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-15T15:18:00.056 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-15T15:18:00.056 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-15T15:18:00.056 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-15T15:18:00.056 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-15T15:18:00.304 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mds-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 135/198 2024-04-15T15:18:00.327 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mds-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 135/198 2024-04-15T15:18:00.327 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-15T15:18:00.327 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-15T15:18:00.328 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-15T15:18:00.328 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-15T15:18:00.328 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-15T15:18:00.780 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mon-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 136/198 2024-04-15T15:18:00.803 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mon-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 136/198 2024-04-15T15:18:00.803 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-15T15:18:00.803 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-15T15:18:00.803 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-15T15:18:00.804 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-15T15:18:00.804 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-15T15:18:00.898 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-04-15T15:18:01.350 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: cephadm-2:19.0.0-2861.g9bf1c4ba.el8.noarch 138/198 2024-04-15T15:18:01.388 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : cephadm-2:19.0.0-2861.g9bf1c4ba.el8.noarch 138/198 2024-04-15T15:18:01.420 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2861.g9bf1c4ba.e 139/198 2024-04-15T15:18:01.482 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2861.g9bf1c4ba. 140/198 2024-04-15T15:18:01.511 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2861.g9bf1c4ba.el8.noa 141/198 2024-04-15T15:18:01.741 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-15T15:18:01.741 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 1/198 2024-04-15T15:18:01.741 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-base-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2/198 2024-04-15T15:18:01.741 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-common-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 3/198 2024-04-15T15:18:01.741 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-fuse-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 4/198 2024-04-15T15:18:01.741 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2861.g9bf1c 5/198 2024-04-15T15:18:01.741 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mds-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 6/198 2024-04-15T15:18:01.742 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 7/198 2024-04-15T15:18:01.742 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mon-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 8/198 2024-04-15T15:18:01.742 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-osd-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 9/198 2024-04-15T15:18:01.742 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-radosgw-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 10/198 2024-04-15T15:18:01.742 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-selinux-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 11/198 2024-04-15T15:18:01.742 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-test-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 12/198 2024-04-15T15:18:01.742 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : cephfs-mirror-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 13/198 2024-04-15T15:18:01.742 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : cephfs-shell-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 14/198 2024-04-15T15:18:01.742 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libcephfs-devel-2:19.0.0-2861.g9bf1c4ba.el8.x86_ 15/198 2024-04-15T15:18:01.742 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libcephfs2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 16/198 2024-04-15T15:18:01.742 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libcephsqlite-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 17/198 2024-04-15T15:18:01.742 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librados-devel-2:19.0.0-2861.g9bf1c4ba.el8.x86_6 18/198 2024-04-15T15:18:01.742 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libradosstriper1-2:19.0.0-2861.g9bf1c4ba.el8.x86 19/198 2024-04-15T15:18:01.743 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librgw2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 20/198 2024-04-15T15:18:01.743 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2861.g9bf1c4ba.el 21/198 2024-04-15T15:18:01.743 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-ceph-common-2:19.0.0-2861.g9bf1c4ba.el8. 22/198 2024-04-15T15:18:01.743 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cephfs-2:19.0.0-2861.g9bf1c4ba.el8.x86_6 23/198 2024-04-15T15:18:01.743 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rados-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 24/198 2024-04-15T15:18:01.743 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rbd-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 25/198 2024-04-15T15:18:01.743 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rgw-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 26/198 2024-04-15T15:18:01.743 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : rbd-fuse-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 27/198 2024-04-15T15:18:01.743 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : rbd-mirror-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 28/198 2024-04-15T15:18:01.743 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : rbd-nbd-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 29/198 2024-04-15T15:18:01.743 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2861.g9bf1c4ba. 30/198 2024-04-15T15:18:01.743 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2861.g9bf1c4ba.el8.noa 31/198 2024-04-15T15:18:01.743 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2861.g9bf1c4ba.el8.n 32/198 2024-04-15T15:18:01.744 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2861.g9bf 33/198 2024-04-15T15:18:01.744 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2861.g9bf1c4ba.el 34/198 2024-04-15T15:18:01.744 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2861.g9bf1c4ba.el8.noarch 35/198 2024-04-15T15:18:01.744 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2861.g9bf1c4ba.e 36/198 2024-04-15T15:18:01.744 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-volume-2:19.0.0-2861.g9bf1c4ba.el8.noarch 37/198 2024-04-15T15:18:01.744 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : cephadm-2:19.0.0-2861.g9bf1c4ba.el8.noarch 38/198 2024-04-15T15:18:01.744 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : cephfs-top-2:19.0.0-2861.g9bf1c4ba.el8.noarch 39/198 2024-04-15T15:18:01.744 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-04-15T15:18:01.744 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-04-15T15:18:01.744 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-04-15T15:18:01.744 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-04-15T15:18:01.744 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-04-15T15:18:01.744 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-04-15T15:18:01.745 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-04-15T15:18:01.745 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-04-15T15:18:01.745 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-04-15T15:18:01.745 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-04-15T15:18:01.745 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-04-15T15:18:01.745 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-04-15T15:18:01.745 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-04-15T15:18:01.745 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-04-15T15:18:01.745 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-04-15T15:18:01.745 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-04-15T15:18:01.745 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-04-15T15:18:01.745 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-04-15T15:18:01.745 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : postgresql-13.14-1.el9.x86_64 58/198 2024-04-15T15:18:01.745 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 59/198 2024-04-15T15:18:01.746 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 60/198 2024-04-15T15:18:01.746 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 61/198 2024-04-15T15:18:01.746 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-04-15T15:18:01.746 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-04-15T15:18:01.746 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-04-15T15:18:01.746 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-04-15T15:18:01.746 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-04-15T15:18:01.746 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-04-15T15:18:01.746 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-04-15T15:18:01.746 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-04-15T15:18:01.746 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-04-15T15:18:01.746 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-04-15T15:18:01.746 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-04-15T15:18:01.747 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-04-15T15:18:01.747 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-04-15T15:18:01.747 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-04-15T15:18:01.747 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-04-15T15:18:01.747 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-04-15T15:18:01.747 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-04-15T15:18:01.747 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-04-15T15:18:01.747 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-04-15T15:18:01.747 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-04-15T15:18:01.747 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 82/198 2024-04-15T15:18:01.747 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 83/198 2024-04-15T15:18:01.747 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 84/198 2024-04-15T15:18:01.747 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 85/198 2024-04-15T15:18:01.747 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 86/198 2024-04-15T15:18:01.748 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 87/198 2024-04-15T15:18:01.748 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 88/198 2024-04-15T15:18:01.748 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-15T15:18:01.748 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 90/198 2024-04-15T15:18:01.748 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 91/198 2024-04-15T15:18:01.748 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-ply-3.11-14.el9.noarch 92/198 2024-04-15T15:18:01.748 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 93/198 2024-04-15T15:18:01.748 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/198 2024-04-15T15:18:01.748 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : jq-1.6-16.el9.x86_64 95/198 2024-04-15T15:18:01.748 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 96/198 2024-04-15T15:18:01.748 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 97/198 2024-04-15T15:18:01.748 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 98/198 2024-04-15T15:18:01.748 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 99/198 2024-04-15T15:18:01.748 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 100/198 2024-04-15T15:18:01.749 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 101/198 2024-04-15T15:18:01.749 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 102/198 2024-04-15T15:18:01.749 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 103/198 2024-04-15T15:18:01.749 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-04-15T15:18:01.749 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-04-15T15:18:01.749 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-04-15T15:18:01.749 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-04-15T15:18:01.749 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-04-15T15:18:01.749 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-04-15T15:18:01.749 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-04-15T15:18:01.749 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-04-15T15:18:01.749 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-04-15T15:18:01.749 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-04-15T15:18:01.749 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-04-15T15:18:01.749 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-15T15:18:01.749 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-04-15T15:18:01.750 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-04-15T15:18:01.750 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-04-15T15:18:01.750 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-04-15T15:18:01.750 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-04-15T15:18:01.750 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-04-15T15:18:01.750 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-04-15T15:18:01.750 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-04-15T15:18:01.750 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-04-15T15:18:01.750 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-04-15T15:18:01.750 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-04-15T15:18:01.750 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-04-15T15:18:01.750 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-04-15T15:18:01.750 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-04-15T15:18:01.751 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-04-15T15:18:01.751 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-04-15T15:18:01.751 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-04-15T15:18:01.751 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-04-15T15:18:01.751 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-04-15T15:18:01.751 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-04-15T15:18:01.751 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-04-15T15:18:01.751 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-04-15T15:18:01.751 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-04-15T15:18:01.751 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-04-15T15:18:01.751 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-04-15T15:18:01.751 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-04-15T15:18:01.751 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-04-15T15:18:01.751 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-04-15T15:18:01.752 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-04-15T15:18:01.752 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-04-15T15:18:01.752 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-04-15T15:18:01.752 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-04-15T15:18:01.752 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-04-15T15:18:01.752 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-04-15T15:18:01.752 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-04-15T15:18:01.752 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librados2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 151/198 2024-04-15T15:18:01.752 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-04-15T15:18:01.752 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librbd1-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 153/198 2024-04-15T15:18:01.752 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-04-15T15:18:01.752 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 155/198 2024-04-15T15:18:01.752 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-04-15T15:18:01.752 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 157/198 2024-04-15T15:18:01.753 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-04-15T15:18:01.753 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 159/198 2024-04-15T15:18:01.753 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-04-15T15:18:01.753 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/198 2024-04-15T15:18:01.753 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-04-15T15:18:01.753 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 163/198 2024-04-15T15:18:01.753 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 164/198 2024-04-15T15:18:01.753 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-04-15T15:18:01.753 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-04-15T15:18:01.753 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-04-15T15:18:01.753 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-04-15T15:18:01.753 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-04-15T15:18:01.753 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-04-15T15:18:01.754 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-04-15T15:18:01.754 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-04-15T15:18:01.754 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-04-15T15:18:01.754 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-04-15T15:18:01.754 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-04-15T15:18:01.754 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-04-15T15:18:01.754 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-04-15T15:18:01.754 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-04-15T15:18:01.754 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-04-15T15:18:01.754 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-04-15T15:18:01.754 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-04-15T15:18:01.754 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-04-15T15:18:01.754 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-04-15T15:18:01.754 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-04-15T15:18:01.755 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-04-15T15:18:01.755 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-04-15T15:18:01.755 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-04-15T15:18:01.755 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-04-15T15:18:01.755 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-04-15T15:18:01.755 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-04-15T15:18:01.755 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-04-15T15:18:01.755 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-04-15T15:18:01.755 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-04-15T15:18:01.755 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-04-15T15:18:01.755 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-04-15T15:18:01.755 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-04-15T15:18:01.755 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-04-15T15:18:02.549 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2861.g9bf1c4ba.el8.noa 141/198 2024-04-15T15:18:02.579 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2861.g9bf1c4ba.el8.n 142/198 2024-04-15T15:18:03.538 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2861.g9bf1c4ba.el8.n 142/198 2024-04-15T15:18:03.572 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2861.g9bf 143/198 2024-04-15T15:18:03.672 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2861.g9bf 143/198 2024-04-15T15:18:03.835 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2861.g9bf1c4ba.el 144/198 2024-04-15T15:18:03.873 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 145/198 2024-04-15T15:18:03.898 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 145/198 2024-04-15T15:18:03.898 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-15T15:18:03.898 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-15T15:18:03.898 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-15T15:18:03.898 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-15T15:18:03.898 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-15T15:18:03.957 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-rook-2:19.0.0-2861.g9bf1c4ba.el8.noarch 146/198 2024-04-15T15:18:03.969 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2861.g9bf1c4ba.el8.noarch 146/198 2024-04-15T15:18:04.017 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 147/198 2024-04-15T15:18:04.881 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-04-15T15:18:04.881 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:18:04.881 INFO:teuthology.orchestra.run.smithi105.stdout:Upgraded: 2024-04-15T15:18:04.881 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-1:1.47.4-1.el9.x86_64 2024-04-15T15:18:04.881 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 2024-04-15T15:18:04.882 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-team-1:1.47.4-1.el9.x86_64 2024-04-15T15:18:04.882 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-tui-1:1.47.4-1.el9.x86_64 2024-04-15T15:18:04.882 INFO:teuthology.orchestra.run.smithi105.stdout: acl-2.3.1-4.el9.x86_64 2024-04-15T15:18:04.882 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-04-15T15:18:04.882 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-04-15T15:18:04.882 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-04-15T15:18:04.882 INFO:teuthology.orchestra.run.smithi105.stdout: libacl-2.3.1-4.el9.x86_64 2024-04-15T15:18:04.882 INFO:teuthology.orchestra.run.smithi105.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-04-15T15:18:04.882 INFO:teuthology.orchestra.run.smithi105.stdout: libblkid-2.37.4-18.el9.x86_64 2024-04-15T15:18:04.882 INFO:teuthology.orchestra.run.smithi105.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-04-15T15:18:04.882 INFO:teuthology.orchestra.run.smithi105.stdout: libmount-2.37.4-18.el9.x86_64 2024-04-15T15:18:04.883 INFO:teuthology.orchestra.run.smithi105.stdout: librados2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:18:04.883 INFO:teuthology.orchestra.run.smithi105.stdout: librbd1-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:18:04.883 INFO:teuthology.orchestra.run.smithi105.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-04-15T15:18:04.883 INFO:teuthology.orchestra.run.smithi105.stdout: libuuid-2.37.4-18.el9.x86_64 2024-04-15T15:18:04.883 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-04-15T15:18:04.883 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-04-15T15:18:04.883 INFO:teuthology.orchestra.run.smithi105.stdout: util-linux-2.37.4-18.el9.x86_64 2024-04-15T15:18:04.883 INFO:teuthology.orchestra.run.smithi105.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-04-15T15:18:04.883 INFO:teuthology.orchestra.run.smithi105.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-04-15T15:18:04.883 INFO:teuthology.orchestra.run.smithi105.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-04-15T15:18:04.883 INFO:teuthology.orchestra.run.smithi105.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-04-15T15:18:04.883 INFO:teuthology.orchestra.run.smithi105.stdout:Installed: 2024-04-15T15:18:04.883 INFO:teuthology.orchestra.run.smithi105.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-15T15:18:04.884 INFO:teuthology.orchestra.run.smithi105.stdout: automake-1.16.2-8.el9.noarch 2024-04-15T15:18:04.884 INFO:teuthology.orchestra.run.smithi105.stdout: bison-3.7.4-5.el9.x86_64 2024-04-15T15:18:04.884 INFO:teuthology.orchestra.run.smithi105.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-15T15:18:04.884 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:18:04.884 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-base-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:18:04.884 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-common-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:18:04.884 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-fuse-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:18:04.884 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-grafana-dashboards-2:19.0.0-2861.g9bf1c4ba.el8.noarch 2024-04-15T15:18:04.884 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-immutable-object-cache-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:18:04.884 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mds-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:18:04.885 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:18:04.885 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-cephadm-2:19.0.0-2861.g9bf1c4ba.el8.noarch 2024-04-15T15:18:04.885 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-dashboard-2:19.0.0-2861.g9bf1c4ba.el8.noarch 2024-04-15T15:18:04.885 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2861.g9bf1c4ba.el8.noarch 2024-04-15T15:18:04.885 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-modules-core-2:19.0.0-2861.g9bf1c4ba.el8.noarch 2024-04-15T15:18:04.885 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-rook-2:19.0.0-2861.g9bf1c4ba.el8.noarch 2024-04-15T15:18:04.885 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mon-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:18:04.885 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-osd-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:18:04.885 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-prometheus-alerts-2:19.0.0-2861.g9bf1c4ba.el8.noarch 2024-04-15T15:18:04.885 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-radosgw-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:18:04.885 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-selinux-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:18:04.885 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-test-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:18:04.885 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-volume-2:19.0.0-2861.g9bf1c4ba.el8.noarch 2024-04-15T15:18:04.886 INFO:teuthology.orchestra.run.smithi105.stdout: cephadm-2:19.0.0-2861.g9bf1c4ba.el8.noarch 2024-04-15T15:18:04.886 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-mirror-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:18:04.886 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-shell-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:18:04.886 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-top-2:19.0.0-2861.g9bf1c4ba.el8.noarch 2024-04-15T15:18:04.886 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-15T15:18:04.886 INFO:teuthology.orchestra.run.smithi105.stdout: flex-2.6.4-9.el9.x86_64 2024-04-15T15:18:04.886 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-15T15:18:04.886 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-15T15:18:04.886 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-15T15:18:04.886 INFO:teuthology.orchestra.run.smithi105.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-15T15:18:04.886 INFO:teuthology.orchestra.run.smithi105.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-15T15:18:04.886 INFO:teuthology.orchestra.run.smithi105.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-15T15:18:04.886 INFO:teuthology.orchestra.run.smithi105.stdout: jq-1.6-16.el9.x86_64 2024-04-15T15:18:04.887 INFO:teuthology.orchestra.run.smithi105.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-15T15:18:04.887 INFO:teuthology.orchestra.run.smithi105.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-15T15:18:04.887 INFO:teuthology.orchestra.run.smithi105.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-15T15:18:04.887 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs-devel-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:18:04.887 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:18:04.887 INFO:teuthology.orchestra.run.smithi105.stdout: libcephsqlite-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:18:04.887 INFO:teuthology.orchestra.run.smithi105.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-15T15:18:04.887 INFO:teuthology.orchestra.run.smithi105.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-15T15:18:04.887 INFO:teuthology.orchestra.run.smithi105.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-15T15:18:04.887 INFO:teuthology.orchestra.run.smithi105.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-15T15:18:04.887 INFO:teuthology.orchestra.run.smithi105.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-15T15:18:04.887 INFO:teuthology.orchestra.run.smithi105.stdout: librados-devel-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:18:04.887 INFO:teuthology.orchestra.run.smithi105.stdout: libradosstriper1-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:18:04.887 INFO:teuthology.orchestra.run.smithi105.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-15T15:18:04.888 INFO:teuthology.orchestra.run.smithi105.stdout: librgw2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:18:04.888 INFO:teuthology.orchestra.run.smithi105.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-15T15:18:04.888 INFO:teuthology.orchestra.run.smithi105.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-15T15:18:04.888 INFO:teuthology.orchestra.run.smithi105.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-15T15:18:04.888 INFO:teuthology.orchestra.run.smithi105.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-15T15:18:04.888 INFO:teuthology.orchestra.run.smithi105.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-15T15:18:04.888 INFO:teuthology.orchestra.run.smithi105.stdout: lua-5.4.4-4.el9.x86_64 2024-04-15T15:18:04.888 INFO:teuthology.orchestra.run.smithi105.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-15T15:18:04.888 INFO:teuthology.orchestra.run.smithi105.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-15T15:18:04.888 INFO:teuthology.orchestra.run.smithi105.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-15T15:18:04.888 INFO:teuthology.orchestra.run.smithi105.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-15T15:18:04.888 INFO:teuthology.orchestra.run.smithi105.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-15T15:18:04.888 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-15T15:18:04.888 INFO:teuthology.orchestra.run.smithi105.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-15T15:18:04.889 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-13.14-1.el9.x86_64 2024-04-15T15:18:04.889 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-15T15:18:04.889 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-15T15:18:04.889 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-15T15:18:04.889 INFO:teuthology.orchestra.run.smithi105.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-15T15:18:04.889 INFO:teuthology.orchestra.run.smithi105.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-15T15:18:04.889 INFO:teuthology.orchestra.run.smithi105.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-15T15:18:04.889 INFO:teuthology.orchestra.run.smithi105.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-15T15:18:04.890 INFO:teuthology.orchestra.run.smithi105.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-15T15:18:04.890 INFO:teuthology.orchestra.run.smithi105.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-15T15:18:04.890 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-15T15:18:04.890 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-argparse-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:18:04.890 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-common-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:18:04.891 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cephfs-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:18:04.891 INFO:teuthology.orchestra.run.smithi105.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-15T15:18:04.891 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-15T15:18:04.891 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-15T15:18:04.891 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-15T15:18:04.891 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-15T15:18:04.892 INFO:teuthology.orchestra.run.smithi105.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-15T15:18:04.892 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-15T15:18:04.892 INFO:teuthology.orchestra.run.smithi105.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-15T15:18:04.892 INFO:teuthology.orchestra.run.smithi105.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-15T15:18:04.892 INFO:teuthology.orchestra.run.smithi105.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-15T15:18:04.892 INFO:teuthology.orchestra.run.smithi105.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-15T15:18:04.893 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-15T15:18:04.893 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-15T15:18:04.893 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-15T15:18:04.893 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-15T15:18:04.893 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-15T15:18:04.893 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-15T15:18:04.893 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-15T15:18:04.894 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-15T15:18:04.894 INFO:teuthology.orchestra.run.smithi105.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-15T15:18:04.894 INFO:teuthology.orchestra.run.smithi105.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-15T15:18:04.894 INFO:teuthology.orchestra.run.smithi105.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-15T15:18:04.894 INFO:teuthology.orchestra.run.smithi105.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-15T15:18:04.894 INFO:teuthology.orchestra.run.smithi105.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-15T15:18:04.895 INFO:teuthology.orchestra.run.smithi105.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-15T15:18:04.895 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-15T15:18:04.895 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-15T15:18:04.896 INFO:teuthology.orchestra.run.smithi105.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-15T15:18:04.896 INFO:teuthology.orchestra.run.smithi105.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-15T15:18:04.896 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-15T15:18:04.896 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ply-3.11-14.el9.noarch 2024-04-15T15:18:04.896 INFO:teuthology.orchestra.run.smithi105.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-15T15:18:04.896 INFO:teuthology.orchestra.run.smithi105.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-15T15:18:04.896 INFO:teuthology.orchestra.run.smithi105.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-15T15:18:04.897 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-15T15:18:04.897 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-15T15:18:04.897 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-15T15:18:04.897 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-15T15:18:04.897 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-15T15:18:04.897 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-15T15:18:04.898 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rados-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:18:04.898 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rbd-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:18:04.898 INFO:teuthology.orchestra.run.smithi105.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-15T15:18:04.898 INFO:teuthology.orchestra.run.smithi105.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-15T15:18:04.899 INFO:teuthology.orchestra.run.smithi105.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-15T15:18:04.899 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rgw-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:18:04.899 INFO:teuthology.orchestra.run.smithi105.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-15T15:18:04.899 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-15T15:18:04.899 INFO:teuthology.orchestra.run.smithi105.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-15T15:18:04.899 INFO:teuthology.orchestra.run.smithi105.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-15T15:18:04.899 INFO:teuthology.orchestra.run.smithi105.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-15T15:18:04.899 INFO:teuthology.orchestra.run.smithi105.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-15T15:18:04.899 INFO:teuthology.orchestra.run.smithi105.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-15T15:18:04.899 INFO:teuthology.orchestra.run.smithi105.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-15T15:18:04.899 INFO:teuthology.orchestra.run.smithi105.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-15T15:18:04.899 INFO:teuthology.orchestra.run.smithi105.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-15T15:18:04.899 INFO:teuthology.orchestra.run.smithi105.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-15T15:18:04.900 INFO:teuthology.orchestra.run.smithi105.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-15T15:18:04.900 INFO:teuthology.orchestra.run.smithi105.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-15T15:18:04.900 INFO:teuthology.orchestra.run.smithi105.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-15T15:18:04.900 INFO:teuthology.orchestra.run.smithi105.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-15T15:18:04.900 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-fuse-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:18:04.900 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-mirror-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:18:04.900 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-nbd-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:18:04.900 INFO:teuthology.orchestra.run.smithi105.stdout: re2-1:20211101-3.el9.x86_64 2024-04-15T15:18:04.900 INFO:teuthology.orchestra.run.smithi105.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-15T15:18:04.900 INFO:teuthology.orchestra.run.smithi105.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-15T15:18:04.900 INFO:teuthology.orchestra.run.smithi105.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-15T15:18:04.900 INFO:teuthology.orchestra.run.smithi105.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-15T15:18:04.900 INFO:teuthology.orchestra.run.smithi105.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-15T15:18:04.901 INFO:teuthology.orchestra.run.smithi105.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-04-15T15:18:04.901 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:18:04.901 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2024-04-15T15:18:04.912 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-04-15T15:18:04.943 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-radosgw-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 149/198 2024-04-15T15:18:04.967 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 149/198 2024-04-15T15:18:04.967 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-15T15:18:04.967 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-15T15:18:04.967 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-15T15:18:04.967 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-15T15:18:04.967 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-15T15:18:05.010 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2861.g9bf1c 150/198 2024-04-15T15:18:05.033 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2861.g9bf1c 150/198 2024-04-15T15:18:05.033 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-15T15:18:05.033 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-15T15:18:05.033 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-15T15:18:05.094 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : cephfs-mirror-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 151/198 2024-04-15T15:18:05.118 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 151/198 2024-04-15T15:18:05.118 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-15T15:18:05.118 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-15T15:18:05.118 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-15T15:18:05.118 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-15T15:18:05.118 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-15T15:18:05.350 DEBUG:teuthology.parallel:result is None 2024-04-15T15:18:05.428 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : rbd-mirror-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 152/198 2024-04-15T15:18:05.453 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 152/198 2024-04-15T15:18:05.453 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-15T15:18:05.453 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-15T15:18:05.453 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-15T15:18:05.453 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-15T15:18:05.453 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-15T15:18:10.024 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-test-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 153/198 2024-04-15T15:18:10.065 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : rbd-fuse-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 154/198 2024-04-15T15:18:10.096 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : rbd-nbd-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 155/198 2024-04-15T15:18:10.123 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-04-15T15:18:10.317 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : cephfs-shell-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 157/198 2024-04-15T15:18:10.430 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-04-15T15:18:10.471 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-fuse-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 159/198 2024-04-15T15:18:10.536 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-04-15T15:18:10.596 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-04-15T15:18:10.639 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-04-15T15:18:10.671 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libcephfs-devel-2:19.0.0-2861.g9bf1c4ba.el8.x86_ 163/198 2024-04-15T15:18:10.704 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-04-15T15:18:10.734 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : NetworkManager-team-1:1.47.4-1.el9.x86_64 165/198 2024-04-15T15:18:10.810 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : NetworkManager-tui-1:1.47.4-1.el9.x86_64 166/198 2024-04-15T15:18:10.879 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-04-15T15:18:10.916 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-04-15T15:18:11.490 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-15T15:18:11.540 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-15T15:18:11.562 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-15T15:18:12.404 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : cephfs-top-2:19.0.0-2861.g9bf1c4ba.el8.noarch 170/198 2024-04-15T15:18:12.485 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-04-15T15:18:12.534 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-04-15T15:18:12.626 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-04-15T15:18:12.664 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-04-15T15:18:12.688 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-04-15T15:18:12.722 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-04-15T15:18:12.749 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-04-15T15:18:12.749 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-15T15:18:12.837 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-15T15:18:12.866 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-04-15T15:18:12.867 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-15T15:18:12.976 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-15T15:18:13.008 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-04-15T15:18:13.035 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-04-15T15:18:13.061 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-04-15T15:18:13.086 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-04-15T15:18:13.121 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-04-15T15:18:13.167 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-04-15T15:18:13.207 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-04-15T15:18:13.232 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-04-15T15:18:13.258 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-04-15T15:18:13.283 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-04-15T15:18:13.308 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-04-15T15:18:13.315 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-15T15:18:13.315 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-15T15:18:13.503 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-15T15:18:13.503 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-15T15:18:13.594 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-15T15:18:13.626 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-04-15T15:18:13.653 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-04-15T15:18:13.678 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-04-15T15:18:13.703 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-04-15T15:18:13.769 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-15T15:18:19.476 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-15T15:18:19.477 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 1/198 2024-04-15T15:18:19.477 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-base-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2/198 2024-04-15T15:18:19.477 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-common-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 3/198 2024-04-15T15:18:19.477 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-fuse-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 4/198 2024-04-15T15:18:19.477 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2861.g9bf1c 5/198 2024-04-15T15:18:19.477 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mds-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 6/198 2024-04-15T15:18:19.477 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 7/198 2024-04-15T15:18:19.477 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mon-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 8/198 2024-04-15T15:18:19.477 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-osd-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 9/198 2024-04-15T15:18:19.477 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-radosgw-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 10/198 2024-04-15T15:18:19.477 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-selinux-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 11/198 2024-04-15T15:18:19.478 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-test-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 12/198 2024-04-15T15:18:19.478 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : cephfs-mirror-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 13/198 2024-04-15T15:18:19.478 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : cephfs-shell-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 14/198 2024-04-15T15:18:19.478 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libcephfs-devel-2:19.0.0-2861.g9bf1c4ba.el8.x86_ 15/198 2024-04-15T15:18:19.478 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libcephfs2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 16/198 2024-04-15T15:18:19.478 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libcephsqlite-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 17/198 2024-04-15T15:18:19.478 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librados-devel-2:19.0.0-2861.g9bf1c4ba.el8.x86_6 18/198 2024-04-15T15:18:19.478 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libradosstriper1-2:19.0.0-2861.g9bf1c4ba.el8.x86 19/198 2024-04-15T15:18:19.478 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librgw2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 20/198 2024-04-15T15:18:19.478 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2861.g9bf1c4ba.el 21/198 2024-04-15T15:18:19.478 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-ceph-common-2:19.0.0-2861.g9bf1c4ba.el8. 22/198 2024-04-15T15:18:19.478 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cephfs-2:19.0.0-2861.g9bf1c4ba.el8.x86_6 23/198 2024-04-15T15:18:19.478 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-rados-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 24/198 2024-04-15T15:18:19.479 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-rbd-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 25/198 2024-04-15T15:18:19.479 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-rgw-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 26/198 2024-04-15T15:18:19.479 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : rbd-fuse-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 27/198 2024-04-15T15:18:19.479 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : rbd-mirror-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 28/198 2024-04-15T15:18:19.479 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : rbd-nbd-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 29/198 2024-04-15T15:18:19.479 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2861.g9bf1c4ba. 30/198 2024-04-15T15:18:19.479 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2861.g9bf1c4ba.el8.noa 31/198 2024-04-15T15:18:19.479 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2861.g9bf1c4ba.el8.n 32/198 2024-04-15T15:18:19.479 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2861.g9bf 33/198 2024-04-15T15:18:19.479 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2861.g9bf1c4ba.el 34/198 2024-04-15T15:18:19.479 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2861.g9bf1c4ba.el8.noarch 35/198 2024-04-15T15:18:19.479 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2861.g9bf1c4ba.e 36/198 2024-04-15T15:18:19.479 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-volume-2:19.0.0-2861.g9bf1c4ba.el8.noarch 37/198 2024-04-15T15:18:19.479 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : cephadm-2:19.0.0-2861.g9bf1c4ba.el8.noarch 38/198 2024-04-15T15:18:19.480 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : cephfs-top-2:19.0.0-2861.g9bf1c4ba.el8.noarch 39/198 2024-04-15T15:18:19.480 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-04-15T15:18:19.480 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-04-15T15:18:19.480 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-04-15T15:18:19.480 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-04-15T15:18:19.480 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-04-15T15:18:19.480 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-04-15T15:18:19.480 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-04-15T15:18:19.480 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-04-15T15:18:19.480 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-04-15T15:18:19.481 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-04-15T15:18:19.481 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-04-15T15:18:19.481 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-04-15T15:18:19.481 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-04-15T15:18:19.481 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-04-15T15:18:19.481 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-04-15T15:18:19.481 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-04-15T15:18:19.481 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-04-15T15:18:19.481 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-04-15T15:18:19.481 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : postgresql-13.14-1.el9.x86_64 58/198 2024-04-15T15:18:19.481 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 59/198 2024-04-15T15:18:19.481 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 60/198 2024-04-15T15:18:19.481 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 61/198 2024-04-15T15:18:19.482 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-04-15T15:18:19.482 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-04-15T15:18:19.482 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-04-15T15:18:19.482 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-04-15T15:18:19.482 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-04-15T15:18:19.482 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-04-15T15:18:19.482 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-04-15T15:18:19.482 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-04-15T15:18:19.482 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-04-15T15:18:19.482 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-04-15T15:18:19.482 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-04-15T15:18:19.482 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-04-15T15:18:19.482 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-04-15T15:18:19.482 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-04-15T15:18:19.483 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-04-15T15:18:19.483 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-04-15T15:18:19.483 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-04-15T15:18:19.483 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-04-15T15:18:19.483 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-04-15T15:18:19.483 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-04-15T15:18:19.483 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 82/198 2024-04-15T15:18:19.483 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 83/198 2024-04-15T15:18:19.483 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 84/198 2024-04-15T15:18:19.483 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 85/198 2024-04-15T15:18:19.483 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 86/198 2024-04-15T15:18:19.483 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 87/198 2024-04-15T15:18:19.483 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 88/198 2024-04-15T15:18:19.484 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-15T15:18:19.484 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 90/198 2024-04-15T15:18:19.484 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 91/198 2024-04-15T15:18:19.484 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-ply-3.11-14.el9.noarch 92/198 2024-04-15T15:18:19.484 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 93/198 2024-04-15T15:18:19.484 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/198 2024-04-15T15:18:19.484 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : jq-1.6-16.el9.x86_64 95/198 2024-04-15T15:18:19.484 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 96/198 2024-04-15T15:18:19.484 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 97/198 2024-04-15T15:18:19.484 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 98/198 2024-04-15T15:18:19.484 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 99/198 2024-04-15T15:18:19.484 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 100/198 2024-04-15T15:18:19.484 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 101/198 2024-04-15T15:18:19.484 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 102/198 2024-04-15T15:18:19.485 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 103/198 2024-04-15T15:18:19.485 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-04-15T15:18:19.485 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-04-15T15:18:19.485 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-04-15T15:18:19.485 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-04-15T15:18:19.485 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-04-15T15:18:19.485 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-04-15T15:18:19.485 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-04-15T15:18:19.485 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-04-15T15:18:19.485 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-04-15T15:18:19.485 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-04-15T15:18:19.485 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-04-15T15:18:19.485 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-15T15:18:19.485 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-04-15T15:18:19.486 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-04-15T15:18:19.486 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-04-15T15:18:19.486 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-04-15T15:18:19.486 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-04-15T15:18:19.486 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-04-15T15:18:19.486 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-04-15T15:18:19.486 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-04-15T15:18:19.486 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-04-15T15:18:19.486 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-04-15T15:18:19.486 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-04-15T15:18:19.486 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-04-15T15:18:19.486 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-04-15T15:18:19.486 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-04-15T15:18:19.486 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-04-15T15:18:19.487 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-04-15T15:18:19.487 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-04-15T15:18:19.487 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-04-15T15:18:19.487 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-04-15T15:18:19.487 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-04-15T15:18:19.487 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-04-15T15:18:19.487 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-04-15T15:18:19.487 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-04-15T15:18:19.487 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-04-15T15:18:19.487 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-04-15T15:18:19.487 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-04-15T15:18:19.487 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-04-15T15:18:19.487 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-04-15T15:18:19.487 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-04-15T15:18:19.488 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-04-15T15:18:19.488 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-04-15T15:18:19.488 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-04-15T15:18:19.488 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-04-15T15:18:19.488 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-04-15T15:18:19.488 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-04-15T15:18:19.488 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librados2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 151/198 2024-04-15T15:18:19.488 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-04-15T15:18:19.488 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librbd1-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 153/198 2024-04-15T15:18:19.488 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-04-15T15:18:19.488 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 155/198 2024-04-15T15:18:19.488 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-04-15T15:18:19.488 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 157/198 2024-04-15T15:18:19.488 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-04-15T15:18:19.488 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 159/198 2024-04-15T15:18:19.489 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-04-15T15:18:19.489 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/198 2024-04-15T15:18:19.489 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-04-15T15:18:19.489 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 163/198 2024-04-15T15:18:19.489 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 164/198 2024-04-15T15:18:19.489 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-04-15T15:18:19.489 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-04-15T15:18:19.489 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-04-15T15:18:19.489 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-04-15T15:18:19.489 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-04-15T15:18:19.489 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-04-15T15:18:19.489 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-04-15T15:18:19.489 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-04-15T15:18:19.489 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-04-15T15:18:19.490 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-04-15T15:18:19.490 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-04-15T15:18:19.490 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-04-15T15:18:19.490 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-04-15T15:18:19.490 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-04-15T15:18:19.490 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-04-15T15:18:19.490 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-04-15T15:18:19.490 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-04-15T15:18:19.490 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-04-15T15:18:19.490 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-04-15T15:18:19.490 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-04-15T15:18:19.490 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-04-15T15:18:19.490 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-04-15T15:18:19.490 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-04-15T15:18:19.491 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-04-15T15:18:19.491 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-04-15T15:18:19.491 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-04-15T15:18:19.491 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-04-15T15:18:19.491 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-04-15T15:18:19.491 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-04-15T15:18:19.491 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-04-15T15:18:19.491 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-04-15T15:18:19.491 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-04-15T15:18:19.491 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-04-15T15:18:22.717 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-04-15T15:18:22.717 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-15T15:18:22.717 INFO:teuthology.orchestra.run.smithi154.stdout:Upgraded: 2024-04-15T15:18:22.717 INFO:teuthology.orchestra.run.smithi154.stdout: NetworkManager-1:1.47.4-1.el9.x86_64 2024-04-15T15:18:22.717 INFO:teuthology.orchestra.run.smithi154.stdout: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 2024-04-15T15:18:22.717 INFO:teuthology.orchestra.run.smithi154.stdout: NetworkManager-team-1:1.47.4-1.el9.x86_64 2024-04-15T15:18:22.717 INFO:teuthology.orchestra.run.smithi154.stdout: NetworkManager-tui-1:1.47.4-1.el9.x86_64 2024-04-15T15:18:22.718 INFO:teuthology.orchestra.run.smithi154.stdout: acl-2.3.1-4.el9.x86_64 2024-04-15T15:18:22.718 INFO:teuthology.orchestra.run.smithi154.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-04-15T15:18:22.718 INFO:teuthology.orchestra.run.smithi154.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-04-15T15:18:22.718 INFO:teuthology.orchestra.run.smithi154.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-04-15T15:18:22.718 INFO:teuthology.orchestra.run.smithi154.stdout: libacl-2.3.1-4.el9.x86_64 2024-04-15T15:18:22.718 INFO:teuthology.orchestra.run.smithi154.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-04-15T15:18:22.718 INFO:teuthology.orchestra.run.smithi154.stdout: libblkid-2.37.4-18.el9.x86_64 2024-04-15T15:18:22.718 INFO:teuthology.orchestra.run.smithi154.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-04-15T15:18:22.718 INFO:teuthology.orchestra.run.smithi154.stdout: libmount-2.37.4-18.el9.x86_64 2024-04-15T15:18:22.718 INFO:teuthology.orchestra.run.smithi154.stdout: librados2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:18:22.718 INFO:teuthology.orchestra.run.smithi154.stdout: librbd1-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:18:22.718 INFO:teuthology.orchestra.run.smithi154.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-04-15T15:18:22.718 INFO:teuthology.orchestra.run.smithi154.stdout: libuuid-2.37.4-18.el9.x86_64 2024-04-15T15:18:22.719 INFO:teuthology.orchestra.run.smithi154.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-04-15T15:18:22.719 INFO:teuthology.orchestra.run.smithi154.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-04-15T15:18:22.719 INFO:teuthology.orchestra.run.smithi154.stdout: util-linux-2.37.4-18.el9.x86_64 2024-04-15T15:18:22.719 INFO:teuthology.orchestra.run.smithi154.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-04-15T15:18:22.719 INFO:teuthology.orchestra.run.smithi154.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-04-15T15:18:22.719 INFO:teuthology.orchestra.run.smithi154.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-04-15T15:18:22.719 INFO:teuthology.orchestra.run.smithi154.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-04-15T15:18:22.719 INFO:teuthology.orchestra.run.smithi154.stdout:Installed: 2024-04-15T15:18:22.719 INFO:teuthology.orchestra.run.smithi154.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-15T15:18:22.719 INFO:teuthology.orchestra.run.smithi154.stdout: automake-1.16.2-8.el9.noarch 2024-04-15T15:18:22.719 INFO:teuthology.orchestra.run.smithi154.stdout: bison-3.7.4-5.el9.x86_64 2024-04-15T15:18:22.719 INFO:teuthology.orchestra.run.smithi154.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-15T15:18:22.719 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:18:22.719 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-base-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:18:22.720 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-common-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:18:22.720 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-fuse-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:18:22.720 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-grafana-dashboards-2:19.0.0-2861.g9bf1c4ba.el8.noarch 2024-04-15T15:18:22.720 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-immutable-object-cache-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:18:22.720 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mds-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:18:22.720 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:18:22.720 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-cephadm-2:19.0.0-2861.g9bf1c4ba.el8.noarch 2024-04-15T15:18:22.720 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-dashboard-2:19.0.0-2861.g9bf1c4ba.el8.noarch 2024-04-15T15:18:22.720 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2861.g9bf1c4ba.el8.noarch 2024-04-15T15:18:22.720 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-modules-core-2:19.0.0-2861.g9bf1c4ba.el8.noarch 2024-04-15T15:18:22.720 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-rook-2:19.0.0-2861.g9bf1c4ba.el8.noarch 2024-04-15T15:18:22.720 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mon-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:18:22.720 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-osd-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:18:22.721 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-prometheus-alerts-2:19.0.0-2861.g9bf1c4ba.el8.noarch 2024-04-15T15:18:22.721 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-radosgw-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:18:22.721 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-selinux-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:18:22.721 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-test-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:18:22.721 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-volume-2:19.0.0-2861.g9bf1c4ba.el8.noarch 2024-04-15T15:18:22.721 INFO:teuthology.orchestra.run.smithi154.stdout: cephadm-2:19.0.0-2861.g9bf1c4ba.el8.noarch 2024-04-15T15:18:22.721 INFO:teuthology.orchestra.run.smithi154.stdout: cephfs-mirror-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:18:22.721 INFO:teuthology.orchestra.run.smithi154.stdout: cephfs-shell-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:18:22.721 INFO:teuthology.orchestra.run.smithi154.stdout: cephfs-top-2:19.0.0-2861.g9bf1c4ba.el8.noarch 2024-04-15T15:18:22.721 INFO:teuthology.orchestra.run.smithi154.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-15T15:18:22.722 INFO:teuthology.orchestra.run.smithi154.stdout: flex-2.6.4-9.el9.x86_64 2024-04-15T15:18:22.722 INFO:teuthology.orchestra.run.smithi154.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-15T15:18:22.722 INFO:teuthology.orchestra.run.smithi154.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-15T15:18:22.722 INFO:teuthology.orchestra.run.smithi154.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-15T15:18:22.722 INFO:teuthology.orchestra.run.smithi154.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-15T15:18:22.722 INFO:teuthology.orchestra.run.smithi154.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-15T15:18:22.722 INFO:teuthology.orchestra.run.smithi154.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-15T15:18:22.722 INFO:teuthology.orchestra.run.smithi154.stdout: jq-1.6-16.el9.x86_64 2024-04-15T15:18:22.722 INFO:teuthology.orchestra.run.smithi154.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-15T15:18:22.723 INFO:teuthology.orchestra.run.smithi154.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-15T15:18:22.723 INFO:teuthology.orchestra.run.smithi154.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-15T15:18:22.723 INFO:teuthology.orchestra.run.smithi154.stdout: libcephfs-devel-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:18:22.723 INFO:teuthology.orchestra.run.smithi154.stdout: libcephfs2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:18:22.723 INFO:teuthology.orchestra.run.smithi154.stdout: libcephsqlite-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:18:22.723 INFO:teuthology.orchestra.run.smithi154.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-15T15:18:22.723 INFO:teuthology.orchestra.run.smithi154.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-15T15:18:22.724 INFO:teuthology.orchestra.run.smithi154.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-15T15:18:22.724 INFO:teuthology.orchestra.run.smithi154.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-15T15:18:22.724 INFO:teuthology.orchestra.run.smithi154.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-15T15:18:22.724 INFO:teuthology.orchestra.run.smithi154.stdout: librados-devel-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:18:22.724 INFO:teuthology.orchestra.run.smithi154.stdout: libradosstriper1-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:18:22.724 INFO:teuthology.orchestra.run.smithi154.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-15T15:18:22.724 INFO:teuthology.orchestra.run.smithi154.stdout: librgw2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:18:22.725 INFO:teuthology.orchestra.run.smithi154.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-15T15:18:22.725 INFO:teuthology.orchestra.run.smithi154.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-15T15:18:22.725 INFO:teuthology.orchestra.run.smithi154.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-15T15:18:22.725 INFO:teuthology.orchestra.run.smithi154.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-15T15:18:22.725 INFO:teuthology.orchestra.run.smithi154.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-15T15:18:22.725 INFO:teuthology.orchestra.run.smithi154.stdout: lua-5.4.4-4.el9.x86_64 2024-04-15T15:18:22.726 INFO:teuthology.orchestra.run.smithi154.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-15T15:18:22.726 INFO:teuthology.orchestra.run.smithi154.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-15T15:18:22.726 INFO:teuthology.orchestra.run.smithi154.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-15T15:18:22.726 INFO:teuthology.orchestra.run.smithi154.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-15T15:18:22.726 INFO:teuthology.orchestra.run.smithi154.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-15T15:18:22.727 INFO:teuthology.orchestra.run.smithi154.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-15T15:18:22.727 INFO:teuthology.orchestra.run.smithi154.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-15T15:18:22.727 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-13.14-1.el9.x86_64 2024-04-15T15:18:22.727 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-15T15:18:22.727 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-15T15:18:22.727 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-15T15:18:22.728 INFO:teuthology.orchestra.run.smithi154.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-15T15:18:22.728 INFO:teuthology.orchestra.run.smithi154.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-15T15:18:22.728 INFO:teuthology.orchestra.run.smithi154.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-15T15:18:22.728 INFO:teuthology.orchestra.run.smithi154.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-15T15:18:22.728 INFO:teuthology.orchestra.run.smithi154.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-15T15:18:22.728 INFO:teuthology.orchestra.run.smithi154.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-15T15:18:22.729 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-15T15:18:22.729 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ceph-argparse-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:18:22.729 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ceph-common-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:18:22.729 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cephfs-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:18:22.729 INFO:teuthology.orchestra.run.smithi154.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-15T15:18:22.730 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-15T15:18:22.730 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-15T15:18:22.730 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-15T15:18:22.730 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-15T15:18:22.730 INFO:teuthology.orchestra.run.smithi154.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-15T15:18:22.730 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-15T15:18:22.731 INFO:teuthology.orchestra.run.smithi154.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-15T15:18:22.731 INFO:teuthology.orchestra.run.smithi154.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-15T15:18:22.731 INFO:teuthology.orchestra.run.smithi154.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-15T15:18:22.731 INFO:teuthology.orchestra.run.smithi154.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-15T15:18:22.731 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-15T15:18:22.731 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-15T15:18:22.732 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-15T15:18:22.732 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-15T15:18:22.732 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-15T15:18:22.732 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-15T15:18:22.732 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-15T15:18:22.732 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-15T15:18:22.733 INFO:teuthology.orchestra.run.smithi154.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-15T15:18:22.733 INFO:teuthology.orchestra.run.smithi154.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-15T15:18:22.733 INFO:teuthology.orchestra.run.smithi154.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-15T15:18:22.733 INFO:teuthology.orchestra.run.smithi154.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-15T15:18:22.733 INFO:teuthology.orchestra.run.smithi154.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-15T15:18:22.733 INFO:teuthology.orchestra.run.smithi154.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-15T15:18:22.734 INFO:teuthology.orchestra.run.smithi154.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-15T15:18:22.734 INFO:teuthology.orchestra.run.smithi154.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-15T15:18:22.734 INFO:teuthology.orchestra.run.smithi154.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-15T15:18:22.734 INFO:teuthology.orchestra.run.smithi154.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-15T15:18:22.734 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-15T15:18:22.734 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ply-3.11-14.el9.noarch 2024-04-15T15:18:22.735 INFO:teuthology.orchestra.run.smithi154.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-15T15:18:22.735 INFO:teuthology.orchestra.run.smithi154.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-15T15:18:22.735 INFO:teuthology.orchestra.run.smithi154.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-15T15:18:22.735 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-15T15:18:22.735 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-15T15:18:22.735 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-15T15:18:22.736 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-15T15:18:22.736 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-15T15:18:22.736 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-15T15:18:22.736 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rados-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:18:22.736 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rbd-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:18:22.736 INFO:teuthology.orchestra.run.smithi154.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-15T15:18:22.737 INFO:teuthology.orchestra.run.smithi154.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-15T15:18:22.737 INFO:teuthology.orchestra.run.smithi154.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-15T15:18:22.737 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rgw-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:18:22.737 INFO:teuthology.orchestra.run.smithi154.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-15T15:18:22.737 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-15T15:18:22.737 INFO:teuthology.orchestra.run.smithi154.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-15T15:18:22.738 INFO:teuthology.orchestra.run.smithi154.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-15T15:18:22.738 INFO:teuthology.orchestra.run.smithi154.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-15T15:18:22.738 INFO:teuthology.orchestra.run.smithi154.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-15T15:18:22.738 INFO:teuthology.orchestra.run.smithi154.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-15T15:18:22.738 INFO:teuthology.orchestra.run.smithi154.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-15T15:18:22.739 INFO:teuthology.orchestra.run.smithi154.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-15T15:18:22.739 INFO:teuthology.orchestra.run.smithi154.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-15T15:18:22.739 INFO:teuthology.orchestra.run.smithi154.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-15T15:18:22.739 INFO:teuthology.orchestra.run.smithi154.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-15T15:18:22.739 INFO:teuthology.orchestra.run.smithi154.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-15T15:18:22.739 INFO:teuthology.orchestra.run.smithi154.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-15T15:18:22.740 INFO:teuthology.orchestra.run.smithi154.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-15T15:18:22.740 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-fuse-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:18:22.740 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-mirror-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:18:22.740 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-nbd-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:18:22.740 INFO:teuthology.orchestra.run.smithi154.stdout: re2-1:20211101-3.el9.x86_64 2024-04-15T15:18:22.740 INFO:teuthology.orchestra.run.smithi154.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-15T15:18:22.741 INFO:teuthology.orchestra.run.smithi154.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-15T15:18:22.741 INFO:teuthology.orchestra.run.smithi154.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-15T15:18:22.741 INFO:teuthology.orchestra.run.smithi154.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-15T15:18:22.741 INFO:teuthology.orchestra.run.smithi154.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-15T15:18:22.741 INFO:teuthology.orchestra.run.smithi154.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-04-15T15:18:22.741 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-15T15:18:22.741 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2024-04-15T15:18:23.208 DEBUG:teuthology.parallel:result is None 2024-04-15T15:18:23.209 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9bf1c4ba967dd24077be6e5b87edda4ca87a6e14 2024-04-15T15:18:23.439 DEBUG:teuthology.orchestra.run.smithi105:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-15T15:18:23.466 INFO:teuthology.orchestra.run.smithi105.stdout:19.0.0-2861.g9bf1c4ba.el8 2024-04-15T15:18:23.467 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2861.g9bf1c4ba.el8 2024-04-15T15:18:23.467 INFO:teuthology.task.install:The correct ceph version 19.0.0-2861.g9bf1c4ba is installed. 2024-04-15T15:18:23.468 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9bf1c4ba967dd24077be6e5b87edda4ca87a6e14 2024-04-15T15:18:23.683 DEBUG:teuthology.orchestra.run.smithi154:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-15T15:18:23.708 INFO:teuthology.orchestra.run.smithi154.stdout:19.0.0-2861.g9bf1c4ba.el8 2024-04-15T15:18:23.708 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2861.g9bf1c4ba.el8 2024-04-15T15:18:23.708 INFO:teuthology.task.install:The correct ceph version 19.0.0-2861.g9bf1c4ba is installed. 2024-04-15T15:18:23.710 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-15T15:18:23.711 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-04-15T15:18:23.711 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-15T15:18:23.747 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-15T15:18:23.747 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-15T15:18:23.782 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-15T15:18:23.783 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-04-15T15:18:23.783 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/usr/bin/daemon-helper 2024-04-15T15:18:23.819 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-15T15:18:23.890 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-15T15:18:23.890 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/usr/bin/daemon-helper 2024-04-15T15:18:23.921 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-15T15:18:23.992 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-15T15:18:23.993 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-04-15T15:18:23.993 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-15T15:18:24.021 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-15T15:18:24.090 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-15T15:18:24.090 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-15T15:18:24.121 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-15T15:18:24.192 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-15T15:18:24.192 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-04-15T15:18:24.192 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/usr/bin/stdin-killer 2024-04-15T15:18:24.222 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-15T15:18:24.291 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-15T15:18:24.292 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/usr/bin/stdin-killer 2024-04-15T15:18:24.322 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-15T15:18:24.394 INFO:teuthology.run_tasks:Running task ceph... 2024-04-15T15:18:24.487 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-15T15:18:24.487 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 777 /var/log/ceph 2024-04-15T15:18:24.490 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 777 /var/log/ceph 2024-04-15T15:18:24.520 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-15T15:18:24.520 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-15T15:18:24.562 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-15T15:18:24.591 INFO:tasks.ceph:Creating extra log directories... 2024-04-15T15:18:24.592 DEBUG:teuthology.orchestra.run.smithi105:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-15T15:18:24.630 DEBUG:teuthology.orchestra.run.smithi154:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-15T15:18:24.667 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-15T15:18:24.668 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-15T15:18:24.668 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-15T15:18:24.668 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-04-15T15:18:24.668 DEBUG:teuthology.orchestra.run.smithi105:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-15T15:18:24.712 DEBUG:teuthology.orchestra.run.smithi105:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-15T15:18:24.833 DEBUG:teuthology.orchestra.run.smithi105:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-15T15:18:24.862 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-15T15:18:24.862 DEBUG:teuthology.orchestra.run.smithi154:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-15T15:18:24.878 DEBUG:teuthology.orchestra.run.smithi154:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-15T15:18:24.998 DEBUG:teuthology.orchestra.run.smithi154:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-15T15:18:25.032 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-15T15:18:25.032 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow request'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-15T15:18:25.032 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/leonidus-2024-04-15_14:54:08-fs-wip-lusov-quiesce-xlock-distro-default-smithi/7657132', 'branch': 'wip-lusov-quiesce-xlock', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/quiesce}', 'email': None, 'first_in_suite': False, 'job_id': '7657132', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'leonidus-2024-04-15_14:54:08-fs-wip-lusov-quiesce-xlock-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-lusov-quiesce-xlock'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow request'], 'sha1': '9bf1c4ba967dd24077be6e5b87edda4ca87a6e14'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '9bf1c4ba967dd24077be6e5b87edda4ca87a6e14'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 1, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-lusov-quiesce-xlock', 'sha1': '9bf1c4ba967dd24077be6e5b87edda4ca87a6e14'}}, 'owner': 'scheduled_leonidus@teuthology', 'priority': 70, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 5929, 'sha1': '9bf1c4ba967dd24077be6e5b87edda4ca87a6e14', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'wip-lusov-quiesce-xlock', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '9bf1c4ba967dd24077be6e5b87edda4ca87a6e14', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi105.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGtK/GTXibZ8m3DeYPVGTDDt7HFJ9BpvJbR1yk41DfL7oxO7ofgneKZl4qsIl1MlnWxCCYjI3Wgx1TP31Ne8oRM=', 'smithi154.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEfwTt4MW0X5hlmNE9lb0QI3waz4vvcx9Y/jhtC4t1Ux/qKV/DMJmxst2RT3YbWTceKKt3owaT80t6MOstAUvfs='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '9bf1c4ba967dd24077be6e5b87edda4ca87a6e14'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow request'], 'sha1': '9bf1c4ba967dd24077be6e5b87edda4ca87a6e14', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_quiesce']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '6c637841c215537a4502385240412f1966e0faab', 'timestamp': '2024-04-15_14:54:08', 'tube': 'smithi', 'user': 'leonidus', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-15T15:18:25.033 DEBUG:teuthology.orchestra.run.smithi105:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-15T15:18:25.035 DEBUG:teuthology.orchestra.run.smithi154:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-15T15:18:25.083 DEBUG:teuthology.orchestra.run.smithi105:> sudo install -d -m0777 -- /var/run/ceph 2024-04-15T15:18:25.091 DEBUG:teuthology.orchestra.run.smithi154:> sudo install -d -m0777 -- /var/run/ceph 2024-04-15T15:18:25.153 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-04-15T15:18:25.153 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/scratch_devs of=/dev/stdout 2024-04-15T15:18:25.174 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-15T15:18:25.174 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_1 2024-04-15T15:18:25.230 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-15T15:18:25.231 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-15T15:18:25.231 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-15T15:18:25.231 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-15T15:18:25.231 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2024-04-15T15:18:25.231 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2024-04-15 15:17:56.787226695 +0000 2024-04-15T15:18:25.231 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2024-04-15 15:15:51.241857013 +0000 2024-04-15T15:18:25.231 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2024-04-15 15:15:51.241857013 +0000 2024-04-15T15:18:25.231 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: 2024-04-15 15:15:51.241857013 +0000 2024-04-15T15:18:25.231 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-15T15:18:25.297 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2024-04-15T15:18:25.297 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2024-04-15T15:18:25.297 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000206212 s, 2.5 MB/s 2024-04-15T15:18:25.298 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-15T15:18:25.355 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_2 2024-04-15T15:18:25.411 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-15T15:18:25.411 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-15T15:18:25.411 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-15T15:18:25.411 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-15T15:18:25.411 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2024-04-15T15:18:25.411 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2024-04-15 15:17:56.787226695 +0000 2024-04-15T15:18:25.412 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2024-04-15 15:15:51.579850315 +0000 2024-04-15T15:18:25.412 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2024-04-15 15:15:51.579850315 +0000 2024-04-15T15:18:25.412 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: 2024-04-15 15:15:51.579850315 +0000 2024-04-15T15:18:25.412 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-15T15:18:25.478 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2024-04-15T15:18:25.479 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2024-04-15T15:18:25.479 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000181962 s, 2.8 MB/s 2024-04-15T15:18:25.480 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-15T15:18:25.536 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_3 2024-04-15T15:18:25.592 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-15T15:18:25.592 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-15T15:18:25.593 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-15T15:18:25.593 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-15T15:18:25.593 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2024-04-15T15:18:25.593 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2024-04-15 15:17:56.787226695 +0000 2024-04-15T15:18:25.593 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2024-04-15 15:15:51.915843658 +0000 2024-04-15T15:18:25.593 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2024-04-15 15:15:51.915843658 +0000 2024-04-15T15:18:25.593 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: 2024-04-15 15:15:51.915843658 +0000 2024-04-15T15:18:25.593 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-15T15:18:25.659 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2024-04-15T15:18:25.659 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2024-04-15T15:18:25.659 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000200421 s, 2.6 MB/s 2024-04-15T15:18:25.660 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-15T15:18:25.716 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_4 2024-04-15T15:18:25.771 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-15T15:18:25.772 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-15T15:18:25.772 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-15T15:18:25.772 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-15T15:18:25.772 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2024-04-15T15:18:25.772 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2024-04-15 15:17:56.788226675 +0000 2024-04-15T15:18:25.772 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2024-04-15 15:15:52.244837139 +0000 2024-04-15T15:18:25.772 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2024-04-15 15:15:52.244837139 +0000 2024-04-15T15:18:25.772 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: 2024-04-15 15:15:52.244837139 +0000 2024-04-15T15:18:25.772 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-15T15:18:25.838 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2024-04-15T15:18:25.838 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2024-04-15T15:18:25.838 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000154258 s, 3.3 MB/s 2024-04-15T15:18:25.839 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-15T15:18:25.895 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-15T15:18:25.896 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-15T15:18:25.896 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/scratch_devs of=/dev/stdout 2024-04-15T15:18:25.913 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-15T15:18:25.914 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_1 2024-04-15T15:18:25.971 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-15T15:18:25.971 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-15T15:18:25.971 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-15T15:18:25.971 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-15T15:18:25.971 INFO:teuthology.orchestra.run.smithi154.stdout:Context: system_u:object_r:device_t:s0 2024-04-15T15:18:25.971 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2024-04-15 15:18:14.116883405 +0000 2024-04-15T15:18:25.971 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2024-04-15 15:15:51.254559483 +0000 2024-04-15T15:18:25.971 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2024-04-15 15:15:51.254559483 +0000 2024-04-15T15:18:25.972 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: 2024-04-15 15:15:51.254559483 +0000 2024-04-15T15:18:25.972 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-15T15:18:26.041 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2024-04-15T15:18:26.041 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2024-04-15T15:18:26.042 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000192004 s, 2.7 MB/s 2024-04-15T15:18:26.043 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-15T15:18:26.102 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_2 2024-04-15T15:18:26.160 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-15T15:18:26.161 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-15T15:18:26.161 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-15T15:18:26.161 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-15T15:18:26.161 INFO:teuthology.orchestra.run.smithi154.stdout:Context: system_u:object_r:device_t:s0 2024-04-15T15:18:26.161 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2024-04-15 15:18:14.116883405 +0000 2024-04-15T15:18:26.161 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2024-04-15 15:15:51.598549707 +0000 2024-04-15T15:18:26.161 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2024-04-15 15:15:51.598549707 +0000 2024-04-15T15:18:26.161 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: 2024-04-15 15:15:51.598549707 +0000 2024-04-15T15:18:26.162 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-15T15:18:26.229 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2024-04-15T15:18:26.229 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2024-04-15T15:18:26.229 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.00019919 s, 2.6 MB/s 2024-04-15T15:18:26.230 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-15T15:18:26.287 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_3 2024-04-15T15:18:26.344 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-15T15:18:26.344 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-15T15:18:26.344 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-04-15T15:18:26.344 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-15T15:18:26.344 INFO:teuthology.orchestra.run.smithi154.stdout:Context: system_u:object_r:device_t:s0 2024-04-15T15:18:26.344 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2024-04-15 15:18:14.116883405 +0000 2024-04-15T15:18:26.344 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2024-04-15 15:15:51.934540157 +0000 2024-04-15T15:18:26.344 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2024-04-15 15:15:51.934540157 +0000 2024-04-15T15:18:26.344 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: 2024-04-15 15:15:51.934540157 +0000 2024-04-15T15:18:26.345 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-15T15:18:26.412 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2024-04-15T15:18:26.412 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2024-04-15T15:18:26.412 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000199702 s, 2.6 MB/s 2024-04-15T15:18:26.413 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-15T15:18:26.470 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_4 2024-04-15T15:18:26.526 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-15T15:18:26.526 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-15T15:18:26.526 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-15T15:18:26.526 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-15T15:18:26.526 INFO:teuthology.orchestra.run.smithi154.stdout:Context: system_u:object_r:device_t:s0 2024-04-15T15:18:26.526 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2024-04-15 15:18:14.117883378 +0000 2024-04-15T15:18:26.526 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2024-04-15 15:15:52.270530608 +0000 2024-04-15T15:18:26.526 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2024-04-15 15:15:52.270530608 +0000 2024-04-15T15:18:26.526 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: 2024-04-15 15:15:52.270530608 +0000 2024-04-15T15:18:26.527 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-15T15:18:26.594 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2024-04-15T15:18:26.594 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2024-04-15T15:18:26.594 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000221915 s, 2.3 MB/s 2024-04-15T15:18:26.595 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-15T15:18:26.652 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-15T15:18:26.652 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi105.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi154.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-15T15:18:26.652 INFO:tasks.ceph:Generating config... 2024-04-15T15:18:26.653 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-15T15:18:26.653 INFO:tasks.ceph:[client] debug client = 20 2024-04-15T15:18:26.654 INFO:tasks.ceph:[client] debug ms = 1 2024-04-15T15:18:26.654 INFO:tasks.ceph:[client] pid file = 2024-04-15T15:18:26.654 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-15T15:18:26.654 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-15T15:18:26.654 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-15T15:18:26.654 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-15T15:18:26.654 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-15T15:18:26.654 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-15T15:18:26.654 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-15T15:18:26.654 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-15T15:18:26.654 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-15T15:18:26.654 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-15T15:18:26.654 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-15T15:18:26.654 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-15T15:18:26.655 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-15T15:18:26.655 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-15T15:18:26.655 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-15T15:18:26.655 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-15T15:18:26.655 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-15T15:18:26.655 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-15T15:18:26.655 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-15T15:18:26.655 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-15T15:18:26.655 INFO:tasks.ceph:[osd] bdev async discard = True 2024-04-15T15:18:26.655 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-04-15T15:18:26.655 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-04-15T15:18:26.655 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-15T15:18:26.655 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-15T15:18:26.655 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-04-15T15:18:26.655 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-04-15T15:18:26.656 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-15T15:18:26.656 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-15T15:18:26.656 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-04-15T15:18:26.656 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-15T15:18:26.656 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-15T15:18:26.656 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-15T15:18:26.656 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-15T15:18:26.656 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-15T15:18:26.656 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-15T15:18:26.656 INFO:tasks.ceph:Setting up mon.a... 2024-04-15T15:18:26.656 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-15T15:18:26.712 INFO:teuthology.orchestra.run.smithi105.stdout:creating /etc/ceph/ceph.keyring 2024-04-15T15:18:26.715 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-15T15:18:26.761 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-15T15:18:26.789 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.105'), ('mon.b', '172.21.15.154'), ('mon.c', '[v2:172.21.15.154:3301,v1:172.21.15.154:6790]')] 2024-04-15T15:18:26.790 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.105,172.21.15.154,[v2:172.21.15.154:3301,v1:172.21.15.154:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-15T15:18:26.791 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-04-15T15:18:26.791 DEBUG:teuthology.orchestra.run.smithi105:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-15T15:18:26.845 DEBUG:teuthology.orchestra.run.smithi105:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.105 --add b 172.21.15.154 --addv c '[v2:172.21.15.154:3301,v1:172.21.15.154:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-15T15:18:26.931 INFO:teuthology.orchestra.run.smithi105.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-15T15:18:26.931 INFO:teuthology.orchestra.run.smithi105.stdout:monmaptool: generated fsid 09e4d2a4-a324-4793-af77-644b25e520e1 2024-04-15T15:18:26.931 INFO:teuthology.orchestra.run.smithi105.stdout:setting min_mon_release = quincy 2024-04-15T15:18:26.931 INFO:teuthology.orchestra.run.smithi105.stdout:epoch 0 2024-04-15T15:18:26.932 INFO:teuthology.orchestra.run.smithi105.stdout:fsid 09e4d2a4-a324-4793-af77-644b25e520e1 2024-04-15T15:18:26.932 INFO:teuthology.orchestra.run.smithi105.stdout:last_changed 2024-04-15T15:18:26.930637+0000 2024-04-15T15:18:26.932 INFO:teuthology.orchestra.run.smithi105.stdout:created 2024-04-15T15:18:26.930637+0000 2024-04-15T15:18:26.932 INFO:teuthology.orchestra.run.smithi105.stdout:min_mon_release 17 (quincy) 2024-04-15T15:18:26.932 INFO:teuthology.orchestra.run.smithi105.stdout:election_strategy: 1 2024-04-15T15:18:26.932 INFO:teuthology.orchestra.run.smithi105.stdout:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-04-15T15:18:26.932 INFO:teuthology.orchestra.run.smithi105.stdout:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-15T15:18:26.932 INFO:teuthology.orchestra.run.smithi105.stdout:2: [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] mon.c 2024-04-15T15:18:26.932 INFO:teuthology.orchestra.run.smithi105.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-15T15:18:26.933 DEBUG:teuthology.orchestra.run.smithi105:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-15T15:18:26.948 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 09e4d2a4-a324-4793-af77-644b25e520e1... 2024-04-15T15:18:26.949 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-15T15:18:26.991 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-15T15:18:27.041 INFO:teuthology.orchestra.run.smithi105.stdout:[global] 2024-04-15T15:18:27.041 INFO:teuthology.orchestra.run.smithi105.stdout: chdir = "" 2024-04-15T15:18:27.041 INFO:teuthology.orchestra.run.smithi105.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-15T15:18:27.042 INFO:teuthology.orchestra.run.smithi105.stdout: auth supported = cephx 2024-04-15T15:18:27.042 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:18:27.042 INFO:teuthology.orchestra.run.smithi105.stdout: filestore xattr use omap = true 2024-04-15T15:18:27.042 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:18:27.042 INFO:teuthology.orchestra.run.smithi105.stdout: mon clock drift allowed = 1.000 2024-04-15T15:18:27.042 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:18:27.042 INFO:teuthology.orchestra.run.smithi105.stdout: osd crush chooseleaf type = 0 2024-04-15T15:18:27.042 INFO:teuthology.orchestra.run.smithi105.stdout: auth debug = true 2024-04-15T15:18:27.042 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:18:27.042 INFO:teuthology.orchestra.run.smithi105.stdout: ms die on old message = true 2024-04-15T15:18:27.042 INFO:teuthology.orchestra.run.smithi105.stdout: ms die on bug = true 2024-04-15T15:18:27.042 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:18:27.043 INFO:teuthology.orchestra.run.smithi105.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-15T15:18:27.043 INFO:teuthology.orchestra.run.smithi105.stdout: mon pg warn max object skew = 0 2024-04-15T15:18:27.043 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:18:27.043 INFO:teuthology.orchestra.run.smithi105.stdout: # disable pg_autoscaler by default for new pools 2024-04-15T15:18:27.043 INFO:teuthology.orchestra.run.smithi105.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-15T15:18:27.043 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:18:27.043 INFO:teuthology.orchestra.run.smithi105.stdout: osd pool default size = 2 2024-04-15T15:18:27.043 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:18:27.043 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd allow primary affinity = true 2024-04-15T15:18:27.043 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd allow pg remap = true 2024-04-15T15:18:27.043 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on legacy crush tunables = false 2024-04-15T15:18:27.043 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on crush straw calc version zero = false 2024-04-15T15:18:27.044 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on no sortbitwise = false 2024-04-15T15:18:27.044 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on osd down out interval zero = false 2024-04-15T15:18:27.044 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on too few osds = false 2024-04-15T15:18:27.044 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-15T15:18:27.044 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-15T15:18:27.044 INFO:teuthology.orchestra.run.smithi105.stdout: mon_allow_pool_size_one = true 2024-04-15T15:18:27.044 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:18:27.044 INFO:teuthology.orchestra.run.smithi105.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-15T15:18:27.044 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:18:27.044 INFO:teuthology.orchestra.run.smithi105.stdout: osd default data pool replay window = 5 2024-04-15T15:18:27.044 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:18:27.044 INFO:teuthology.orchestra.run.smithi105.stdout: mon allow pool delete = true 2024-04-15T15:18:27.045 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:18:27.045 INFO:teuthology.orchestra.run.smithi105.stdout: mon cluster log file level = debug 2024-04-15T15:18:27.045 INFO:teuthology.orchestra.run.smithi105.stdout: debug asserts on shutdown = true 2024-04-15T15:18:27.045 INFO:teuthology.orchestra.run.smithi105.stdout: mon health detail to clog = false 2024-04-15T15:18:27.045 INFO:teuthology.orchestra.run.smithi105.stdout: mon host = "172.21.15.105,172.21.15.154,[v2:172.21.15.154:3301,v1:172.21.15.154:6790]" 2024-04-15T15:18:27.045 INFO:teuthology.orchestra.run.smithi105.stdout: fsid = 09e4d2a4-a324-4793-af77-644b25e520e1 2024-04-15T15:18:27.045 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:18:27.045 INFO:teuthology.orchestra.run.smithi105.stdout:[osd] 2024-04-15T15:18:27.045 INFO:teuthology.orchestra.run.smithi105.stdout: osd journal size = 100 2024-04-15T15:18:27.045 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:18:27.045 INFO:teuthology.orchestra.run.smithi105.stdout: osd scrub load threshold = 5.0 2024-04-15T15:18:27.045 INFO:teuthology.orchestra.run.smithi105.stdout: osd scrub max interval = 600 2024-04-15T15:18:27.045 INFO:teuthology.orchestra.run.smithi105.stdout: osd mclock profile = high_recovery_ops 2024-04-15T15:18:27.046 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:18:27.046 INFO:teuthology.orchestra.run.smithi105.stdout: osd recover clone overlap = true 2024-04-15T15:18:27.046 INFO:teuthology.orchestra.run.smithi105.stdout: osd recovery max chunk = 1048576 2024-04-15T15:18:27.046 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:18:27.046 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug shutdown = true 2024-04-15T15:18:27.046 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug op order = true 2024-04-15T15:18:27.046 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug verify stray on activate = true 2024-04-15T15:18:27.046 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug trim objects = true 2024-04-15T15:18:27.046 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:18:27.046 INFO:teuthology.orchestra.run.smithi105.stdout: osd open classes on start = true 2024-04-15T15:18:27.046 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug pg log writeout = true 2024-04-15T15:18:27.047 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:18:27.047 INFO:teuthology.orchestra.run.smithi105.stdout: osd deep scrub update digest min age = 30 2024-04-15T15:18:27.047 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:18:27.047 INFO:teuthology.orchestra.run.smithi105.stdout: osd map max advance = 10 2024-04-15T15:18:27.047 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:18:27.047 INFO:teuthology.orchestra.run.smithi105.stdout: journal zero on create = true 2024-04-15T15:18:27.047 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:18:27.047 INFO:teuthology.orchestra.run.smithi105.stdout: filestore ondisk finisher threads = 3 2024-04-15T15:18:27.047 INFO:teuthology.orchestra.run.smithi105.stdout: filestore apply finisher threads = 3 2024-04-15T15:18:27.047 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:18:27.047 INFO:teuthology.orchestra.run.smithi105.stdout: bdev debug aio = true 2024-04-15T15:18:27.047 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug misdirected ops = true 2024-04-15T15:18:27.048 INFO:teuthology.orchestra.run.smithi105.stdout: bdev async discard = True 2024-04-15T15:18:27.048 INFO:teuthology.orchestra.run.smithi105.stdout: bdev enable discard = True 2024-04-15T15:18:27.048 INFO:teuthology.orchestra.run.smithi105.stdout: bluestore allocator = bitmap 2024-04-15T15:18:27.048 INFO:teuthology.orchestra.run.smithi105.stdout: bluestore block size = 96636764160 2024-04-15T15:18:27.048 INFO:teuthology.orchestra.run.smithi105.stdout: bluestore fsck on mount = True 2024-04-15T15:18:27.048 INFO:teuthology.orchestra.run.smithi105.stdout: debug bluefs = 1/20 2024-04-15T15:18:27.048 INFO:teuthology.orchestra.run.smithi105.stdout: debug bluestore = 1/20 2024-04-15T15:18:27.048 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2024-04-15T15:18:27.048 INFO:teuthology.orchestra.run.smithi105.stdout: debug osd = 20 2024-04-15T15:18:27.048 INFO:teuthology.orchestra.run.smithi105.stdout: debug rocksdb = 4/10 2024-04-15T15:18:27.048 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd backfillfull_ratio = 0.85 2024-04-15T15:18:27.048 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd full ratio = 0.9 2024-04-15T15:18:27.049 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd nearfull ratio = 0.8 2024-04-15T15:18:27.049 INFO:teuthology.orchestra.run.smithi105.stdout: osd failsafe full ratio = 0.95 2024-04-15T15:18:27.049 INFO:teuthology.orchestra.run.smithi105.stdout: osd objectstore = bluestore 2024-04-15T15:18:27.049 INFO:teuthology.orchestra.run.smithi105.stdout: osd op complaint time = 180 2024-04-15T15:18:27.049 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:18:27.049 INFO:teuthology.orchestra.run.smithi105.stdout:[mgr] 2024-04-15T15:18:27.049 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2024-04-15T15:18:27.049 INFO:teuthology.orchestra.run.smithi105.stdout: debug mgr = 20 2024-04-15T15:18:27.049 INFO:teuthology.orchestra.run.smithi105.stdout: debug mon = 20 2024-04-15T15:18:27.049 INFO:teuthology.orchestra.run.smithi105.stdout: debug auth = 20 2024-04-15T15:18:27.049 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min pgs per osd = 4 2024-04-15T15:18:27.050 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min bytes per osd = 10 2024-04-15T15:18:27.050 INFO:teuthology.orchestra.run.smithi105.stdout: mgr/telemetry/nag = false 2024-04-15T15:18:27.050 INFO:teuthology.orchestra.run.smithi105.stdout: debug client = 20 2024-04-15T15:18:27.050 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:18:27.050 INFO:teuthology.orchestra.run.smithi105.stdout:[mon] 2024-04-15T15:18:27.050 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2024-04-15T15:18:27.050 INFO:teuthology.orchestra.run.smithi105.stdout: debug mon = 20 2024-04-15T15:18:27.050 INFO:teuthology.orchestra.run.smithi105.stdout: debug paxos = 20 2024-04-15T15:18:27.050 INFO:teuthology.orchestra.run.smithi105.stdout: debug auth = 20 2024-04-15T15:18:27.050 INFO:teuthology.orchestra.run.smithi105.stdout: mon data avail warn = 5 2024-04-15T15:18:27.050 INFO:teuthology.orchestra.run.smithi105.stdout: mon mgr mkfs grace = 240 2024-04-15T15:18:27.050 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min pgs per osd = 4 2024-04-15T15:18:27.051 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd reporter subtree level = osd 2024-04-15T15:18:27.051 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd prime pg temp = true 2024-04-15T15:18:27.051 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min bytes per osd = 10 2024-04-15T15:18:27.051 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:18:27.051 INFO:teuthology.orchestra.run.smithi105.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-15T15:18:27.051 INFO:teuthology.orchestra.run.smithi105.stdout: auth mon ticket ttl = 660 # 11m 2024-04-15T15:18:27.051 INFO:teuthology.orchestra.run.smithi105.stdout: auth service ticket ttl = 240 # 4m 2024-04-15T15:18:27.051 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:18:27.051 INFO:teuthology.orchestra.run.smithi105.stdout: # don't complain about insecure global_id in the test suite 2024-04-15T15:18:27.051 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-15T15:18:27.051 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-15T15:18:27.051 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:18:27.052 INFO:teuthology.orchestra.run.smithi105.stdout: # 1m isn't quite enough 2024-04-15T15:18:27.052 INFO:teuthology.orchestra.run.smithi105.stdout: mon_down_mkfs_grace = 2m 2024-04-15T15:18:27.052 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:18:27.052 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_filestore_osds = false 2024-04-15T15:18:27.052 INFO:teuthology.orchestra.run.smithi105.stdout: mon down mkfs grace = 300 2024-04-15T15:18:27.052 INFO:teuthology.orchestra.run.smithi105.stdout: mon op complaint time = 120 2024-04-15T15:18:27.052 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:18:27.052 INFO:teuthology.orchestra.run.smithi105.stdout:[client] 2024-04-15T15:18:27.052 INFO:teuthology.orchestra.run.smithi105.stdout: rgw cache enabled = true 2024-04-15T15:18:27.052 INFO:teuthology.orchestra.run.smithi105.stdout: rgw enable ops log = true 2024-04-15T15:18:27.052 INFO:teuthology.orchestra.run.smithi105.stdout: rgw enable usage log = true 2024-04-15T15:18:27.052 INFO:teuthology.orchestra.run.smithi105.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-15T15:18:27.053 INFO:teuthology.orchestra.run.smithi105.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-15T15:18:27.053 INFO:teuthology.orchestra.run.smithi105.stdout: client mount timeout = 600 2024-04-15T15:18:27.053 INFO:teuthology.orchestra.run.smithi105.stdout: debug client = 20 2024-04-15T15:18:27.053 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2024-04-15T15:18:27.053 INFO:teuthology.orchestra.run.smithi105.stdout: pid file = "" 2024-04-15T15:18:27.053 INFO:teuthology.orchestra.run.smithi105.stdout: rados mon op timeout = 900 2024-04-15T15:18:27.053 INFO:teuthology.orchestra.run.smithi105.stdout: rados osd op timeout = 900 2024-04-15T15:18:27.053 INFO:teuthology.orchestra.run.smithi105.stdout:[mon.a] 2024-04-15T15:18:27.053 INFO:teuthology.orchestra.run.smithi105.stdout:[mon.b] 2024-04-15T15:18:27.053 INFO:teuthology.orchestra.run.smithi105.stdout:[mon.c] 2024-04-15T15:18:27.053 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.a] 2024-04-15T15:18:27.053 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.c] 2024-04-15T15:18:27.053 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.b] 2024-04-15T15:18:27.054 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.d] 2024-04-15T15:18:27.054 INFO:teuthology.orchestra.run.smithi105.stdout:[mds] 2024-04-15T15:18:27.054 INFO:teuthology.orchestra.run.smithi105.stdout: debug mds = 20 2024-04-15T15:18:27.054 INFO:teuthology.orchestra.run.smithi105.stdout: debug mds balancer = 20 2024-04-15T15:18:27.054 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2024-04-15T15:18:27.054 INFO:teuthology.orchestra.run.smithi105.stdout: mds debug frag = True 2024-04-15T15:18:27.054 INFO:teuthology.orchestra.run.smithi105.stdout: mds debug scatterstat = True 2024-04-15T15:18:27.054 INFO:teuthology.orchestra.run.smithi105.stdout: mds op complaint time = 180 2024-04-15T15:18:27.054 INFO:teuthology.orchestra.run.smithi105.stdout: mds verify scatter = True 2024-04-15T15:18:27.054 INFO:teuthology.orchestra.run.smithi105.stdout: osd op complaint time = 180 2024-04-15T15:18:27.054 INFO:teuthology.orchestra.run.smithi105.stdout: rados mon op timeout = 900 2024-04-15T15:18:27.054 INFO:teuthology.orchestra.run.smithi105.stdout: rados osd op timeout = 900 2024-04-15T15:18:27.055 INFO:teuthology.orchestra.run.smithi154.stdout:[global] 2024-04-15T15:18:27.055 INFO:teuthology.orchestra.run.smithi154.stdout: chdir = "" 2024-04-15T15:18:27.055 INFO:teuthology.orchestra.run.smithi154.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-15T15:18:27.055 INFO:teuthology.orchestra.run.smithi154.stdout: auth supported = cephx 2024-04-15T15:18:27.055 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-15T15:18:27.055 INFO:teuthology.orchestra.run.smithi154.stdout: filestore xattr use omap = true 2024-04-15T15:18:27.055 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-15T15:18:27.056 INFO:teuthology.orchestra.run.smithi154.stdout: mon clock drift allowed = 1.000 2024-04-15T15:18:27.056 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-15T15:18:27.056 INFO:teuthology.orchestra.run.smithi154.stdout: osd crush chooseleaf type = 0 2024-04-15T15:18:27.056 INFO:teuthology.orchestra.run.smithi154.stdout: auth debug = true 2024-04-15T15:18:27.056 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-15T15:18:27.056 INFO:teuthology.orchestra.run.smithi154.stdout: ms die on old message = true 2024-04-15T15:18:27.056 INFO:teuthology.orchestra.run.smithi154.stdout: ms die on bug = true 2024-04-15T15:18:27.056 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-15T15:18:27.056 INFO:teuthology.orchestra.run.smithi154.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-15T15:18:27.056 INFO:teuthology.orchestra.run.smithi154.stdout: mon pg warn max object skew = 0 2024-04-15T15:18:27.056 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-15T15:18:27.056 INFO:teuthology.orchestra.run.smithi154.stdout: # disable pg_autoscaler by default for new pools 2024-04-15T15:18:27.056 INFO:teuthology.orchestra.run.smithi154.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-15T15:18:27.057 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-15T15:18:27.057 INFO:teuthology.orchestra.run.smithi154.stdout: osd pool default size = 2 2024-04-15T15:18:27.057 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-15T15:18:27.057 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd allow primary affinity = true 2024-04-15T15:18:27.057 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd allow pg remap = true 2024-04-15T15:18:27.057 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on legacy crush tunables = false 2024-04-15T15:18:27.057 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on crush straw calc version zero = false 2024-04-15T15:18:27.057 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on no sortbitwise = false 2024-04-15T15:18:27.057 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on osd down out interval zero = false 2024-04-15T15:18:27.057 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on too few osds = false 2024-04-15T15:18:27.057 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-15T15:18:27.057 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-15T15:18:27.058 INFO:teuthology.orchestra.run.smithi154.stdout: mon_allow_pool_size_one = true 2024-04-15T15:18:27.058 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-15T15:18:27.058 INFO:teuthology.orchestra.run.smithi154.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-15T15:18:27.058 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-15T15:18:27.058 INFO:teuthology.orchestra.run.smithi154.stdout: osd default data pool replay window = 5 2024-04-15T15:18:27.058 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-15T15:18:27.058 INFO:teuthology.orchestra.run.smithi154.stdout: mon allow pool delete = true 2024-04-15T15:18:27.058 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-15T15:18:27.058 INFO:teuthology.orchestra.run.smithi154.stdout: mon cluster log file level = debug 2024-04-15T15:18:27.058 INFO:teuthology.orchestra.run.smithi154.stdout: debug asserts on shutdown = true 2024-04-15T15:18:27.058 INFO:teuthology.orchestra.run.smithi154.stdout: mon health detail to clog = false 2024-04-15T15:18:27.058 INFO:teuthology.orchestra.run.smithi154.stdout: mon host = "172.21.15.105,172.21.15.154,[v2:172.21.15.154:3301,v1:172.21.15.154:6790]" 2024-04-15T15:18:27.058 INFO:teuthology.orchestra.run.smithi154.stdout: fsid = 09e4d2a4-a324-4793-af77-644b25e520e1 2024-04-15T15:18:27.059 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-15T15:18:27.059 INFO:teuthology.orchestra.run.smithi154.stdout:[osd] 2024-04-15T15:18:27.059 INFO:teuthology.orchestra.run.smithi154.stdout: osd journal size = 100 2024-04-15T15:18:27.059 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-15T15:18:27.059 INFO:teuthology.orchestra.run.smithi154.stdout: osd scrub load threshold = 5.0 2024-04-15T15:18:27.059 INFO:teuthology.orchestra.run.smithi154.stdout: osd scrub max interval = 600 2024-04-15T15:18:27.059 INFO:teuthology.orchestra.run.smithi154.stdout: osd mclock profile = high_recovery_ops 2024-04-15T15:18:27.059 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-15T15:18:27.059 INFO:teuthology.orchestra.run.smithi154.stdout: osd recover clone overlap = true 2024-04-15T15:18:27.060 INFO:teuthology.orchestra.run.smithi154.stdout: osd recovery max chunk = 1048576 2024-04-15T15:18:27.060 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-15T15:18:27.060 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug shutdown = true 2024-04-15T15:18:27.060 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug op order = true 2024-04-15T15:18:27.060 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug verify stray on activate = true 2024-04-15T15:18:27.060 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug trim objects = true 2024-04-15T15:18:27.060 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-15T15:18:27.060 INFO:teuthology.orchestra.run.smithi154.stdout: osd open classes on start = true 2024-04-15T15:18:27.060 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug pg log writeout = true 2024-04-15T15:18:27.060 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-15T15:18:27.060 INFO:teuthology.orchestra.run.smithi154.stdout: osd deep scrub update digest min age = 30 2024-04-15T15:18:27.061 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-15T15:18:27.061 INFO:teuthology.orchestra.run.smithi154.stdout: osd map max advance = 10 2024-04-15T15:18:27.061 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-15T15:18:27.061 INFO:teuthology.orchestra.run.smithi154.stdout: journal zero on create = true 2024-04-15T15:18:27.061 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-15T15:18:27.061 INFO:teuthology.orchestra.run.smithi154.stdout: filestore ondisk finisher threads = 3 2024-04-15T15:18:27.061 INFO:teuthology.orchestra.run.smithi154.stdout: filestore apply finisher threads = 3 2024-04-15T15:18:27.061 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-15T15:18:27.061 INFO:teuthology.orchestra.run.smithi154.stdout: bdev debug aio = true 2024-04-15T15:18:27.061 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug misdirected ops = true 2024-04-15T15:18:27.061 INFO:teuthology.orchestra.run.smithi154.stdout: bdev async discard = True 2024-04-15T15:18:27.061 INFO:teuthology.orchestra.run.smithi154.stdout: bdev enable discard = True 2024-04-15T15:18:27.061 INFO:teuthology.orchestra.run.smithi154.stdout: bluestore allocator = bitmap 2024-04-15T15:18:27.061 INFO:teuthology.orchestra.run.smithi154.stdout: bluestore block size = 96636764160 2024-04-15T15:18:27.061 INFO:teuthology.orchestra.run.smithi154.stdout: bluestore fsck on mount = True 2024-04-15T15:18:27.062 INFO:teuthology.orchestra.run.smithi154.stdout: debug bluefs = 1/20 2024-04-15T15:18:27.062 INFO:teuthology.orchestra.run.smithi154.stdout: debug bluestore = 1/20 2024-04-15T15:18:27.062 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2024-04-15T15:18:27.062 INFO:teuthology.orchestra.run.smithi154.stdout: debug osd = 20 2024-04-15T15:18:27.062 INFO:teuthology.orchestra.run.smithi154.stdout: debug rocksdb = 4/10 2024-04-15T15:18:27.062 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd backfillfull_ratio = 0.85 2024-04-15T15:18:27.062 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd full ratio = 0.9 2024-04-15T15:18:27.062 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd nearfull ratio = 0.8 2024-04-15T15:18:27.062 INFO:teuthology.orchestra.run.smithi154.stdout: osd failsafe full ratio = 0.95 2024-04-15T15:18:27.062 INFO:teuthology.orchestra.run.smithi154.stdout: osd objectstore = bluestore 2024-04-15T15:18:27.062 INFO:teuthology.orchestra.run.smithi154.stdout: osd op complaint time = 180 2024-04-15T15:18:27.062 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-15T15:18:27.063 INFO:teuthology.orchestra.run.smithi154.stdout:[mgr] 2024-04-15T15:18:27.063 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2024-04-15T15:18:27.063 INFO:teuthology.orchestra.run.smithi154.stdout: debug mgr = 20 2024-04-15T15:18:27.063 INFO:teuthology.orchestra.run.smithi154.stdout: debug mon = 20 2024-04-15T15:18:27.063 INFO:teuthology.orchestra.run.smithi154.stdout: debug auth = 20 2024-04-15T15:18:27.063 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min pgs per osd = 4 2024-04-15T15:18:27.063 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min bytes per osd = 10 2024-04-15T15:18:27.063 INFO:teuthology.orchestra.run.smithi154.stdout: mgr/telemetry/nag = false 2024-04-15T15:18:27.063 INFO:teuthology.orchestra.run.smithi154.stdout: debug client = 20 2024-04-15T15:18:27.063 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-15T15:18:27.063 INFO:teuthology.orchestra.run.smithi154.stdout:[mon] 2024-04-15T15:18:27.063 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2024-04-15T15:18:27.063 INFO:teuthology.orchestra.run.smithi154.stdout: debug mon = 20 2024-04-15T15:18:27.064 INFO:teuthology.orchestra.run.smithi154.stdout: debug paxos = 20 2024-04-15T15:18:27.064 INFO:teuthology.orchestra.run.smithi154.stdout: debug auth = 20 2024-04-15T15:18:27.064 INFO:teuthology.orchestra.run.smithi154.stdout: mon data avail warn = 5 2024-04-15T15:18:27.064 INFO:teuthology.orchestra.run.smithi154.stdout: mon mgr mkfs grace = 240 2024-04-15T15:18:27.064 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min pgs per osd = 4 2024-04-15T15:18:27.064 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd reporter subtree level = osd 2024-04-15T15:18:27.064 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd prime pg temp = true 2024-04-15T15:18:27.064 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min bytes per osd = 10 2024-04-15T15:18:27.064 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-15T15:18:27.064 INFO:teuthology.orchestra.run.smithi154.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-15T15:18:27.064 INFO:teuthology.orchestra.run.smithi154.stdout: auth mon ticket ttl = 660 # 11m 2024-04-15T15:18:27.064 INFO:teuthology.orchestra.run.smithi154.stdout: auth service ticket ttl = 240 # 4m 2024-04-15T15:18:27.064 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-15T15:18:27.065 INFO:teuthology.orchestra.run.smithi154.stdout: # don't complain about insecure global_id in the test suite 2024-04-15T15:18:27.065 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-15T15:18:27.065 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-15T15:18:27.065 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-15T15:18:27.065 INFO:teuthology.orchestra.run.smithi154.stdout: # 1m isn't quite enough 2024-04-15T15:18:27.065 INFO:teuthology.orchestra.run.smithi154.stdout: mon_down_mkfs_grace = 2m 2024-04-15T15:18:27.065 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-15T15:18:27.065 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_filestore_osds = false 2024-04-15T15:18:27.065 INFO:teuthology.orchestra.run.smithi154.stdout: mon down mkfs grace = 300 2024-04-15T15:18:27.065 INFO:teuthology.orchestra.run.smithi154.stdout: mon op complaint time = 120 2024-04-15T15:18:27.065 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-15T15:18:27.065 INFO:teuthology.orchestra.run.smithi154.stdout:[client] 2024-04-15T15:18:27.065 INFO:teuthology.orchestra.run.smithi154.stdout: rgw cache enabled = true 2024-04-15T15:18:27.065 INFO:teuthology.orchestra.run.smithi154.stdout: rgw enable ops log = true 2024-04-15T15:18:27.066 INFO:teuthology.orchestra.run.smithi154.stdout: rgw enable usage log = true 2024-04-15T15:18:27.066 INFO:teuthology.orchestra.run.smithi154.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-15T15:18:27.066 INFO:teuthology.orchestra.run.smithi154.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-15T15:18:27.066 INFO:teuthology.orchestra.run.smithi154.stdout: client mount timeout = 600 2024-04-15T15:18:27.066 INFO:teuthology.orchestra.run.smithi154.stdout: debug client = 20 2024-04-15T15:18:27.066 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2024-04-15T15:18:27.066 INFO:teuthology.orchestra.run.smithi154.stdout: pid file = "" 2024-04-15T15:18:27.066 INFO:teuthology.orchestra.run.smithi154.stdout: rados mon op timeout = 900 2024-04-15T15:18:27.066 INFO:teuthology.orchestra.run.smithi154.stdout: rados osd op timeout = 900 2024-04-15T15:18:27.066 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.a] 2024-04-15T15:18:27.066 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.b] 2024-04-15T15:18:27.067 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.c] 2024-04-15T15:18:27.067 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.a] 2024-04-15T15:18:27.067 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.c] 2024-04-15T15:18:27.067 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.b] 2024-04-15T15:18:27.067 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.d] 2024-04-15T15:18:27.067 INFO:teuthology.orchestra.run.smithi154.stdout:[mds] 2024-04-15T15:18:27.067 INFO:teuthology.orchestra.run.smithi154.stdout: debug mds = 20 2024-04-15T15:18:27.067 INFO:teuthology.orchestra.run.smithi154.stdout: debug mds balancer = 20 2024-04-15T15:18:27.067 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2024-04-15T15:18:27.067 INFO:teuthology.orchestra.run.smithi154.stdout: mds debug frag = True 2024-04-15T15:18:27.068 INFO:teuthology.orchestra.run.smithi154.stdout: mds debug scatterstat = True 2024-04-15T15:18:27.068 INFO:teuthology.orchestra.run.smithi154.stdout: mds op complaint time = 180 2024-04-15T15:18:27.068 INFO:teuthology.orchestra.run.smithi154.stdout: mds verify scatter = True 2024-04-15T15:18:27.068 INFO:teuthology.orchestra.run.smithi154.stdout: osd op complaint time = 180 2024-04-15T15:18:27.068 INFO:teuthology.orchestra.run.smithi154.stdout: rados mon op timeout = 900 2024-04-15T15:18:27.068 INFO:teuthology.orchestra.run.smithi154.stdout: rados osd op timeout = 900 2024-04-15T15:18:27.069 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-15T15:18:27.070 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-15T15:18:27.167 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-15T15:18:27.168 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-04-15T15:18:27.168 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-15T15:18:27.183 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-04-15T15:18:27.183 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-15T15:18:27.239 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi105.front.sepia.ceph.com 2024-04-15T15:18:27.239 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-04-15T15:18:27.239 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-15T15:18:27.239 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-15T15:18:27.320 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-04-15T15:18:27.321 DEBUG:teuthology.orchestra.run.smithi105:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-15T15:18:27.376 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi154.front.sepia.ceph.com 2024-04-15T15:18:27.376 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-15T15:18:27.376 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-15T15:18:27.376 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-15T15:18:27.418 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-15T15:18:27.419 DEBUG:teuthology.orchestra.run.smithi154:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-15T15:18:27.474 INFO:tasks.ceph:Setting up mon nodes... 2024-04-15T15:18:27.475 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-15T15:18:27.475 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-15T15:18:27.532 INFO:teuthology.orchestra.run.smithi105.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-15T15:18:27.535 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-15T15:18:27.600 INFO:teuthology.orchestra.run.smithi154.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-15T15:18:27.603 INFO:tasks.ceph:Setting up mds nodes... 2024-04-15T15:18:27.604 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-15T15:18:27.661 INFO:teuthology.orchestra.run.smithi105.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-15T15:18:27.663 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-15T15:18:27.692 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-15T15:18:27.789 INFO:teuthology.orchestra.run.smithi105.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-15T15:18:27.792 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-15T15:18:27.821 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-15T15:18:27.879 INFO:teuthology.orchestra.run.smithi154.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-15T15:18:27.881 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-15T15:18:27.910 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-15T15:18:28.008 INFO:teuthology.orchestra.run.smithi154.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-15T15:18:28.011 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-15T15:18:28.039 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-15T15:18:28.039 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-15T15:18:28.083 INFO:teuthology.orchestra.run.smithi105.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-15T15:18:28.099 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-15T15:18:28.144 INFO:teuthology.orchestra.run.smithi105.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-15T15:18:28.160 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-15T15:18:28.204 INFO:teuthology.orchestra.run.smithi154.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-15T15:18:28.221 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-15T15:18:28.265 INFO:teuthology.orchestra.run.smithi154.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-15T15:18:28.281 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-15T15:18:28.282 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi105.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi154.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-15T15:18:28.282 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-15T15:18:28.311 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-15T15:18:28.311 INFO:tasks.ceph:role: osd.0 2024-04-15T15:18:28.311 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi105.front.sepia.ceph.com 2024-04-15T15:18:28.312 DEBUG:teuthology.orchestra.run.smithi105:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-15T15:18:28.385 INFO:teuthology.orchestra.run.smithi105.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-15T15:18:28.385 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-15T15:18:28.385 INFO:teuthology.orchestra.run.smithi105.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-15T15:18:28.386 INFO:teuthology.orchestra.run.smithi105.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-15T15:18:28.386 INFO:teuthology.orchestra.run.smithi105.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-15T15:18:28.386 INFO:teuthology.orchestra.run.smithi105.stdout: = sunit=0 swidth=0 blks 2024-04-15T15:18:28.386 INFO:teuthology.orchestra.run.smithi105.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-15T15:18:28.386 INFO:teuthology.orchestra.run.smithi105.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-15T15:18:28.386 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-15T15:18:28.386 INFO:teuthology.orchestra.run.smithi105.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-15T15:18:28.905 INFO:teuthology.orchestra.run.smithi105.stdout:Discarding blocks...Done. 2024-04-15T15:18:28.907 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi105.front.sepia.ceph.com -o noatime 2024-04-15T15:18:28.907 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-15T15:18:29.058 DEBUG:teuthology.orchestra.run.smithi105:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-15T15:18:29.092 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-15T15:18:29.161 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-15T15:18:29.161 INFO:tasks.ceph:role: osd.1 2024-04-15T15:18:29.161 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi105.front.sepia.ceph.com 2024-04-15T15:18:29.161 DEBUG:teuthology.orchestra.run.smithi105:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-15T15:18:29.236 INFO:teuthology.orchestra.run.smithi105.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-15T15:18:29.237 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-15T15:18:29.237 INFO:teuthology.orchestra.run.smithi105.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-15T15:18:29.237 INFO:teuthology.orchestra.run.smithi105.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-15T15:18:29.237 INFO:teuthology.orchestra.run.smithi105.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-15T15:18:29.237 INFO:teuthology.orchestra.run.smithi105.stdout: = sunit=0 swidth=0 blks 2024-04-15T15:18:29.237 INFO:teuthology.orchestra.run.smithi105.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-15T15:18:29.237 INFO:teuthology.orchestra.run.smithi105.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-15T15:18:29.237 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-15T15:18:29.237 INFO:teuthology.orchestra.run.smithi105.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-15T15:18:29.689 INFO:teuthology.orchestra.run.smithi105.stdout:Discarding blocks...Done. 2024-04-15T15:18:29.691 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi105.front.sepia.ceph.com -o noatime 2024-04-15T15:18:29.691 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-15T15:18:29.792 DEBUG:teuthology.orchestra.run.smithi105:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-15T15:18:29.826 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-15T15:18:29.895 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-15T15:18:29.895 INFO:tasks.ceph:role: osd.2 2024-04-15T15:18:29.895 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi105.front.sepia.ceph.com 2024-04-15T15:18:29.895 DEBUG:teuthology.orchestra.run.smithi105:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-15T15:18:29.972 INFO:teuthology.orchestra.run.smithi105.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-15T15:18:29.973 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-15T15:18:29.973 INFO:teuthology.orchestra.run.smithi105.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-15T15:18:29.973 INFO:teuthology.orchestra.run.smithi105.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-15T15:18:29.973 INFO:teuthology.orchestra.run.smithi105.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-15T15:18:29.973 INFO:teuthology.orchestra.run.smithi105.stdout: = sunit=0 swidth=0 blks 2024-04-15T15:18:29.974 INFO:teuthology.orchestra.run.smithi105.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-15T15:18:29.974 INFO:teuthology.orchestra.run.smithi105.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-15T15:18:29.974 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-15T15:18:29.974 INFO:teuthology.orchestra.run.smithi105.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-15T15:18:30.457 INFO:teuthology.orchestra.run.smithi105.stdout:Discarding blocks...Done. 2024-04-15T15:18:30.459 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi105.front.sepia.ceph.com -o noatime 2024-04-15T15:18:30.459 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-15T15:18:30.601 DEBUG:teuthology.orchestra.run.smithi105:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-15T15:18:30.634 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-15T15:18:30.703 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-15T15:18:30.703 INFO:tasks.ceph:role: osd.3 2024-04-15T15:18:30.703 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi105.front.sepia.ceph.com 2024-04-15T15:18:30.703 DEBUG:teuthology.orchestra.run.smithi105:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-15T15:18:30.779 INFO:teuthology.orchestra.run.smithi105.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-15T15:18:30.779 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-15T15:18:30.780 INFO:teuthology.orchestra.run.smithi105.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-15T15:18:30.780 INFO:teuthology.orchestra.run.smithi105.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-15T15:18:30.780 INFO:teuthology.orchestra.run.smithi105.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-15T15:18:30.780 INFO:teuthology.orchestra.run.smithi105.stdout: = sunit=0 swidth=0 blks 2024-04-15T15:18:30.780 INFO:teuthology.orchestra.run.smithi105.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-15T15:18:30.780 INFO:teuthology.orchestra.run.smithi105.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-15T15:18:30.780 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-15T15:18:30.780 INFO:teuthology.orchestra.run.smithi105.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-15T15:18:31.269 INFO:teuthology.orchestra.run.smithi105.stdout:Discarding blocks...Done. 2024-04-15T15:18:31.270 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi105.front.sepia.ceph.com -o noatime 2024-04-15T15:18:31.271 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-15T15:18:31.396 DEBUG:teuthology.orchestra.run.smithi105:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-15T15:18:31.432 DEBUG:teuthology.orchestra.run.smithi105:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-15T15:18:31.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:18:31.538+0000 7f0cd318f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-15T15:18:31.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:18:31.538+0000 7f0cd318f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-15T15:18:31.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:18:31.538+0000 7f0cd318f640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-15T15:18:31.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:18:31.538+0000 7f0cd318f640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-15T15:18:31.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:18:31.538+0000 7f0cd318f640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-15T15:18:31.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:18:31.538+0000 7f0cd318f640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-15T15:18:34.397 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-15T15:18:34.430 DEBUG:teuthology.orchestra.run.smithi105:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-15T15:18:34.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:18:34.523+0000 7fe8d50a0640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-15T15:18:34.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:18:34.523+0000 7fe8d50a0640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-15T15:18:34.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:18:34.523+0000 7fe8d50a0640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-15T15:18:34.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:18:34.523+0000 7fe8d50a0640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-15T15:18:34.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:18:34.523+0000 7fe8d50a0640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-15T15:18:34.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:18:34.524+0000 7fe8d50a0640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-15T15:18:37.376 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-15T15:18:37.409 DEBUG:teuthology.orchestra.run.smithi105:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-15T15:18:37.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:18:37.502+0000 7f629a70f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-15T15:18:37.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:18:37.502+0000 7f629a70f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-15T15:18:37.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:18:37.502+0000 7f629a70f640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-15T15:18:37.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:18:37.502+0000 7f629a70f640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-15T15:18:37.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:18:37.502+0000 7f629a70f640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-15T15:18:37.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:18:37.503+0000 7f629a70f640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-15T15:18:40.352 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-15T15:18:40.384 DEBUG:teuthology.orchestra.run.smithi105:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-15T15:18:40.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:18:40.476+0000 7f95a8a11640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-15T15:18:40.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:18:40.476+0000 7f95a8a11640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-15T15:18:40.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:18:40.476+0000 7f95a8a11640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-15T15:18:40.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:18:40.476+0000 7f95a8a11640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-15T15:18:40.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:18:40.476+0000 7f95a8a11640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-15T15:18:40.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:18:40.476+0000 7f95a8a11640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-15T15:18:43.335 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-15T15:18:43.367 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-15T15:18:43.401 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-15T15:18:43.402 INFO:tasks.ceph:role: osd.4 2024-04-15T15:18:43.402 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi154.front.sepia.ceph.com 2024-04-15T15:18:43.402 DEBUG:teuthology.orchestra.run.smithi154:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-15T15:18:43.481 INFO:teuthology.orchestra.run.smithi154.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-15T15:18:43.481 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-15T15:18:43.481 INFO:teuthology.orchestra.run.smithi154.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-15T15:18:43.481 INFO:teuthology.orchestra.run.smithi154.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-15T15:18:43.481 INFO:teuthology.orchestra.run.smithi154.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-15T15:18:43.481 INFO:teuthology.orchestra.run.smithi154.stdout: = sunit=0 swidth=0 blks 2024-04-15T15:18:43.481 INFO:teuthology.orchestra.run.smithi154.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-15T15:18:43.482 INFO:teuthology.orchestra.run.smithi154.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-15T15:18:43.482 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-15T15:18:43.482 INFO:teuthology.orchestra.run.smithi154.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-15T15:18:43.978 INFO:teuthology.orchestra.run.smithi154.stdout:Discarding blocks...Done. 2024-04-15T15:18:43.980 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi154.front.sepia.ceph.com -o noatime 2024-04-15T15:18:43.980 DEBUG:teuthology.orchestra.run.smithi154:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-15T15:18:44.103 DEBUG:teuthology.orchestra.run.smithi154:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-15T15:18:44.141 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-15T15:18:44.212 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-15T15:18:44.213 INFO:tasks.ceph:role: osd.5 2024-04-15T15:18:44.213 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi154.front.sepia.ceph.com 2024-04-15T15:18:44.213 DEBUG:teuthology.orchestra.run.smithi154:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-15T15:18:44.290 INFO:teuthology.orchestra.run.smithi154.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-15T15:18:44.291 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-15T15:18:44.291 INFO:teuthology.orchestra.run.smithi154.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-15T15:18:44.291 INFO:teuthology.orchestra.run.smithi154.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-15T15:18:44.291 INFO:teuthology.orchestra.run.smithi154.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-15T15:18:44.291 INFO:teuthology.orchestra.run.smithi154.stdout: = sunit=0 swidth=0 blks 2024-04-15T15:18:44.291 INFO:teuthology.orchestra.run.smithi154.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-15T15:18:44.291 INFO:teuthology.orchestra.run.smithi154.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-15T15:18:44.291 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-15T15:18:44.291 INFO:teuthology.orchestra.run.smithi154.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-15T15:18:44.747 INFO:teuthology.orchestra.run.smithi154.stdout:Discarding blocks...Done. 2024-04-15T15:18:44.751 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi154.front.sepia.ceph.com -o noatime 2024-04-15T15:18:44.751 DEBUG:teuthology.orchestra.run.smithi154:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-15T15:18:44.844 DEBUG:teuthology.orchestra.run.smithi154:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-15T15:18:44.877 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-15T15:18:44.946 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-15T15:18:44.946 INFO:tasks.ceph:role: osd.6 2024-04-15T15:18:44.946 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi154.front.sepia.ceph.com 2024-04-15T15:18:44.946 DEBUG:teuthology.orchestra.run.smithi154:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-15T15:18:45.023 INFO:teuthology.orchestra.run.smithi154.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-15T15:18:45.023 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-15T15:18:45.023 INFO:teuthology.orchestra.run.smithi154.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-15T15:18:45.023 INFO:teuthology.orchestra.run.smithi154.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-15T15:18:45.023 INFO:teuthology.orchestra.run.smithi154.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-15T15:18:45.023 INFO:teuthology.orchestra.run.smithi154.stdout: = sunit=0 swidth=0 blks 2024-04-15T15:18:45.023 INFO:teuthology.orchestra.run.smithi154.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-15T15:18:45.023 INFO:teuthology.orchestra.run.smithi154.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-15T15:18:45.023 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-15T15:18:45.024 INFO:teuthology.orchestra.run.smithi154.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-15T15:18:45.507 INFO:teuthology.orchestra.run.smithi154.stdout:Discarding blocks...Done. 2024-04-15T15:18:45.509 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi154.front.sepia.ceph.com -o noatime 2024-04-15T15:18:45.509 DEBUG:teuthology.orchestra.run.smithi154:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-15T15:18:45.637 DEBUG:teuthology.orchestra.run.smithi154:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-15T15:18:45.672 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-15T15:18:45.741 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-15T15:18:45.741 INFO:tasks.ceph:role: osd.7 2024-04-15T15:18:45.741 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi154.front.sepia.ceph.com 2024-04-15T15:18:45.742 DEBUG:teuthology.orchestra.run.smithi154:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-15T15:18:45.817 INFO:teuthology.orchestra.run.smithi154.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-15T15:18:45.817 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-15T15:18:45.817 INFO:teuthology.orchestra.run.smithi154.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-15T15:18:45.817 INFO:teuthology.orchestra.run.smithi154.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-15T15:18:45.817 INFO:teuthology.orchestra.run.smithi154.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-15T15:18:45.817 INFO:teuthology.orchestra.run.smithi154.stdout: = sunit=0 swidth=0 blks 2024-04-15T15:18:45.817 INFO:teuthology.orchestra.run.smithi154.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-15T15:18:45.818 INFO:teuthology.orchestra.run.smithi154.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-15T15:18:45.818 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-15T15:18:45.818 INFO:teuthology.orchestra.run.smithi154.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-15T15:18:46.348 INFO:teuthology.orchestra.run.smithi154.stdout:Discarding blocks...Done. 2024-04-15T15:18:46.350 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi154.front.sepia.ceph.com -o noatime 2024-04-15T15:18:46.350 DEBUG:teuthology.orchestra.run.smithi154:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-15T15:18:46.489 DEBUG:teuthology.orchestra.run.smithi154:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-15T15:18:46.524 DEBUG:teuthology.orchestra.run.smithi154:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-15T15:18:46.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:18:46.627+0000 7f6970536640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-15T15:18:46.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:18:46.627+0000 7f6970536640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-15T15:18:46.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:18:46.627+0000 7f6970536640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-15T15:18:46.631 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:18:46.627+0000 7f6970536640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-15T15:18:46.631 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:18:46.627+0000 7f6970536640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-15T15:18:46.631 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:18:46.628+0000 7f6970536640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-15T15:18:49.485 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-15T15:18:49.519 DEBUG:teuthology.orchestra.run.smithi154:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-15T15:18:49.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:18:49.611+0000 7fead4011640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-15T15:18:49.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:18:49.611+0000 7fead4011640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-15T15:18:49.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:18:49.611+0000 7fead4011640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-15T15:18:49.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:18:49.611+0000 7fead4011640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-15T15:18:49.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:18:49.611+0000 7fead4011640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-15T15:18:49.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:18:49.612+0000 7fead4011640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-15T15:18:52.459 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-15T15:18:52.493 DEBUG:teuthology.orchestra.run.smithi154:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-15T15:18:52.585 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:18:52.583+0000 7ff993e19640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-15T15:18:52.585 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:18:52.583+0000 7ff993e19640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-15T15:18:52.586 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:18:52.583+0000 7ff993e19640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-15T15:18:52.586 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:18:52.583+0000 7ff993e19640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-15T15:18:52.586 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:18:52.583+0000 7ff993e19640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-15T15:18:52.586 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:18:52.584+0000 7ff993e19640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-15T15:18:55.034 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-15T15:18:55.037 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-15T15:18:55.439 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-15T15:18:55.470 DEBUG:teuthology.orchestra.run.smithi154:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-15T15:18:55.563 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:18:55.561+0000 7fd1e600d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-15T15:18:55.563 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:18:55.561+0000 7fd1e600d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-15T15:18:55.564 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:18:55.561+0000 7fd1e600d640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-15T15:18:55.564 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:18:55.562+0000 7fd1e600d640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-15T15:18:55.564 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:18:55.562+0000 7fd1e600d640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-15T15:18:55.564 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:18:55.562+0000 7fd1e600d640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-15T15:18:58.420 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-15T15:18:58.454 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-15T15:18:58.454 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-04-15T15:18:58.454 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-15T15:18:58.488 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-04-15T15:18:58.488 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-15T15:18:58.558 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-04-15T15:18:58.559 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-15T15:18:58.629 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-04-15T15:18:58.629 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-15T15:18:58.699 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-04-15T15:18:58.699 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-15T15:18:58.768 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-04-15T15:18:58.769 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-15T15:18:58.838 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-04-15T15:18:58.838 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-15T15:18:58.906 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-15T15:18:58.906 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-15T15:18:58.940 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-15T15:18:58.940 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-15T15:18:59.013 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-15T15:18:59.013 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-15T15:18:59.085 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-15T15:18:59.085 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-15T15:18:59.156 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-15T15:18:59.156 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-15T15:18:59.226 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-15T15:18:59.226 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-15T15:18:59.296 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-15T15:18:59.296 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-15T15:18:59.364 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-04-15T15:18:59.365 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-15T15:18:59.383 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-04-15T15:18:59.383 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-15T15:18:59.439 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-15T15:18:59.439 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-15T15:18:59.455 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-15T15:18:59.455 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-15T15:18:59.512 INFO:tasks.ceph:Adding keys to all mons... 2024-04-15T15:18:59.512 DEBUG:teuthology.orchestra.run.smithi105:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-15T15:18:59.515 DEBUG:teuthology.orchestra.run.smithi154:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-15T15:18:59.556 INFO:teuthology.orchestra.run.smithi105.stdout:[mgr.y] 2024-04-15T15:18:59.556 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQBDRR1mNFKxHxAAsjYdZCnA7fcIVzXB9FWxXg== 2024-04-15T15:18:59.556 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.a] 2024-04-15T15:18:59.556 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQBDRR1myeBfJxAAvHlKDMW3De4FFrvZ0r0WRw== 2024-04-15T15:18:59.556 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.c] 2024-04-15T15:18:59.556 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQBDRR1m8HsELxAAe4IxWY/OGr9Jr4n8k+JhvQ== 2024-04-15T15:18:59.556 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.0] 2024-04-15T15:18:59.556 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQBHRR1mT7whIBAABC1sgtSsUmjcvEyhTT5oPA== 2024-04-15T15:18:59.556 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.1] 2024-04-15T15:18:59.557 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQBKRR1mNulAHxAAMfDgB2AsfxPimbWOcJATHA== 2024-04-15T15:18:59.557 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.2] 2024-04-15T15:18:59.557 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQBNRR1mQML+HRAAkQe3VRgWiylsdC5EZrDE5Q== 2024-04-15T15:18:59.557 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.3] 2024-04-15T15:18:59.557 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQBQRR1mxLduHBAAaqVeV0mMeqXIrNyKDws4iA== 2024-04-15T15:18:59.557 INFO:teuthology.orchestra.run.smithi105.stdout:[mgr.x] 2024-04-15T15:18:59.557 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQBDRR1mmkXDIxAAz2Dpih9eXQIHLl94qsCjvQ== 2024-04-15T15:18:59.557 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.b] 2024-04-15T15:18:59.557 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQBDRR1mTM5dNBAA36E7C2eeFskEfxTIBTwRJw== 2024-04-15T15:18:59.557 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.d] 2024-04-15T15:18:59.557 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQBERR1m2op6ABAAVnXL6UQjmdYDgaihrxNqjQ== 2024-04-15T15:18:59.557 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.4] 2024-04-15T15:18:59.558 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQBWRR1m2gqGJRAApBMj7eVuqtTUDJBgl4nGag== 2024-04-15T15:18:59.558 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.5] 2024-04-15T15:18:59.558 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQBZRR1mcGR9JBAArgDqBSvm3q6wF28o+DzCRg== 2024-04-15T15:18:59.558 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.6] 2024-04-15T15:18:59.558 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQBcRR1mOYfWIhAA+CwyZLGcoScCZXdAYih9xw== 2024-04-15T15:18:59.558 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.7] 2024-04-15T15:18:59.558 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQBfRR1mwdeIIRAAwPA5PsCibACLK3Eydl1YlA== 2024-04-15T15:18:59.558 INFO:teuthology.orchestra.run.smithi105.stdout:[client.0] 2024-04-15T15:18:59.558 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQBERR1mNnrzBBAAg6vNvYozUQe3sI+Xxt0QvA== 2024-04-15T15:18:59.558 INFO:teuthology.orchestra.run.smithi105.stdout:[client.2] 2024-04-15T15:18:59.558 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQBERR1mUf2WCBAACYRnwP9EJeGOTdRVDHIjhA== 2024-04-15T15:18:59.558 INFO:teuthology.orchestra.run.smithi105.stdout:[client.1] 2024-04-15T15:18:59.558 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQBERR1mvcwqDBAAPw/HCrSX2PdKn2owrVjJXQ== 2024-04-15T15:18:59.558 INFO:teuthology.orchestra.run.smithi105.stdout:[client.3] 2024-04-15T15:18:59.559 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQBERR1mBV/EDxAAeGJ4NqcoKlf7sJMH9klH/w== 2024-04-15T15:18:59.579 INFO:teuthology.orchestra.run.smithi154.stdout:[mgr.y] 2024-04-15T15:18:59.580 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQBDRR1mNFKxHxAAsjYdZCnA7fcIVzXB9FWxXg== 2024-04-15T15:18:59.580 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.a] 2024-04-15T15:18:59.580 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQBDRR1myeBfJxAAvHlKDMW3De4FFrvZ0r0WRw== 2024-04-15T15:18:59.580 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.c] 2024-04-15T15:18:59.580 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQBDRR1m8HsELxAAe4IxWY/OGr9Jr4n8k+JhvQ== 2024-04-15T15:18:59.580 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.0] 2024-04-15T15:18:59.580 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQBHRR1mT7whIBAABC1sgtSsUmjcvEyhTT5oPA== 2024-04-15T15:18:59.580 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.1] 2024-04-15T15:18:59.580 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQBKRR1mNulAHxAAMfDgB2AsfxPimbWOcJATHA== 2024-04-15T15:18:59.580 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.2] 2024-04-15T15:18:59.580 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQBNRR1mQML+HRAAkQe3VRgWiylsdC5EZrDE5Q== 2024-04-15T15:18:59.580 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.3] 2024-04-15T15:18:59.580 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQBQRR1mxLduHBAAaqVeV0mMeqXIrNyKDws4iA== 2024-04-15T15:18:59.581 INFO:teuthology.orchestra.run.smithi154.stdout:[mgr.x] 2024-04-15T15:18:59.581 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQBDRR1mmkXDIxAAz2Dpih9eXQIHLl94qsCjvQ== 2024-04-15T15:18:59.581 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.b] 2024-04-15T15:18:59.581 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQBDRR1mTM5dNBAA36E7C2eeFskEfxTIBTwRJw== 2024-04-15T15:18:59.581 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.d] 2024-04-15T15:18:59.581 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQBERR1m2op6ABAAVnXL6UQjmdYDgaihrxNqjQ== 2024-04-15T15:18:59.581 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.4] 2024-04-15T15:18:59.581 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQBWRR1m2gqGJRAApBMj7eVuqtTUDJBgl4nGag== 2024-04-15T15:18:59.581 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.5] 2024-04-15T15:18:59.581 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQBZRR1mcGR9JBAArgDqBSvm3q6wF28o+DzCRg== 2024-04-15T15:18:59.581 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.6] 2024-04-15T15:18:59.581 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQBcRR1mOYfWIhAA+CwyZLGcoScCZXdAYih9xw== 2024-04-15T15:18:59.581 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.7] 2024-04-15T15:18:59.581 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQBfRR1mwdeIIRAAwPA5PsCibACLK3Eydl1YlA== 2024-04-15T15:18:59.581 INFO:teuthology.orchestra.run.smithi154.stdout:[client.0] 2024-04-15T15:18:59.582 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQBERR1mNnrzBBAAg6vNvYozUQe3sI+Xxt0QvA== 2024-04-15T15:18:59.582 INFO:teuthology.orchestra.run.smithi154.stdout:[client.2] 2024-04-15T15:18:59.582 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQBERR1mUf2WCBAACYRnwP9EJeGOTdRVDHIjhA== 2024-04-15T15:18:59.582 INFO:teuthology.orchestra.run.smithi154.stdout:[client.1] 2024-04-15T15:18:59.582 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQBERR1mvcwqDBAAPw/HCrSX2PdKn2owrVjJXQ== 2024-04-15T15:18:59.582 INFO:teuthology.orchestra.run.smithi154.stdout:[client.3] 2024-04-15T15:18:59.582 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQBERR1mBV/EDxAAeGJ4NqcoKlf7sJMH9klH/w== 2024-04-15T15:18:59.583 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-15T15:18:59.602 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-15T15:18:59.678 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-15T15:18:59.680 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-15T15:18:59.737 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-15T15:18:59.740 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-15T15:18:59.797 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-15T15:18:59.799 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-15T15:18:59.853 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-15T15:18:59.855 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-15T15:18:59.909 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-15T15:18:59.911 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-15T15:18:59.965 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-15T15:18:59.968 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-15T15:19:00.023 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-15T15:19:00.025 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-15T15:19:00.079 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-15T15:19:00.081 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-15T15:19:00.138 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-15T15:19:00.141 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-15T15:19:00.198 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-15T15:19:00.200 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-15T15:19:00.253 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-15T15:19:00.256 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-15T15:19:00.310 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-15T15:19:00.312 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-15T15:19:00.366 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-15T15:19:00.368 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-15T15:19:00.422 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-15T15:19:00.424 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-15T15:19:00.481 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-15T15:19:00.484 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-15T15:19:00.540 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-15T15:19:00.543 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-15T15:19:00.599 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-15T15:19:00.602 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-15T15:19:00.659 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-15T15:19:00.659 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-15T15:19:00.687 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-15T15:19:00.786 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-15T15:19:00.814 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-15T15:19:00.843 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-15T15:19:00.943 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-15T15:19:00.972 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-15T15:19:01.041 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-15T15:19:01.142 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-15T15:19:01.170 DEBUG:teuthology.orchestra.run.smithi105:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-15T15:19:01.173 DEBUG:teuthology.orchestra.run.smithi154:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-15T15:19:01.226 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-15T15:19:01.226 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-15T15:19:01.226 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-15T15:19:01.228 INFO:tasks.ceph.mon.a:Started 2024-04-15T15:19:01.229 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-15T15:19:01.229 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-15T15:19:01.268 INFO:tasks.ceph.mon.b:Started 2024-04-15T15:19:01.268 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-15T15:19:01.268 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-15T15:19:01.270 INFO:tasks.ceph.mon.c:Started 2024-04-15T15:19:01.270 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-15T15:19:01.270 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-15T15:19:01.271 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-15T15:19:01.273 INFO:tasks.ceph.mgr.y:Started 2024-04-15T15:19:01.273 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-15T15:19:01.273 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-15T15:19:01.275 INFO:tasks.ceph.mgr.x:Started 2024-04-15T15:19:01.275 INFO:tasks.ceph:Setting crush tunables to default 2024-04-15T15:19:01.275 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph osd crush tunables default 2024-04-15T15:19:01.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.376+0000 7f8cd0ad9640 1 Processor -- start 2024-04-15T15:19:01.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.376+0000 7f8cd0ad9640 1 -- start start 2024-04-15T15:19:01.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.377+0000 7f8cd0ad9640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ccc155130 0x7f8ccc06eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:01.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.377+0000 7f8cd0ad9640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8ccc0b5110 0x7f8ccc15e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:01.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.377+0000 7f8cd0ad9640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8ccc15eda0 0x7f8ccc15b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:01.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.377+0000 7f8cd0ad9640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f8ccc15f1c0 con 0x7f8ccc0b5110 2024-04-15T15:19:01.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.377+0000 7f8cd0ad9640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f8ccc06f4d0 con 0x7f8ccc15eda0 2024-04-15T15:19:01.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.377+0000 7f8cd0ad9640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f8ccc06f7b0 con 0x7f8ccc155130 2024-04-15T15:19:01.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.377+0000 7f8cc1d9b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8ccc0b5110 0x7f8ccc15e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:01.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.377+0000 7f8cc1d9b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8ccc0b5110 0x7f8ccc15e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55526/0 (socket says 172.21.15.105:55526) 2024-04-15T15:19:01.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.377+0000 7f8cc1d9b640 1 -- 172.21.15.105:0/1792196936 learned_addr learned my addr 172.21.15.105:0/1792196936 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:01.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.377+0000 7f8cc1d9b640 1 -- 172.21.15.105:0/1792196936 >> v2:172.21.15.105:3300/0 conn(0x7f8ccc0b5110 msgr2=0x7f8ccc15e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-15T15:19:01.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.377+0000 7f8cc1d9b640 1 -- 172.21.15.105:0/1792196936 >> v2:172.21.15.105:3300/0 conn(0x7f8ccc0b5110 msgr2=0x7f8ccc15e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-15T15:19:01.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.377+0000 7f8cc1d9b640 1 --2- 172.21.15.105:0/1792196936 >> v2:172.21.15.105:3300/0 conn(0x7f8ccc0b5110 0x7f8ccc15e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-15T15:19:01.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.377+0000 7f8cc1d9b640 1 --2- 172.21.15.105:0/1792196936 >> v2:172.21.15.105:3300/0 conn(0x7f8ccc0b5110 0x7f8ccc15e860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-15T15:19:01.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.377+0000 7f8cca59c640 1 --2- 172.21.15.105:0/1792196936 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ccc155130 0x7f8ccc06eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:01.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.377+0000 7f8ccad9d640 1 --2- 172.21.15.105:0/1792196936 >> v2:172.21.15.154:3300/0 conn(0x7f8ccc15eda0 0x7f8ccc15b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:01.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.377+0000 7f8cca59c640 1 -- 172.21.15.105:0/1792196936 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ccc155130 msgr2=0x7f8ccc06eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-15T15:19:01.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.377+0000 7f8cca59c640 1 -- 172.21.15.105:0/1792196936 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ccc155130 msgr2=0x7f8ccc06eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-15T15:19:01.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.377+0000 7f8cca59c640 1 --2- 172.21.15.105:0/1792196936 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ccc155130 0x7f8ccc06eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-15T15:19:01.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.377+0000 7f8cca59c640 1 --2- 172.21.15.105:0/1792196936 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ccc155130 0x7f8ccc06eef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-15T15:19:01.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.377+0000 7f8ccad9d640 1 -- 172.21.15.105:0/1792196936 >> v2:172.21.15.154:3300/0 conn(0x7f8ccc15eda0 msgr2=0x7f8ccc15b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-15T15:19:01.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.377+0000 7f8ccad9d640 1 -- 172.21.15.105:0/1792196936 >> v2:172.21.15.154:3300/0 conn(0x7f8ccc15eda0 msgr2=0x7f8ccc15b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-15T15:19:01.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.377+0000 7f8ccad9d640 1 --2- 172.21.15.105:0/1792196936 >> v2:172.21.15.154:3300/0 conn(0x7f8ccc15eda0 0x7f8ccc15b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-15T15:19:01.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.377+0000 7f8ccad9d640 1 --2- 172.21.15.105:0/1792196936 >> v2:172.21.15.154:3300/0 conn(0x7f8ccc15eda0 0x7f8ccc15b270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-15T15:19:01.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.578+0000 7f8cc1d9b640 1 --2- 172.21.15.105:0/1792196936 >> v2:172.21.15.105:3300/0 conn(0x7f8ccc0b5110 0x7f8ccc15e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:01.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.578+0000 7f8cca59c640 1 --2- 172.21.15.105:0/1792196936 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ccc155130 0x7f8ccc06eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:01.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.578+0000 7f8ccad9d640 1 --2- 172.21.15.105:0/1792196936 >> v2:172.21.15.154:3300/0 conn(0x7f8ccc15eda0 0x7f8ccc15b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:01.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.578+0000 7f8cc1d9b640 1 -- 172.21.15.105:0/1792196936 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ccc155130 msgr2=0x7f8ccc06eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:01.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.578+0000 7f8cc1d9b640 1 --2- 172.21.15.105:0/1792196936 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ccc155130 0x7f8ccc06eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:01.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.578+0000 7f8cc1d9b640 1 -- 172.21.15.105:0/1792196936 >> v2:172.21.15.154:3300/0 conn(0x7f8ccc15eda0 msgr2=0x7f8ccc15b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:01.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.578+0000 7f8cc1d9b640 1 --2- 172.21.15.105:0/1792196936 >> v2:172.21.15.154:3300/0 conn(0x7f8ccc15eda0 0x7f8ccc15b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:01.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.578+0000 7f8cc1d9b640 1 -- 172.21.15.105:0/1792196936 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ccc06fa90 con 0x7f8ccc0b5110 2024-04-15T15:19:01.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.578+0000 7f8ccad9d640 1 --2- 172.21.15.105:0/1792196936 >> v2:172.21.15.154:3300/0 conn(0x7f8ccc15eda0 0x7f8ccc15b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:19:01.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.579+0000 7f8cc1d9b640 1 --2- 172.21.15.105:0/1792196936 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ccc0b5110 0x7f8ccc15e860 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc0d1c90 tx=0x7f8cb402bc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ba0fae0bcebc129 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:01.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.579+0000 7f8cc9d9b640 1 -- 172.21.15.105:0/1792196936 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb4039070 con 0x7f8ccc0b5110 2024-04-15T15:19:01.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.579+0000 7f8cc9d9b640 1 -- 172.21.15.105:0/1792196936 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cb402ca20 con 0x7f8ccc0b5110 2024-04-15T15:19:01.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.579+0000 7f8cc9d9b640 1 -- 172.21.15.105:0/1792196936 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb400f040 con 0x7f8ccc0b5110 2024-04-15T15:19:01.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.579+0000 7f8cd0ad9640 1 -- 172.21.15.105:0/1792196936 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ccc0b5110 msgr2=0x7f8ccc15e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:01.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.579+0000 7f8cd0ad9640 1 --2- 172.21.15.105:0/1792196936 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ccc0b5110 0x7f8ccc15e860 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc0d1c90 tx=0x7f8cb402bc70 comp rx=0 tx=0).stop 2024-04-15T15:19:01.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.580+0000 7f8cd0ad9640 1 -- 172.21.15.105:0/1792196936 shutdown_connections 2024-04-15T15:19:01.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.580+0000 7f8cd0ad9640 1 --2- 172.21.15.105:0/1792196936 >> v2:172.21.15.154:3300/0 conn(0x7f8ccc15eda0 0x7f8ccc15b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:01.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.580+0000 7f8cd0ad9640 1 --2- 172.21.15.105:0/1792196936 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ccc0b5110 0x7f8ccc15e860 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:01.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.580+0000 7f8cd0ad9640 1 --2- 172.21.15.105:0/1792196936 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ccc155130 0x7f8ccc06eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:01.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.580+0000 7f8cd0ad9640 1 -- 172.21.15.105:0/1792196936 >> 172.21.15.105:0/1792196936 conn(0x7f8ccc0b3380 msgr2=0x7f8ccc0d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:01.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.580+0000 7f8cd0ad9640 1 -- 172.21.15.105:0/1792196936 shutdown_connections 2024-04-15T15:19:01.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.580+0000 7f8cd0ad9640 1 -- 172.21.15.105:0/1792196936 wait complete. 2024-04-15T15:19:01.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.581+0000 7f8cd0ad9640 1 Processor -- start 2024-04-15T15:19:01.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.581+0000 7f8cd0ad9640 1 -- start start 2024-04-15T15:19:01.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.582+0000 7f8cd0ad9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ccc0b5110 0x7f8ccc0b1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:01.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.582+0000 7f8cd0ad9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ccc155130 0x7f8ccc1e5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:01.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.582+0000 7f8cd0ad9640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ccc15eda0 0x7f8ccc1e5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:01.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.582+0000 7f8cd0ad9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc155510 con 0x7f8ccc155130 2024-04-15T15:19:01.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.582+0000 7f8cd0ad9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc070050 con 0x7f8ccc0b5110 2024-04-15T15:19:01.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.582+0000 7f8cd0ad9640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f8ccc155690 con 0x7f8ccc15eda0 2024-04-15T15:19:01.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.582+0000 7f8cc1d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ccc155130 0x7f8ccc1e5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:01.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.582+0000 7f8cc1d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ccc155130 0x7f8ccc1e5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55568/0 (socket says 172.21.15.105:55568) 2024-04-15T15:19:01.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.582+0000 7f8cc1d9b640 1 -- 172.21.15.105:0/1566802806 learned_addr learned my addr 172.21.15.105:0/1566802806 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:01.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.582+0000 7f8ccad9d640 1 --2- 172.21.15.105:0/1566802806 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ccc15eda0 0x7f8ccc1e5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:01.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.582+0000 7f8cca59c640 1 --2- 172.21.15.105:0/1566802806 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ccc0b5110 0x7f8ccc0b1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:01.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.582+0000 7f8cc1d9b640 1 -- 172.21.15.105:0/1566802806 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ccc15eda0 msgr2=0x7f8ccc1e5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:01.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.582+0000 7f8cc1d9b640 1 --2- 172.21.15.105:0/1566802806 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ccc15eda0 0x7f8ccc1e5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:01.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.582+0000 7f8cc1d9b640 1 -- 172.21.15.105:0/1566802806 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ccc0b5110 msgr2=0x7f8ccc0b1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:01.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.582+0000 7f8cc1d9b640 1 --2- 172.21.15.105:0/1566802806 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ccc0b5110 0x7f8ccc0b1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:01.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.582+0000 7f8cc1d9b640 1 -- 172.21.15.105:0/1566802806 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ccc0b23d0 con 0x7f8ccc155130 2024-04-15T15:19:01.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.583+0000 7f8cc1d9b640 1 --2- 172.21.15.105:0/1566802806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ccc155130 0x7f8ccc1e5000 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8cb4037040 tx=0x7f8cb4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:01.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.583+0000 7f8cc37fe640 1 -- 172.21.15.105:0/1566802806 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb4044020 con 0x7f8ccc155130 2024-04-15T15:19:01.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.583+0000 7f8cc37fe640 1 -- 172.21.15.105:0/1566802806 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cb4003610 con 0x7f8ccc155130 2024-04-15T15:19:01.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.583+0000 7f8cc37fe640 1 -- 172.21.15.105:0/1566802806 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb4039040 con 0x7f8ccc155130 2024-04-15T15:19:01.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.583+0000 7f8cd0ad9640 1 -- 172.21.15.105:0/1566802806 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ccc0b20f0 con 0x7f8ccc155130 2024-04-15T15:19:01.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.583+0000 7f8cd0ad9640 1 -- 172.21.15.105:0/1566802806 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ccc151860 con 0x7f8ccc155130 2024-04-15T15:19:01.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.583+0000 7f8cc37fe640 1 -- 172.21.15.105:0/1566802806 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8cb4047020 con 0x7f8ccc155130 2024-04-15T15:19:01.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.583+0000 7f8cc37fe640 1 -- 172.21.15.105:0/1566802806 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8cb40064b0 con 0x7f8ccc155130 2024-04-15T15:19:01.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.584+0000 7f8cd0ad9640 1 -- 172.21.15.105:0/1566802806 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ccc069150 con 0x7f8ccc155130 2024-04-15T15:19:01.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.586+0000 7f8cc37fe640 1 -- 172.21.15.105:0/1566802806 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f8cb404e070 con 0x7f8ccc155130 2024-04-15T15:19:01.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.652+0000 7f8cd0ad9640 1 -- 172.21.15.105:0/1566802806 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f8ccc0ccd10 con 0x7f8ccc155130 2024-04-15T15:19:01.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.653+0000 7f8cc37fe640 1 -- 172.21.15.105:0/1566802806 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f8cb4006bd0 con 0x7f8ccc155130 2024-04-15T15:19:01.656 INFO:teuthology.orchestra.run.smithi105.stderr:adjusted tunables profile to default 2024-04-15T15:19:01.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.654+0000 7f8cd0ad9640 1 -- 172.21.15.105:0/1566802806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ccc155130 msgr2=0x7f8ccc1e5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:01.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.654+0000 7f8cd0ad9640 1 --2- 172.21.15.105:0/1566802806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ccc155130 0x7f8ccc1e5000 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8cb4037040 tx=0x7f8cb4004730 comp rx=0 tx=0).stop 2024-04-15T15:19:01.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.654+0000 7f8cd0ad9640 1 -- 172.21.15.105:0/1566802806 shutdown_connections 2024-04-15T15:19:01.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.654+0000 7f8cd0ad9640 1 --2- 172.21.15.105:0/1566802806 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ccc15eda0 0x7f8ccc1e5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:01.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.654+0000 7f8cd0ad9640 1 --2- 172.21.15.105:0/1566802806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ccc155130 0x7f8ccc1e5000 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:01.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.654+0000 7f8cd0ad9640 1 --2- 172.21.15.105:0/1566802806 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ccc0b5110 0x7f8ccc0b1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:01.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.654+0000 7f8cd0ad9640 1 -- 172.21.15.105:0/1566802806 >> 172.21.15.105:0/1566802806 conn(0x7f8ccc0b3380 msgr2=0x7f8ccc154700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:01.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.655+0000 7f8cd0ad9640 1 -- 172.21.15.105:0/1566802806 shutdown_connections 2024-04-15T15:19:01.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:01.655+0000 7f8cd0ad9640 1 -- 172.21.15.105:0/1566802806 wait complete. 2024-04-15T15:19:01.668 INFO:tasks.ceph:check_enable_crimson: False 2024-04-15T15:19:01.668 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-15T15:19:01.668 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-04-15T15:19:01.669 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-15T15:19:01.697 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-04-15T15:19:01.697 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-15T15:19:01.764 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-04-15T15:19:01.764 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-15T15:19:01.833 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-04-15T15:19:01.833 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-15T15:19:01.901 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-15T15:19:01.902 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-15T15:19:01.930 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-15T15:19:01.931 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-15T15:19:02.000 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-15T15:19:02.000 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-15T15:19:02.014 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-04-15T15:19:02.012+0000 7f64fa816040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-15T15:19:02.041 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-04-15T15:19:02.040+0000 7ff178e16040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-15T15:19:02.043 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-15T15:19:02.043 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-15T15:19:02.072 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-04-15T15:19:02.071+0000 7f64fa816040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-15T15:19:02.094 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-04-15T15:19:02.093+0000 7ff178e16040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-15T15:19:02.101 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd new b6f6292b-2d7e-4aa0-9d1c-6ddd0f97287c 0 2024-04-15T15:19:02.244 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.242+0000 7fbbb6d8f640 1 Processor -- start 2024-04-15T15:19:02.244 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.242+0000 7fbbb6d8f640 1 -- start start 2024-04-15T15:19:02.245 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.244+0000 7fbbb6d8f640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbbb009b150 0x7fbbb00b1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:02.246 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.244+0000 7fbbb6d8f640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbb00d0e10 con 0x7fbbb00b3450 2024-04-15T15:19:02.246 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.244+0000 7fbbb6d8f640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbb00d10f0 con 0x7fbbb015de50 2024-04-15T15:19:02.246 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.244+0000 7fbbb6d8f640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fbbb00d0bc0 con 0x7fbbb009b150 2024-04-15T15:19:02.246 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.244+0000 7fbbb532c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbbb009b150 0x7fbbb00b1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:02.246 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.244+0000 7fbbb532c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbbb009b150 0x7fbbb00b1dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50756/0 (socket says 172.21.15.154:50756) 2024-04-15T15:19:02.246 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.244+0000 7fbbb532c640 1 -- 172.21.15.154:0/3807657679 learned_addr learned my addr 172.21.15.154:0/3807657679 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-04-15T15:19:02.246 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.244+0000 7fbbb532c640 1 -- 172.21.15.154:0/3807657679 >> v1:172.21.15.154:6789/0 conn(0x7fbbb015de50 legacy=0x7fbbb009b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:02.246 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.244+0000 7fbbb532c640 1 -- 172.21.15.154:0/3807657679 >> v1:172.21.15.105:6789/0 conn(0x7fbbb00b3450 legacy=0x7fbbb009aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:02.246 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.244+0000 7fbbb532c640 1 -- 172.21.15.154:0/3807657679 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbb00cd1d0 con 0x7fbbb009b150 2024-04-15T15:19:02.247 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.245+0000 7fbbb532c640 1 --2- 172.21.15.154:0/3807657679 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbbb009b150 0x7fbbb00b1dd0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbba8009e50 tx=0x7fbba802c010 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f68e0e99f6c26f server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:02.247 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.245+0000 7fbbaf7fe640 1 -- 172.21.15.154:0/3807657679 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba802cbb0 con 0x7fbbb009b150 2024-04-15T15:19:02.247 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.245+0000 7fbbaf7fe640 1 -- 172.21.15.154:0/3807657679 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbba802cd50 con 0x7fbbb009b150 2024-04-15T15:19:02.247 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.245+0000 7fbbaf7fe640 1 -- 172.21.15.154:0/3807657679 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba8034610 con 0x7fbbb009b150 2024-04-15T15:19:02.248 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.246+0000 7fbbb6d8f640 1 -- 172.21.15.154:0/3807657679 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbbb009b150 msgr2=0x7fbbb00b1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:02.248 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.246+0000 7fbbb6d8f640 1 --2- 172.21.15.154:0/3807657679 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbbb009b150 0x7fbbb00b1dd0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbba8009e50 tx=0x7fbba802c010 comp rx=0 tx=0).stop 2024-04-15T15:19:02.248 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.246+0000 7fbbb6d8f640 1 -- 172.21.15.154:0/3807657679 shutdown_connections 2024-04-15T15:19:02.248 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.246+0000 7fbbb6d8f640 1 --2- 172.21.15.154:0/3807657679 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbbb009b150 0x7fbbb00b1dd0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:02.248 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.246+0000 7fbbb6d8f640 1 -- 172.21.15.154:0/3807657679 >> 172.21.15.154:0/3807657679 conn(0x7fbbb00d25e0 msgr2=0x7fbbb00d29f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:02.248 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.246+0000 7fbbb6d8f640 1 -- 172.21.15.154:0/3807657679 shutdown_connections 2024-04-15T15:19:02.248 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.246+0000 7fbbb6d8f640 1 -- 172.21.15.154:0/3807657679 wait complete. 2024-04-15T15:19:02.249 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.247+0000 7fbbb6d8f640 1 Processor -- start 2024-04-15T15:19:02.249 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.247+0000 7fbbb6d8f640 1 -- start start 2024-04-15T15:19:02.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.248+0000 7fbbb6d8f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbbb009b150 0x7fbbb00cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:02.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.248+0000 7fbbb6d8f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbbb00b3450 0x7fbbb01e7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:02.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.248+0000 7fbbb6d8f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbbb015de50 0x7fbbb01f4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:02.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.248+0000 7fbbb6d8f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb00cd870 con 0x7fbbb009b150 2024-04-15T15:19:02.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.248+0000 7fbbb6d8f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb00cd6f0 con 0x7fbbb015de50 2024-04-15T15:19:02.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.248+0000 7fbbb6d8f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fbbb00cd9f0 con 0x7fbbb00b3450 2024-04-15T15:19:02.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.248+0000 7fbbaffff640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbbb00b3450 0x7fbbb01e7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:02.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.248+0000 7fbbb532c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbbb015de50 0x7fbbb01f4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:02.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.248+0000 7fbbb532c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbbb015de50 0x7fbbb01f4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50764/0 (socket says 172.21.15.154:50764) 2024-04-15T15:19:02.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.248+0000 7fbbaffff640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbbb00b3450 0x7fbbb01e7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.154:37838/0 (socket says 172.21.15.154:37838) 2024-04-15T15:19:02.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.248+0000 7fbbb532c640 1 -- 172.21.15.154:0/2752909843 learned_addr learned my addr 172.21.15.154:0/2752909843 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-04-15T15:19:02.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.248+0000 7fbbb4b2b640 1 --2- 172.21.15.154:0/2752909843 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbbb009b150 0x7fbbb00cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:02.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.248+0000 7fbbb532c640 1 -- 172.21.15.154:0/2752909843 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbbb00b3450 msgr2=0x7fbbb01e7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:02.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.248+0000 7fbbb532c640 1 --2- 172.21.15.154:0/2752909843 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbbb00b3450 0x7fbbb01e7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:02.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.248+0000 7fbbb532c640 1 -- 172.21.15.154:0/2752909843 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbbb009b150 msgr2=0x7fbbb00cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:02.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.248+0000 7fbbb532c640 1 --2- 172.21.15.154:0/2752909843 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbbb009b150 0x7fbbb00cc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:02.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.248+0000 7fbbb532c640 1 -- 172.21.15.154:0/2752909843 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbb00cc8a0 con 0x7fbbb015de50 2024-04-15T15:19:02.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.248+0000 7fbbaffff640 1 --2- 172.21.15.154:0/2752909843 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbbb00b3450 0x7fbbb01e7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:19:02.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.249+0000 7fbbb532c640 1 --2- 172.21.15.154:0/2752909843 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbbb015de50 0x7fbbb01f4360 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbba8009e50 tx=0x7fbba8004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:02.253 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.249+0000 7fbbad7fa640 1 -- 172.21.15.154:0/2752909843 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba8043020 con 0x7fbbb015de50 2024-04-15T15:19:02.253 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.249+0000 7fbbad7fa640 1 -- 172.21.15.154:0/2752909843 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbba803e450 con 0x7fbbb015de50 2024-04-15T15:19:02.253 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.249+0000 7fbbb6d8f640 1 -- 172.21.15.154:0/2752909843 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbb006d3f0 con 0x7fbbb015de50 2024-04-15T15:19:02.253 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.249+0000 7fbbad7fa640 1 -- 172.21.15.154:0/2752909843 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba8009510 con 0x7fbbb015de50 2024-04-15T15:19:02.253 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.249+0000 7fbbb6d8f640 1 -- 172.21.15.154:0/2752909843 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbb006d9b0 con 0x7fbbb015de50 2024-04-15T15:19:02.253 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.249+0000 7fbbad7fa640 1 -- 172.21.15.154:0/2752909843 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbba8046020 con 0x7fbbb015de50 2024-04-15T15:19:02.253 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.249+0000 7fbbad7fa640 1 -- 172.21.15.154:0/2752909843 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7fbba8048530 con 0x7fbbb015de50 2024-04-15T15:19:02.253 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.250+0000 7fbbb6d8f640 1 -- 172.21.15.154:0/2752909843 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbb0069150 con 0x7fbbb015de50 2024-04-15T15:19:02.253 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.252+0000 7fbbad7fa640 1 -- 172.21.15.154:0/2752909843 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fbba804d030 con 0x7fbbb015de50 2024-04-15T15:19:02.326 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.324+0000 7fbbb6d8f640 1 -- 172.21.15.154:0/2752909843 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b6f6292b-2d7e-4aa0-9d1c-6ddd0f97287c", "id": 0} v 0) -- 0x7fbbb00d3200 con 0x7fbbb015de50 2024-04-15T15:19:02.329 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.327+0000 7fbbad7fa640 1 -- 172.21.15.154:0/2752909843 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b6f6292b-2d7e-4aa0-9d1c-6ddd0f97287c", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7fbba803e840 con 0x7fbbb015de50 2024-04-15T15:19:02.329 INFO:teuthology.orchestra.run.smithi154.stdout:0 2024-04-15T15:19:02.330 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.328+0000 7fbbb6d8f640 1 -- 172.21.15.154:0/2752909843 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbbb015de50 msgr2=0x7fbbb01f4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:02.330 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.328+0000 7fbbb6d8f640 1 --2- 172.21.15.154:0/2752909843 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbbb015de50 0x7fbbb01f4360 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbba8009e50 tx=0x7fbba8004a40 comp rx=0 tx=0).stop 2024-04-15T15:19:02.330 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.328+0000 7fbbb6d8f640 1 -- 172.21.15.154:0/2752909843 shutdown_connections 2024-04-15T15:19:02.330 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.328+0000 7fbbb6d8f640 1 --2- 172.21.15.154:0/2752909843 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbbb00b3450 0x7fbbb01e7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:02.331 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.328+0000 7fbbb6d8f640 1 --2- 172.21.15.154:0/2752909843 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbbb015de50 0x7fbbb01f4360 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:02.331 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.328+0000 7fbbb6d8f640 1 --2- 172.21.15.154:0/2752909843 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbbb009b150 0x7fbbb00cc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:02.331 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.328+0000 7fbbb6d8f640 1 -- 172.21.15.154:0/2752909843 >> 172.21.15.154:0/2752909843 conn(0x7fbbb00d25e0 msgr2=0x7fbbb0070dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:02.331 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.328+0000 7fbbb6d8f640 1 -- 172.21.15.154:0/2752909843 shutdown_connections 2024-04-15T15:19:02.331 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.328+0000 7fbbb6d8f640 1 -- 172.21.15.154:0/2752909843 wait complete. 2024-04-15T15:19:02.341 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd new 8900966a-b393-41e3-8f93-e5a43bd23622 1 2024-04-15T15:19:02.444 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.442+0000 7f6d30677640 1 Processor -- start 2024-04-15T15:19:02.444 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.442+0000 7f6d30677640 1 -- start start 2024-04-15T15:19:02.446 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.443+0000 7f6d30677640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6d2806b9c0 0x7f6d280cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:02.446 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.443+0000 7f6d30677640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6d2806bda0 0x7f6d2806f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:02.446 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.443+0000 7f6d30677640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d280d0840 con 0x7f6d2815ca00 2024-04-15T15:19:02.446 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.443+0000 7f6d30677640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f6d280d0610 con 0x7f6d2806b9c0 2024-04-15T15:19:02.446 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.443+0000 7f6d30677640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f6d280b4180 con 0x7f6d2806bda0 2024-04-15T15:19:02.446 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.444+0000 7f6d2ec14640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6d2806bda0 0x7f6d2806f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:02.446 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.444+0000 7f6d2ec14640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6d2806bda0 0x7f6d2806f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50774/0 (socket says 172.21.15.154:50774) 2024-04-15T15:19:02.446 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.444+0000 7f6d2ec14640 1 -- 172.21.15.154:0/1236533370 learned_addr learned my addr 172.21.15.154:0/1236533370 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-04-15T15:19:02.446 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.444+0000 7f6d2e413640 1 --2- 172.21.15.154:0/1236533370 >> v2:172.21.15.105:3300/0 conn(0x7f6d2806b9c0 0x7f6d280cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:02.446 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.444+0000 7f6d2ec14640 1 -- 172.21.15.154:0/1236533370 >> v2:172.21.15.105:3300/0 conn(0x7f6d2806b9c0 msgr2=0x7f6d280cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:02.446 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.444+0000 7f6d2ec14640 1 --2- 172.21.15.154:0/1236533370 >> v2:172.21.15.105:3300/0 conn(0x7f6d2806b9c0 0x7f6d280cd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:02.446 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.444+0000 7f6d2ec14640 1 -- 172.21.15.154:0/1236533370 >> v1:172.21.15.105:6789/0 conn(0x7f6d2815ca00 legacy=0x7f6d280b39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:02.447 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.444+0000 7f6d2ec14640 1 -- 172.21.15.154:0/1236533370 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d280d0b20 con 0x7f6d2806bda0 2024-04-15T15:19:02.447 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.444+0000 7f6d2e413640 1 --2- 172.21.15.154:0/1236533370 >> v2:172.21.15.105:3300/0 conn(0x7f6d2806b9c0 0x7f6d280cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:19:02.448 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.445+0000 7f6d2ec14640 1 --2- 172.21.15.154:0/1236533370 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d2806bda0 0x7f6d2806f230 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6d2809f890 tx=0x7f6d2002bf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d9c6951591cef26 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:02.448 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.445+0000 7f6d2d411640 1 -- 172.21.15.154:0/1236533370 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d2002cb20 con 0x7f6d2806bda0 2024-04-15T15:19:02.448 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.445+0000 7f6d2d411640 1 -- 172.21.15.154:0/1236533370 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d2002ccc0 con 0x7f6d2806bda0 2024-04-15T15:19:02.448 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.445+0000 7f6d2d411640 1 -- 172.21.15.154:0/1236533370 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d20034610 con 0x7f6d2806bda0 2024-04-15T15:19:02.448 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.445+0000 7f6d30677640 1 -- 172.21.15.154:0/1236533370 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d2806bda0 msgr2=0x7f6d2806f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:02.449 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.445+0000 7f6d30677640 1 --2- 172.21.15.154:0/1236533370 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d2806bda0 0x7f6d2806f230 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6d2809f890 tx=0x7f6d2002bf80 comp rx=0 tx=0).stop 2024-04-15T15:19:02.449 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.445+0000 7f6d30677640 1 -- 172.21.15.154:0/1236533370 shutdown_connections 2024-04-15T15:19:02.449 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.445+0000 7f6d30677640 1 --2- 172.21.15.154:0/1236533370 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d2806bda0 0x7f6d2806f230 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:02.449 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.445+0000 7f6d30677640 1 --2- 172.21.15.154:0/1236533370 >> v2:172.21.15.105:3300/0 conn(0x7f6d2806b9c0 0x7f6d280cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:02.449 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.445+0000 7f6d30677640 1 -- 172.21.15.154:0/1236533370 >> 172.21.15.154:0/1236533370 conn(0x7f6d2809aa40 msgr2=0x7f6d280b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:02.449 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.445+0000 7f6d30677640 1 -- 172.21.15.154:0/1236533370 shutdown_connections 2024-04-15T15:19:02.449 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.445+0000 7f6d30677640 1 -- 172.21.15.154:0/1236533370 wait complete. 2024-04-15T15:19:02.449 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.447+0000 7f6d30677640 1 Processor -- start 2024-04-15T15:19:02.449 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.447+0000 7f6d30677640 1 -- start start 2024-04-15T15:19:02.451 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.447+0000 7f6d30677640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d2806b9c0 0x7f6d2814b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:02.451 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.447+0000 7f6d30677640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6d2806bda0 0x7f6d280c7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:02.452 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.447+0000 7f6d30677640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d2815ca00 0x7f6d280c8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:02.452 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.447+0000 7f6d30677640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6d2806c1e0 con 0x7f6d2806b9c0 2024-04-15T15:19:02.452 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.447+0000 7f6d30677640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6d280d10e0 con 0x7f6d2815ca00 2024-04-15T15:19:02.452 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.447+0000 7f6d30677640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f6d2814bb00 con 0x7f6d2806bda0 2024-04-15T15:19:02.452 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.447+0000 7f6d2ec14640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d2815ca00 0x7f6d280c8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:02.452 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.447+0000 7f6d2dc12640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6d2806bda0 0x7f6d280c7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:02.452 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.448+0000 7f6d2ec14640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d2815ca00 0x7f6d280c8730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50790/0 (socket says 172.21.15.154:50790) 2024-04-15T15:19:02.452 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.448+0000 7f6d2ec14640 1 -- 172.21.15.154:0/2479319227 learned_addr learned my addr 172.21.15.154:0/2479319227 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-04-15T15:19:02.452 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.448+0000 7f6d2dc12640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6d2806bda0 0x7f6d280c7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.154:37846/0 (socket says 172.21.15.154:37846) 2024-04-15T15:19:02.452 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.448+0000 7f6d2e413640 1 --2- 172.21.15.154:0/2479319227 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d2806b9c0 0x7f6d2814b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:02.452 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.448+0000 7f6d2ec14640 1 -- 172.21.15.154:0/2479319227 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6d2806bda0 msgr2=0x7f6d280c7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:02.452 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.448+0000 7f6d2ec14640 1 --2- 172.21.15.154:0/2479319227 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6d2806bda0 0x7f6d280c7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:02.452 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.448+0000 7f6d2ec14640 1 -- 172.21.15.154:0/2479319227 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d2806b9c0 msgr2=0x7f6d2814b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:02.452 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.448+0000 7f6d2ec14640 1 --2- 172.21.15.154:0/2479319227 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d2806b9c0 0x7f6d2814b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:02.453 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.448+0000 7f6d2ec14640 1 -- 172.21.15.154:0/2479319227 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d2815d870 con 0x7f6d2815ca00 2024-04-15T15:19:02.453 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.448+0000 7f6d2dc12640 1 --2- 172.21.15.154:0/2479319227 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6d2806bda0 0x7f6d280c7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:19:02.453 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.448+0000 7f6d2ec14640 1 --2- 172.21.15.154:0/2479319227 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d2815ca00 0x7f6d280c8730 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6d20036040 tx=0x7f6d20004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:02.453 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.448+0000 7f6d1affd640 1 -- 172.21.15.154:0/2479319227 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d20043020 con 0x7f6d2815ca00 2024-04-15T15:19:02.453 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.448+0000 7f6d1affd640 1 -- 172.21.15.154:0/2479319227 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d2003e450 con 0x7f6d2815ca00 2024-04-15T15:19:02.453 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.448+0000 7f6d30677640 1 -- 172.21.15.154:0/2479319227 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d2815db50 con 0x7f6d2815ca00 2024-04-15T15:19:02.453 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.448+0000 7f6d1affd640 1 -- 172.21.15.154:0/2479319227 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d20009510 con 0x7f6d2815ca00 2024-04-15T15:19:02.453 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.448+0000 7f6d30677640 1 -- 172.21.15.154:0/2479319227 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d2815e110 con 0x7f6d2815ca00 2024-04-15T15:19:02.453 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.448+0000 7f6d1affd640 1 -- 172.21.15.154:0/2479319227 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6d20046020 con 0x7f6d2815ca00 2024-04-15T15:19:02.453 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.448+0000 7f6d1affd640 1 -- 172.21.15.154:0/2479319227 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f6d20048850 con 0x7f6d2815ca00 2024-04-15T15:19:02.453 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.449+0000 7f6d30677640 1 -- 172.21.15.154:0/2479319227 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d28069150 con 0x7f6d2815ca00 2024-04-15T15:19:02.454 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.451+0000 7f6d1affd640 1 -- 172.21.15.154:0/2479319227 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f6d2004d030 con 0x7f6d2815ca00 2024-04-15T15:19:02.526 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.525+0000 7f6d30677640 1 -- 172.21.15.154:0/2479319227 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd new", "uuid": "8900966a-b393-41e3-8f93-e5a43bd23622", "id": 1} v 0) -- 0x7f6d280aefb0 con 0x7f6d2815ca00 2024-04-15T15:19:02.529 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.528+0000 7f6d1affd640 1 -- 172.21.15.154:0/2479319227 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8900966a-b393-41e3-8f93-e5a43bd23622", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f6d2002cc00 con 0x7f6d2815ca00 2024-04-15T15:19:02.529 INFO:teuthology.orchestra.run.smithi154.stdout:1 2024-04-15T15:19:02.531 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.529+0000 7f6d30677640 1 -- 172.21.15.154:0/2479319227 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d2815ca00 msgr2=0x7f6d280c8730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:02.531 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.529+0000 7f6d30677640 1 --2- 172.21.15.154:0/2479319227 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d2815ca00 0x7f6d280c8730 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6d20036040 tx=0x7f6d20004a40 comp rx=0 tx=0).stop 2024-04-15T15:19:02.531 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.529+0000 7f6d30677640 1 -- 172.21.15.154:0/2479319227 shutdown_connections 2024-04-15T15:19:02.531 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.529+0000 7f6d30677640 1 --2- 172.21.15.154:0/2479319227 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6d2806bda0 0x7f6d280c7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:02.531 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.529+0000 7f6d30677640 1 --2- 172.21.15.154:0/2479319227 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d2815ca00 0x7f6d280c8730 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:02.531 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.529+0000 7f6d30677640 1 --2- 172.21.15.154:0/2479319227 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d2806b9c0 0x7f6d2814b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:02.531 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.529+0000 7f6d30677640 1 -- 172.21.15.154:0/2479319227 >> 172.21.15.154:0/2479319227 conn(0x7f6d2809aa40 msgr2=0x7f6d2815b330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:02.531 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.529+0000 7f6d30677640 1 -- 172.21.15.154:0/2479319227 shutdown_connections 2024-04-15T15:19:02.531 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.529+0000 7f6d30677640 1 -- 172.21.15.154:0/2479319227 wait complete. 2024-04-15T15:19:02.542 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd new 81a5dc7a-9325-4ff5-941d-a2d5fba21f1d 2 2024-04-15T15:19:02.618 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-04-15T15:19:02.617+0000 7ff178e16040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-15T15:19:02.630 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-04-15T15:19:02.628+0000 7f64fa816040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-15T15:19:02.646 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.644+0000 7f7591b3f640 1 Processor -- start 2024-04-15T15:19:02.646 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.644+0000 7f7591b3f640 1 -- start start 2024-04-15T15:19:02.648 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.645+0000 7f7591b3f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f758c15b970 0x7f758c0cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:02.648 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.645+0000 7f7591b3f640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f758c0ce2a0 0x7f758c06f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:02.648 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.645+0000 7f7591b3f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f758c06fa80 0x7f758c0cf450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:02.648 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.645+0000 7f7591b3f640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f758c0b3ac0 con 0x7f758c15b970 2024-04-15T15:19:02.648 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.645+0000 7f7591b3f640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f758c0cbb20 con 0x7f758c0ce2a0 2024-04-15T15:19:02.648 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.645+0000 7f7591b3f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f758c0cbe00 con 0x7f758c06fa80 2024-04-15T15:19:02.648 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.645+0000 7f7582ffd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f758c0ce2a0 0x7f758c06f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:02.648 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.645+0000 7f758bfff640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f758c06fa80 0x7f758c0cf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:02.648 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.645+0000 7f758bfff640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f758c06fa80 0x7f758c0cf450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.154:37858/0 (socket says 172.21.15.154:37858) 2024-04-15T15:19:02.649 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.645+0000 7f7582ffd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f758c0ce2a0 0x7f758c06f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50798/0 (socket says 172.21.15.154:50798) 2024-04-15T15:19:02.649 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.645+0000 7f758bfff640 1 -- 172.21.15.154:0/776373231 learned_addr learned my addr 172.21.15.154:0/776373231 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-04-15T15:19:02.649 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.645+0000 7f758b7fe640 1 --2- 172.21.15.154:0/776373231 >> v2:172.21.15.105:3300/0 conn(0x7f758c15b970 0x7f758c0cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:02.649 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.646+0000 7f758bfff640 1 -- 172.21.15.154:0/776373231 >> v2:172.21.15.154:3300/0 conn(0x7f758c0ce2a0 msgr2=0x7f758c06f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:02.649 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.646+0000 7f758bfff640 1 --2- 172.21.15.154:0/776373231 >> v2:172.21.15.154:3300/0 conn(0x7f758c0ce2a0 0x7f758c06f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:02.649 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.646+0000 7f758bfff640 1 -- 172.21.15.154:0/776373231 >> v2:172.21.15.105:3300/0 conn(0x7f758c15b970 msgr2=0x7f758c0cdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:02.649 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.646+0000 7f758bfff640 1 --2- 172.21.15.154:0/776373231 >> v2:172.21.15.105:3300/0 conn(0x7f758c15b970 0x7f758c0cdd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:02.649 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.646+0000 7f758bfff640 1 -- 172.21.15.154:0/776373231 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f758c0cc0e0 con 0x7f758c06fa80 2024-04-15T15:19:02.649 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.646+0000 7f758bfff640 1 --2- 172.21.15.154:0/776373231 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f758c06fa80 0x7f758c0cf450 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f758c073440 tx=0x7f758402be40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ea8f50f923fd0cb server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:02.649 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.646+0000 7f758affd640 1 -- 172.21.15.154:0/776373231 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758402c9e0 con 0x7f758c06fa80 2024-04-15T15:19:02.649 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.646+0000 7f758affd640 1 -- 172.21.15.154:0/776373231 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f758402cb80 con 0x7f758c06fa80 2024-04-15T15:19:02.649 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.646+0000 7f758affd640 1 -- 172.21.15.154:0/776373231 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75840343f0 con 0x7f758c06fa80 2024-04-15T15:19:02.653 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.647+0000 7f7591b3f640 1 -- 172.21.15.154:0/776373231 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f758c06fa80 msgr2=0x7f758c0cf450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:02.653 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.647+0000 7f7591b3f640 1 --2- 172.21.15.154:0/776373231 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f758c06fa80 0x7f758c0cf450 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f758c073440 tx=0x7f758402be40 comp rx=0 tx=0).stop 2024-04-15T15:19:02.653 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.647+0000 7f7591b3f640 1 -- 172.21.15.154:0/776373231 shutdown_connections 2024-04-15T15:19:02.653 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.647+0000 7f7591b3f640 1 --2- 172.21.15.154:0/776373231 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f758c06fa80 0x7f758c0cf450 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:02.653 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.647+0000 7f7591b3f640 1 --2- 172.21.15.154:0/776373231 >> v2:172.21.15.154:3300/0 conn(0x7f758c0ce2a0 0x7f758c06f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:02.653 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.647+0000 7f7591b3f640 1 --2- 172.21.15.154:0/776373231 >> v2:172.21.15.105:3300/0 conn(0x7f758c15b970 0x7f758c0cdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:02.653 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.647+0000 7f7591b3f640 1 -- 172.21.15.154:0/776373231 >> 172.21.15.154:0/776373231 conn(0x7f758c09aa40 msgr2=0x7f758c0b3060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:02.653 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.647+0000 7f7591b3f640 1 -- 172.21.15.154:0/776373231 shutdown_connections 2024-04-15T15:19:02.653 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.647+0000 7f7591b3f640 1 -- 172.21.15.154:0/776373231 wait complete. 2024-04-15T15:19:02.653 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.648+0000 7f7591b3f640 1 Processor -- start 2024-04-15T15:19:02.653 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.649+0000 7f7591b3f640 1 -- start start 2024-04-15T15:19:02.653 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.649+0000 7f7591b3f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f758c06fa80 0x7f758c06bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:02.653 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.649+0000 7f7591b3f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f758c0ce2a0 0x7f758c0c9d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:02.653 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.649+0000 7f7591b3f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f758c15b970 0x7f758c1f7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:02.654 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.649+0000 7f7591b3f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f758c0702a0 con 0x7f758c0ce2a0 2024-04-15T15:19:02.654 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.649+0000 7f7591b3f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f758c070120 con 0x7f758c15b970 2024-04-15T15:19:02.654 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.649+0000 7f7591b3f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f758c0cf990 con 0x7f758c06fa80 2024-04-15T15:19:02.654 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.649+0000 7f758b7fe640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f758c06fa80 0x7f758c06bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:02.654 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.649+0000 7f758bfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f758c15b970 0x7f758c1f7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:02.654 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.649+0000 7f758b7fe640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f758c06fa80 0x7f758c06bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.154:37864/0 (socket says 172.21.15.154:37864) 2024-04-15T15:19:02.654 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.649+0000 7f758bfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f758c15b970 0x7f758c1f7620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50800/0 (socket says 172.21.15.154:50800) 2024-04-15T15:19:02.654 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.649+0000 7f758b7fe640 1 -- 172.21.15.154:0/3811524601 learned_addr learned my addr 172.21.15.154:0/3811524601 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-04-15T15:19:02.654 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.650+0000 7f7582ffd640 1 --2- 172.21.15.154:0/3811524601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f758c0ce2a0 0x7f758c0c9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:02.654 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.650+0000 7f758b7fe640 1 -- 172.21.15.154:0/3811524601 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f758c15b970 msgr2=0x7f758c1f7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:02.654 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.650+0000 7f758b7fe640 1 --2- 172.21.15.154:0/3811524601 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f758c15b970 0x7f758c1f7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:02.654 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.650+0000 7f758b7fe640 1 -- 172.21.15.154:0/3811524601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f758c0ce2a0 msgr2=0x7f758c0c9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:02.654 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.650+0000 7f758b7fe640 1 --2- 172.21.15.154:0/3811524601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f758c0ce2a0 0x7f758c0c9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:02.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.650+0000 7f758b7fe640 1 -- 172.21.15.154:0/3811524601 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f758c0ca680 con 0x7f758c06fa80 2024-04-15T15:19:02.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.650+0000 7f7582ffd640 1 --2- 172.21.15.154:0/3811524601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f758c0ce2a0 0x7f758c0c9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:19:02.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.650+0000 7f758bfff640 1 --2- 172.21.15.154:0/3811524601 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f758c15b970 0x7f758c1f7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:19:02.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.650+0000 7f758b7fe640 1 --2- 172.21.15.154:0/3811524601 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f758c06fa80 0x7f758c06bc30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f757c00e8a0 tx=0x7f757c00ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:02.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.650+0000 7f7588ff9640 1 -- 172.21.15.154:0/3811524601 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757c017070 con 0x7f758c06fa80 2024-04-15T15:19:02.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.650+0000 7f7588ff9640 1 -- 172.21.15.154:0/3811524601 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f757c00c940 con 0x7f758c06fa80 2024-04-15T15:19:02.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.650+0000 7f7591b3f640 1 -- 172.21.15.154:0/3811524601 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f758c0ca960 con 0x7f758c06fa80 2024-04-15T15:19:02.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.650+0000 7f7588ff9640 1 -- 172.21.15.154:0/3811524601 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757c00cd30 con 0x7f758c06fa80 2024-04-15T15:19:02.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.650+0000 7f7591b3f640 1 -- 172.21.15.154:0/3811524601 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f758c0caf20 con 0x7f758c06fa80 2024-04-15T15:19:02.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.650+0000 7f7588ff9640 1 -- 172.21.15.154:0/3811524601 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f757c01a070 con 0x7f758c06fa80 2024-04-15T15:19:02.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.650+0000 7f7588ff9640 1 -- 172.21.15.154:0/3811524601 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f757c01c870 con 0x7f758c06fa80 2024-04-15T15:19:02.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.651+0000 7f7591b3f640 1 -- 172.21.15.154:0/3811524601 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f758c069150 con 0x7f758c06fa80 2024-04-15T15:19:02.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.655+0000 7f7588ff9640 1 -- 172.21.15.154:0/3811524601 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f757c01c450 con 0x7f758c06fa80 2024-04-15T15:19:02.672 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-04-15T15:19:02.671+0000 7ff178e16040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-15T15:19:02.688 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-04-15T15:19:02.687+0000 7f64fa816040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-15T15:19:02.717 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-04-15T15:19:02.715+0000 7ff178e16040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-15T15:19:02.730 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.728+0000 7f7591b3f640 1 -- 172.21.15.154:0/3811524601 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd new", "uuid": "81a5dc7a-9325-4ff5-941d-a2d5fba21f1d", "id": 2} v 0) -- 0x7f758c0b32e0 con 0x7f758c06fa80 2024-04-15T15:19:02.735 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.734+0000 7f7588ff9640 1 -- 172.21.15.154:0/3811524601 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "81a5dc7a-9325-4ff5-941d-a2d5fba21f1d", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f757c01cb10 con 0x7f758c06fa80 2024-04-15T15:19:02.736 INFO:teuthology.orchestra.run.smithi154.stdout:2 2024-04-15T15:19:02.736 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-04-15T15:19:02.734+0000 7f64fa816040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-15T15:19:02.737 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.735+0000 7f7591b3f640 1 -- 172.21.15.154:0/3811524601 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f758c06fa80 msgr2=0x7f758c06bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:02.737 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.735+0000 7f7591b3f640 1 --2- 172.21.15.154:0/3811524601 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f758c06fa80 0x7f758c06bc30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f757c00e8a0 tx=0x7f757c00ed60 comp rx=0 tx=0).stop 2024-04-15T15:19:02.737 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.735+0000 7f7591b3f640 1 -- 172.21.15.154:0/3811524601 shutdown_connections 2024-04-15T15:19:02.737 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.735+0000 7f7591b3f640 1 --2- 172.21.15.154:0/3811524601 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f758c15b970 0x7f758c1f7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:02.737 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.735+0000 7f7591b3f640 1 --2- 172.21.15.154:0/3811524601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f758c0ce2a0 0x7f758c0c9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:02.737 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.735+0000 7f7591b3f640 1 --2- 172.21.15.154:0/3811524601 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f758c06fa80 0x7f758c06bc30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:02.737 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.735+0000 7f7591b3f640 1 -- 172.21.15.154:0/3811524601 >> 172.21.15.154:0/3811524601 conn(0x7f758c09aa40 msgr2=0x7f758c0c7c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:02.737 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.735+0000 7f7591b3f640 1 -- 172.21.15.154:0/3811524601 shutdown_connections 2024-04-15T15:19:02.737 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.735+0000 7f7591b3f640 1 -- 172.21.15.154:0/3811524601 wait complete. 2024-04-15T15:19:02.747 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd new df766ed2-5c71-4805-96f4-3d4970edc960 3 2024-04-15T15:19:02.771 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-04-15T15:19:02.770+0000 7ff178e16040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-15T15:19:02.794 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-04-15T15:19:02.792+0000 7f64fa816040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-15T15:19:02.813 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-04-15T15:19:02.811+0000 7ff178e16040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-15T15:19:02.838 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-04-15T15:19:02.837+0000 7f64fa816040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-15T15:19:02.850 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.849+0000 7f5583de2640 1 Processor -- start 2024-04-15T15:19:02.851 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.849+0000 7f5583de2640 1 -- start start 2024-04-15T15:19:02.852 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.850+0000 7f5583de2640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f557c0cd320 0x7f557c0af780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:02.852 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.850+0000 7f5583de2640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f557c06f4f0 con 0x7f557c155b30 2024-04-15T15:19:02.852 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.850+0000 7f5583de2640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f557c0ae760 con 0x7f557c0afcc0 2024-04-15T15:19:02.852 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.850+0000 7f5583de2640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f557c0b0100 con 0x7f557c0cd320 2024-04-15T15:19:02.852 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.850+0000 7f558137d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f557c0cd320 0x7f557c0af780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:02.852 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.850+0000 7f558237f640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f557c0afcc0 0x7f557c06fc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:45310/0 (socket says 172.21.15.154:45310) 2024-04-15T15:19:02.852 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.850+0000 7f558137d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f557c0cd320 0x7f557c0af780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.154:37876/0 (socket says 172.21.15.154:37876) 2024-04-15T15:19:02.853 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.850+0000 7f558237f640 1 -- 172.21.15.154:0/3626287387 learned_addr learned my addr 172.21.15.154:0/3626287387 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-04-15T15:19:02.853 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.850+0000 7f558137d640 1 -- 172.21.15.154:0/3626287387 >> v1:172.21.15.154:6789/0 conn(0x7f557c0afcc0 legacy=0x7f557c06fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:02.853 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.850+0000 7f558137d640 1 -- 172.21.15.154:0/3626287387 >> v1:172.21.15.105:6789/0 conn(0x7f557c155b30 legacy=0x7f557c0addf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:02.853 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.850+0000 7f558137d640 1 -- 172.21.15.154:0/3626287387 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f557c0aea40 con 0x7f557c0cd320 2024-04-15T15:19:02.854 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.851+0000 7f558137d640 1 --2- 172.21.15.154:0/3626287387 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f557c0cd320 0x7f557c0af780 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f55700097d0 tx=0x7f557002b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d93e03a507cdec05 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:02.854 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.851+0000 7f5580b7c640 1 -- 172.21.15.154:0/3626287387 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5570035020 con 0x7f557c0cd320 2024-04-15T15:19:02.854 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.851+0000 7f5580b7c640 1 -- 172.21.15.154:0/3626287387 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5570031890 con 0x7f557c0cd320 2024-04-15T15:19:02.854 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.851+0000 7f5580b7c640 1 -- 172.21.15.154:0/3626287387 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5570031c60 con 0x7f557c0cd320 2024-04-15T15:19:02.855 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.852+0000 7f5583de2640 1 -- 172.21.15.154:0/3626287387 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f557c0cd320 msgr2=0x7f557c0af780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:02.855 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.852+0000 7f5583de2640 1 --2- 172.21.15.154:0/3626287387 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f557c0cd320 0x7f557c0af780 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f55700097d0 tx=0x7f557002b930 comp rx=0 tx=0).stop 2024-04-15T15:19:02.855 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.852+0000 7f5583de2640 1 -- 172.21.15.154:0/3626287387 shutdown_connections 2024-04-15T15:19:02.855 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.852+0000 7f5583de2640 1 --2- 172.21.15.154:0/3626287387 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f557c0cd320 0x7f557c0af780 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:02.855 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.852+0000 7f5583de2640 1 -- 172.21.15.154:0/3626287387 >> 172.21.15.154:0/3626287387 conn(0x7f557c073680 msgr2=0x7f557c06b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:02.855 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.852+0000 7f5583de2640 1 -- 172.21.15.154:0/3626287387 shutdown_connections 2024-04-15T15:19:02.855 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.852+0000 7f5583de2640 1 -- 172.21.15.154:0/3626287387 wait complete. 2024-04-15T15:19:02.858 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.853+0000 7f5583de2640 1 Processor -- start 2024-04-15T15:19:02.858 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.853+0000 7f5583de2640 1 -- start start 2024-04-15T15:19:02.858 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.854+0000 7f5583de2640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f557c0afcc0 0x7f557c1585c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:02.858 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.854+0000 7f5583de2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f557c0cd320 0x7f557c14f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:02.858 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.854+0000 7f5583de2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f557c14faf0 0x7f557c1e9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:02.858 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.854+0000 7f5583de2640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f557c06f770 con 0x7f557c14faf0 2024-04-15T15:19:02.858 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.854+0000 7f5583de2640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f557c0ae500 con 0x7f557c0cd320 2024-04-15T15:19:02.858 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.854+0000 7f5583de2640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f557c0703a0 con 0x7f557c0afcc0 2024-04-15T15:19:02.858 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.854+0000 7f5581b7e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f557c0afcc0 0x7f557c1585c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:02.858 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.854+0000 7f558137d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f557c0cd320 0x7f557c14f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:02.858 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.854+0000 7f5581b7e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f557c0afcc0 0x7f557c1585c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.154:37884/0 (socket says 172.21.15.154:37884) 2024-04-15T15:19:02.858 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.854+0000 7f5581b7e640 1 -- 172.21.15.154:0/1804554848 learned_addr learned my addr 172.21.15.154:0/1804554848 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-04-15T15:19:02.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.854+0000 7f558237f640 1 --2- 172.21.15.154:0/1804554848 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f557c14faf0 0x7f557c1e9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:02.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.855+0000 7f558137d640 1 -- 172.21.15.154:0/1804554848 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f557c0afcc0 msgr2=0x7f557c1585c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:02.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.855+0000 7f558137d640 1 --2- 172.21.15.154:0/1804554848 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f557c0afcc0 0x7f557c1585c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:02.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.855+0000 7f558137d640 1 -- 172.21.15.154:0/1804554848 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f557c14faf0 msgr2=0x7f557c1e9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:02.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.855+0000 7f558137d640 1 --2- 172.21.15.154:0/1804554848 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f557c14faf0 0x7f557c1e9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:02.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.855+0000 7f558137d640 1 -- 172.21.15.154:0/1804554848 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f557c14ff80 con 0x7f557c0cd320 2024-04-15T15:19:02.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.855+0000 7f5581b7e640 1 --2- 172.21.15.154:0/1804554848 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f557c0afcc0 0x7f557c1585c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:19:02.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.855+0000 7f558137d640 1 --2- 172.21.15.154:0/1804554848 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f557c0cd320 0x7f557c14f5b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5570033000 tx=0x7f55700046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:02.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.855+0000 7f556e7fc640 1 -- 172.21.15.154:0/1804554848 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5570035040 con 0x7f557c0cd320 2024-04-15T15:19:02.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.855+0000 7f556e7fc640 1 -- 172.21.15.154:0/1804554848 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5570046480 con 0x7f557c0cd320 2024-04-15T15:19:02.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.855+0000 7f5583de2640 1 -- 172.21.15.154:0/1804554848 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f557c0cff40 con 0x7f557c0cd320 2024-04-15T15:19:02.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.855+0000 7f556e7fc640 1 -- 172.21.15.154:0/1804554848 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5570046870 con 0x7f557c0cd320 2024-04-15T15:19:02.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.855+0000 7f5583de2640 1 -- 172.21.15.154:0/1804554848 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f557c0d0500 con 0x7f557c0cd320 2024-04-15T15:19:02.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.855+0000 7f556e7fc640 1 -- 172.21.15.154:0/1804554848 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5570038020 con 0x7f557c0cd320 2024-04-15T15:19:02.860 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.855+0000 7f556e7fc640 1 -- 172.21.15.154:0/1804554848 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f557004f3f0 con 0x7f557c0cd320 2024-04-15T15:19:02.860 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.856+0000 7f5583de2640 1 -- 172.21.15.154:0/1804554848 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f557c069150 con 0x7f557c0cd320 2024-04-15T15:19:02.860 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.858+0000 7f556e7fc640 1 -- 172.21.15.154:0/1804554848 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f557002d040 con 0x7f557c0cd320 2024-04-15T15:19:02.930 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-04-15T15:19:02.928+0000 7ff178e16040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-15T15:19:02.933 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.931+0000 7f5583de2640 1 -- 172.21.15.154:0/1804554848 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd new", "uuid": "df766ed2-5c71-4805-96f4-3d4970edc960", "id": 3} v 0) -- 0x7f557c1e92f0 con 0x7f557c0cd320 2024-04-15T15:19:02.938 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.937+0000 7f556e7fc640 1 -- 172.21.15.154:0/1804554848 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "df766ed2-5c71-4805-96f4-3d4970edc960", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f5570031820 con 0x7f557c0cd320 2024-04-15T15:19:02.938 INFO:teuthology.orchestra.run.smithi154.stdout:3 2024-04-15T15:19:02.940 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.937+0000 7f5583de2640 1 -- 172.21.15.154:0/1804554848 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f557c0cd320 msgr2=0x7f557c14f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:02.940 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.937+0000 7f5583de2640 1 --2- 172.21.15.154:0/1804554848 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f557c0cd320 0x7f557c14f5b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5570033000 tx=0x7f55700046c0 comp rx=0 tx=0).stop 2024-04-15T15:19:02.940 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.938+0000 7f5583de2640 1 -- 172.21.15.154:0/1804554848 shutdown_connections 2024-04-15T15:19:02.940 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.938+0000 7f5583de2640 1 --2- 172.21.15.154:0/1804554848 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f557c14faf0 0x7f557c1e9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:02.940 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.938+0000 7f5583de2640 1 --2- 172.21.15.154:0/1804554848 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f557c0cd320 0x7f557c14f5b0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:02.940 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.938+0000 7f5583de2640 1 --2- 172.21.15.154:0/1804554848 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f557c0afcc0 0x7f557c1585c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:02.940 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.938+0000 7f5583de2640 1 -- 172.21.15.154:0/1804554848 >> 172.21.15.154:0/1804554848 conn(0x7f557c073680 msgr2=0x7f557c0b0910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:02.940 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.938+0000 7f5583de2640 1 -- 172.21.15.154:0/1804554848 shutdown_connections 2024-04-15T15:19:02.940 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:02.938+0000 7f5583de2640 1 -- 172.21.15.154:0/1804554848 wait complete. 2024-04-15T15:19:02.950 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd new e1039e03-28b8-49df-b951-c91930b93c9a 4 2024-04-15T15:19:02.962 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-04-15T15:19:02.961+0000 7f64fa816040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-15T15:19:03.000 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-04-15T15:19:02.998+0000 7ff178e16040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-15T15:19:03.036 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-04-15T15:19:03.035+0000 7f64fa816040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-15T15:19:03.046 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-04-15T15:19:03.044+0000 7ff178e16040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-15T15:19:03.054 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.052+0000 7f1837641640 1 Processor -- start 2024-04-15T15:19:03.054 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.052+0000 7f1837641640 1 -- start start 2024-04-15T15:19:03.056 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.053+0000 7f1837641640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f183015b970 0x7f18300cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:03.057 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.053+0000 7f1837641640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f18300ce2a0 0x7f183006f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:03.057 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.053+0000 7f1837641640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18300cbb20 con 0x7f183006fa80 2024-04-15T15:19:03.057 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.053+0000 7f1837641640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f18300b3ac0 con 0x7f18300ce2a0 2024-04-15T15:19:03.057 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.053+0000 7f1837641640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f18300cd8f0 con 0x7f183015b970 2024-04-15T15:19:03.057 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.053+0000 7f18353dd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f183015b970 0x7f18300cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:03.057 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.053+0000 7f18353dd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f183015b970 0x7f18300cdd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.154:37894/0 (socket says 172.21.15.154:37894) 2024-04-15T15:19:03.057 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.053+0000 7f18353dd640 1 -- 172.21.15.154:0/3156644521 learned_addr learned my addr 172.21.15.154:0/3156644521 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-04-15T15:19:03.057 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.054+0000 7f1834bdc640 1 --2- 172.21.15.154:0/3156644521 >> v2:172.21.15.105:3300/0 conn(0x7f18300ce2a0 0x7f183006f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:03.057 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.054+0000 7f18353dd640 1 -- 172.21.15.154:0/3156644521 >> v2:172.21.15.105:3300/0 conn(0x7f18300ce2a0 msgr2=0x7f183006f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:03.057 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.054+0000 7f18353dd640 1 --2- 172.21.15.154:0/3156644521 >> v2:172.21.15.105:3300/0 conn(0x7f18300ce2a0 0x7f183006f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.057 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.054+0000 7f18353dd640 1 -- 172.21.15.154:0/3156644521 >> v1:172.21.15.105:6789/0 conn(0x7f183006fa80 legacy=0x7f18300cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:03.058 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.054+0000 7f18353dd640 1 -- 172.21.15.154:0/3156644521 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18300cbe00 con 0x7f183015b970 2024-04-15T15:19:03.058 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.054+0000 7f1834bdc640 1 --2- 172.21.15.154:0/3156644521 >> v2:172.21.15.105:3300/0 conn(0x7f18300ce2a0 0x7f183006f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:19:03.058 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.054+0000 7f18353dd640 1 --2- 172.21.15.154:0/3156644521 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f183015b970 0x7f18300cdd60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1824009870 tx=0x7f182402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9ce603bb0ea05a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:03.058 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.055+0000 7f1823fff640 1 -- 172.21.15.154:0/3156644521 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1824035020 con 0x7f183015b970 2024-04-15T15:19:03.058 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.055+0000 7f1823fff640 1 -- 172.21.15.154:0/3156644521 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1824031640 con 0x7f183015b970 2024-04-15T15:19:03.058 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.055+0000 7f1823fff640 1 -- 172.21.15.154:0/3156644521 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f182402d040 con 0x7f183015b970 2024-04-15T15:19:03.060 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.055+0000 7f1837641640 1 -- 172.21.15.154:0/3156644521 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f183015b970 msgr2=0x7f18300cdd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:03.061 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.055+0000 7f1837641640 1 --2- 172.21.15.154:0/3156644521 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f183015b970 0x7f18300cdd60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1824009870 tx=0x7f182402b8f0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.061 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.056+0000 7f1837641640 1 -- 172.21.15.154:0/3156644521 shutdown_connections 2024-04-15T15:19:03.061 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.056+0000 7f1837641640 1 --2- 172.21.15.154:0/3156644521 >> v2:172.21.15.105:3300/0 conn(0x7f18300ce2a0 0x7f183006f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.061 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.056+0000 7f1837641640 1 --2- 172.21.15.154:0/3156644521 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f183015b970 0x7f18300cdd60 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.061 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.056+0000 7f1837641640 1 -- 172.21.15.154:0/3156644521 >> 172.21.15.154:0/3156644521 conn(0x7f183009aa40 msgr2=0x7f18300b3060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:03.061 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.056+0000 7f1837641640 1 -- 172.21.15.154:0/3156644521 shutdown_connections 2024-04-15T15:19:03.061 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.056+0000 7f1837641640 1 -- 172.21.15.154:0/3156644521 wait complete. 2024-04-15T15:19:03.061 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.057+0000 7f1837641640 1 Processor -- start 2024-04-15T15:19:03.061 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.057+0000 7f1837641640 1 -- start start 2024-04-15T15:19:03.061 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.058+0000 7f1837641640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f183006fa80 0x7f183014a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:03.061 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.058+0000 7f1837641640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18300ce2a0 0x7f18301f4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:03.061 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.058+0000 7f1837641640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f183015b970 0x7f183015e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:03.061 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.058+0000 7f1837641640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f18300c62d0 con 0x7f183006fa80 2024-04-15T15:19:03.061 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.058+0000 7f1837641640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1830070120 con 0x7f18300ce2a0 2024-04-15T15:19:03.062 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.058+0000 7f1837641640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f183015f0a0 con 0x7f183015b970 2024-04-15T15:19:03.062 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.058+0000 7f1835bde640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f183015b970 0x7f183015e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:03.062 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.058+0000 7f18353dd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f183006fa80 0x7f183014a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:03.062 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.058+0000 7f1835bde640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f183015b970 0x7f183015e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.154:37900/0 (socket says 172.21.15.154:37900) 2024-04-15T15:19:03.062 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.058+0000 7f1835bde640 1 -- 172.21.15.154:0/4037045307 learned_addr learned my addr 172.21.15.154:0/4037045307 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-04-15T15:19:03.062 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.058+0000 7f1834bdc640 1 --2- 172.21.15.154:0/4037045307 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18300ce2a0 0x7f18301f4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:03.062 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.058+0000 7f1835bde640 1 -- 172.21.15.154:0/4037045307 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18300ce2a0 msgr2=0x7f18301f4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:03.062 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.058+0000 7f1835bde640 1 --2- 172.21.15.154:0/4037045307 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18300ce2a0 0x7f18301f4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.062 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.058+0000 7f1835bde640 1 -- 172.21.15.154:0/4037045307 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f183006fa80 msgr2=0x7f183014a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:03.062 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.058+0000 7f1835bde640 1 --2- 172.21.15.154:0/4037045307 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f183006fa80 0x7f183014a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.062 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.058+0000 7f1835bde640 1 -- 172.21.15.154:0/4037045307 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18300c9680 con 0x7f183015b970 2024-04-15T15:19:03.062 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.058+0000 7f1835bde640 1 --2- 172.21.15.154:0/4037045307 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f183015b970 0x7f183015e250 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f182800c6b0 tx=0x7f182800cb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:03.062 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.058+0000 7f1821ffb640 1 -- 172.21.15.154:0/4037045307 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1828003b70 con 0x7f183015b970 2024-04-15T15:19:03.062 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.058+0000 7f1821ffb640 1 -- 172.21.15.154:0/4037045307 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1828003d30 con 0x7f183015b970 2024-04-15T15:19:03.062 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.058+0000 7f1837641640 1 -- 172.21.15.154:0/4037045307 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f183015edc0 con 0x7f183015b970 2024-04-15T15:19:03.063 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.058+0000 7f1821ffb640 1 -- 172.21.15.154:0/4037045307 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1828005310 con 0x7f183015b970 2024-04-15T15:19:03.063 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.059+0000 7f1821ffb640 1 -- 172.21.15.154:0/4037045307 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f18280054b0 con 0x7f183015b970 2024-04-15T15:19:03.063 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.059+0000 7f1837641640 1 -- 172.21.15.154:0/4037045307 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f18301f53e0 con 0x7f183015b970 2024-04-15T15:19:03.063 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.059+0000 7f1821ffb640 1 -- 172.21.15.154:0/4037045307 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f18280057a0 con 0x7f183015b970 2024-04-15T15:19:03.063 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.059+0000 7f1837641640 1 -- 172.21.15.154:0/4037045307 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1830069150 con 0x7f183015b970 2024-04-15T15:19:03.065 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.063+0000 7f1821ffb640 1 -- 172.21.15.154:0/4037045307 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f182800a640 con 0x7f183015b970 2024-04-15T15:19:03.087 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-04-15T15:19:03.085+0000 7f64fa816040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-15T15:19:03.098 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-04-15T15:19:03.097+0000 7ff178e16040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-15T15:19:03.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.135+0000 7f1837641640 1 -- 172.21.15.154:0/4037045307 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd new", "uuid": "e1039e03-28b8-49df-b951-c91930b93c9a", "id": 4} v 0) -- 0x7f18300c99d0 con 0x7f183015b970 2024-04-15T15:19:03.140 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-04-15T15:19:03.138+0000 7f64fa816040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-15T15:19:03.142 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.141+0000 7f1821ffb640 1 -- 172.21.15.154:0/4037045307 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e1039e03-28b8-49df-b951-c91930b93c9a", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f182800a8f0 con 0x7f183015b970 2024-04-15T15:19:03.142 INFO:teuthology.orchestra.run.smithi154.stdout:4 2024-04-15T15:19:03.143 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.141+0000 7f1837641640 1 -- 172.21.15.154:0/4037045307 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f183015b970 msgr2=0x7f183015e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:03.143 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.141+0000 7f1837641640 1 --2- 172.21.15.154:0/4037045307 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f183015b970 0x7f183015e250 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f182800c6b0 tx=0x7f182800cb70 comp rx=0 tx=0).stop 2024-04-15T15:19:03.144 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.142+0000 7f1837641640 1 -- 172.21.15.154:0/4037045307 shutdown_connections 2024-04-15T15:19:03.144 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.142+0000 7f1837641640 1 --2- 172.21.15.154:0/4037045307 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f183015b970 0x7f183015e250 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.144 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.142+0000 7f1837641640 1 --2- 172.21.15.154:0/4037045307 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18300ce2a0 0x7f18301f4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.144 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.142+0000 7f1837641640 1 --2- 172.21.15.154:0/4037045307 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f183006fa80 0x7f183014a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.144 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.142+0000 7f1837641640 1 -- 172.21.15.154:0/4037045307 >> 172.21.15.154:0/4037045307 conn(0x7f183009aa40 msgr2=0x7f18300c65e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:03.144 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.142+0000 7f1837641640 1 -- 172.21.15.154:0/4037045307 shutdown_connections 2024-04-15T15:19:03.144 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.142+0000 7f1837641640 1 -- 172.21.15.154:0/4037045307 wait complete. 2024-04-15T15:19:03.146 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-04-15T15:19:03.144+0000 7ff178e16040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-15T15:19:03.153 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd new a9f5eaf1-66ff-405a-a9e4-ec3340812a13 5 2024-04-15T15:19:03.190 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-04-15T15:19:03.188+0000 7f64fa816040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-15T15:19:03.293 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.292+0000 7f7f51c93640 1 Processor -- start 2024-04-15T15:19:03.294 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.292+0000 7f7f51c93640 1 -- start start 2024-04-15T15:19:03.295 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.293+0000 7f7f51c93640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7f4c15de50 0x7f7f4c0b3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:03.295 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.293+0000 7f7f51c93640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7f4c0b1dd0 0x7f7f4c09b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:03.295 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.293+0000 7f7f51c93640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f4c0d0e10 con 0x7f7f4c09aa40 2024-04-15T15:19:03.295 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.293+0000 7f7f51c93640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f7f4c0d0bc0 con 0x7f7f4c15de50 2024-04-15T15:19:03.295 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.293+0000 7f7f51c93640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f7f4c0b2210 con 0x7f7f4c0b1dd0 2024-04-15T15:19:03.295 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.293+0000 7f7f4affd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7f4c0b1dd0 0x7f7f4c09b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:03.295 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.293+0000 7f7f4affd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7f4c0b1dd0 0x7f7f4c09b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50832/0 (socket says 172.21.15.154:50832) 2024-04-15T15:19:03.295 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.293+0000 7f7f4affd640 1 -- 172.21.15.154:0/3574548424 learned_addr learned my addr 172.21.15.154:0/3574548424 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-04-15T15:19:03.295 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.293+0000 7f7f4b7fe640 1 --2- 172.21.15.154:0/3574548424 >> v2:172.21.15.105:3300/0 conn(0x7f7f4c15de50 0x7f7f4c0b3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:03.295 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.293+0000 7f7f4affd640 1 -- 172.21.15.154:0/3574548424 >> v2:172.21.15.105:3300/0 conn(0x7f7f4c15de50 msgr2=0x7f7f4c0b3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:03.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.293+0000 7f7f4affd640 1 --2- 172.21.15.154:0/3574548424 >> v2:172.21.15.105:3300/0 conn(0x7f7f4c15de50 0x7f7f4c0b3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.293+0000 7f7f4affd640 1 -- 172.21.15.154:0/3574548424 >> v1:172.21.15.105:6789/0 conn(0x7f7f4c09aa40 legacy=0x7f7f4c0cd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:03.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.293+0000 7f7f4affd640 1 -- 172.21.15.154:0/3574548424 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f4c0d10f0 con 0x7f7f4c0b1dd0 2024-04-15T15:19:03.297 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.294+0000 7f7f4affd640 1 --2- 172.21.15.154:0/3574548424 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f4c0b1dd0 0x7f7f4c09b650 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c0d31b0 tx=0x7f7f4002b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17c0450e40eabbb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:03.297 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.294+0000 7f7f4a7fc640 1 -- 172.21.15.154:0/3574548424 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f40034070 con 0x7f7f4c0b1dd0 2024-04-15T15:19:03.297 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.294+0000 7f7f4a7fc640 1 -- 172.21.15.154:0/3574548424 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f40030860 con 0x7f7f4c0b1dd0 2024-04-15T15:19:03.297 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.294+0000 7f7f4a7fc640 1 -- 172.21.15.154:0/3574548424 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f40030c30 con 0x7f7f4c0b1dd0 2024-04-15T15:19:03.297 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.295+0000 7f7f51c93640 1 -- 172.21.15.154:0/3574548424 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f4c0b1dd0 msgr2=0x7f7f4c09b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:03.297 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.295+0000 7f7f51c93640 1 --2- 172.21.15.154:0/3574548424 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f4c0b1dd0 0x7f7f4c09b650 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c0d31b0 tx=0x7f7f4002b9a0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.297 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.295+0000 7f7f51c93640 1 -- 172.21.15.154:0/3574548424 shutdown_connections 2024-04-15T15:19:03.297 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.295+0000 7f7f51c93640 1 --2- 172.21.15.154:0/3574548424 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f4c0b1dd0 0x7f7f4c09b650 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.297 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.295+0000 7f7f51c93640 1 --2- 172.21.15.154:0/3574548424 >> v2:172.21.15.105:3300/0 conn(0x7f7f4c15de50 0x7f7f4c0b3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.298 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.295+0000 7f7f51c93640 1 -- 172.21.15.154:0/3574548424 >> 172.21.15.154:0/3574548424 conn(0x7f7f4c0d25e0 msgr2=0x7f7f4c0d29f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:03.298 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.295+0000 7f7f51c93640 1 -- 172.21.15.154:0/3574548424 shutdown_connections 2024-04-15T15:19:03.298 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.295+0000 7f7f51c93640 1 -- 172.21.15.154:0/3574548424 wait complete. 2024-04-15T15:19:03.298 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.296+0000 7f7f51c93640 1 Processor -- start 2024-04-15T15:19:03.300 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.296+0000 7f7f51c93640 1 -- start start 2024-04-15T15:19:03.300 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.297+0000 7f7f51c93640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f4c09aa40 0x7f7f4c0b6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:03.300 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.297+0000 7f7f51c93640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f4c0b1dd0 0x7f7f4c0ce3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:03.300 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.297+0000 7f7f51c93640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f4c15de50 0x7f7f4c153990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:03.300 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.297+0000 7f7f51c93640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c0d13d0 con 0x7f7f4c09aa40 2024-04-15T15:19:03.300 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.297+0000 7f7f51c93640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c09bb90 con 0x7f7f4c0b1dd0 2024-04-15T15:19:03.300 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.297+0000 7f7f51c93640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f7f4c09aeb0 con 0x7f7f4c15de50 2024-04-15T15:19:03.300 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.297+0000 7f7f4affd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f4c0b1dd0 0x7f7f4c0ce3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:03.300 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.297+0000 7f7f4affd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f4c0b1dd0 0x7f7f4c0ce3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50844/0 (socket says 172.21.15.154:50844) 2024-04-15T15:19:03.300 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.297+0000 7f7f4affd640 1 -- 172.21.15.154:0/1844677220 learned_addr learned my addr 172.21.15.154:0/1844677220 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-04-15T15:19:03.300 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.297+0000 7f7f4bfff640 1 --2- 172.21.15.154:0/1844677220 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f4c15de50 0x7f7f4c153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:03.301 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.297+0000 7f7f4b7fe640 1 --2- 172.21.15.154:0/1844677220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f4c09aa40 0x7f7f4c0b6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:03.301 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.297+0000 7f7f4affd640 1 -- 172.21.15.154:0/1844677220 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f4c15de50 msgr2=0x7f7f4c153990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:03.301 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.297+0000 7f7f4affd640 1 --2- 172.21.15.154:0/1844677220 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f4c15de50 0x7f7f4c153990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.301 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.297+0000 7f7f4affd640 1 -- 172.21.15.154:0/1844677220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f4c09aa40 msgr2=0x7f7f4c0b6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:03.301 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.297+0000 7f7f4affd640 1 --2- 172.21.15.154:0/1844677220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f4c09aa40 0x7f7f4c0b6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.301 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.297+0000 7f7f4affd640 1 -- 172.21.15.154:0/1844677220 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f4c1f6400 con 0x7f7f4c0b1dd0 2024-04-15T15:19:03.301 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.297+0000 7f7f4affd640 1 --2- 172.21.15.154:0/1844677220 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f4c0b1dd0 0x7f7f4c0ce3c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7f4002b9e0 tx=0x7f7f40004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:03.301 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.297+0000 7f7f33fff640 1 -- 172.21.15.154:0/1844677220 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f40048070 con 0x7f7f4c0b1dd0 2024-04-15T15:19:03.301 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.297+0000 7f7f33fff640 1 -- 172.21.15.154:0/1844677220 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f40030660 con 0x7f7f4c0b1dd0 2024-04-15T15:19:03.301 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.297+0000 7f7f33fff640 1 -- 172.21.15.154:0/1844677220 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f40034040 con 0x7f7f4c0b1dd0 2024-04-15T15:19:03.301 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.297+0000 7f7f51c93640 1 -- 172.21.15.154:0/1844677220 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f4c1f66e0 con 0x7f7f4c0b1dd0 2024-04-15T15:19:03.301 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.298+0000 7f7f51c93640 1 -- 172.21.15.154:0/1844677220 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f4c1f6ca0 con 0x7f7f4c0b1dd0 2024-04-15T15:19:03.301 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.298+0000 7f7f33fff640 1 -- 172.21.15.154:0/1844677220 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7f4004b070 con 0x7f7f4c0b1dd0 2024-04-15T15:19:03.301 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.298+0000 7f7f33fff640 1 -- 172.21.15.154:0/1844677220 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f7f40043990 con 0x7f7f4c0b1dd0 2024-04-15T15:19:03.301 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.298+0000 7f7f51c93640 1 -- 172.21.15.154:0/1844677220 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f4c069150 con 0x7f7f4c0b1dd0 2024-04-15T15:19:03.304 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.302+0000 7f7f33fff640 1 -- 172.21.15.154:0/1844677220 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f7f40052020 con 0x7f7f4c0b1dd0 2024-04-15T15:19:03.376 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.375+0000 7f7f51c93640 1 -- 172.21.15.154:0/1844677220 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a9f5eaf1-66ff-405a-a9e4-ec3340812a13", "id": 5} v 0) -- 0x7f7f4c0d3200 con 0x7f7f4c0b1dd0 2024-04-15T15:19:03.382 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.381+0000 7f7f33fff640 1 -- 172.21.15.154:0/1844677220 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a9f5eaf1-66ff-405a-a9e4-ec3340812a13", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f7f40030810 con 0x7f7f4c0b1dd0 2024-04-15T15:19:03.382 INFO:teuthology.orchestra.run.smithi154.stdout:5 2024-04-15T15:19:03.383 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.381+0000 7f7f51c93640 1 -- 172.21.15.154:0/1844677220 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f4c0b1dd0 msgr2=0x7f7f4c0ce3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:03.383 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.381+0000 7f7f51c93640 1 --2- 172.21.15.154:0/1844677220 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f4c0b1dd0 0x7f7f4c0ce3c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7f4002b9e0 tx=0x7f7f40004730 comp rx=0 tx=0).stop 2024-04-15T15:19:03.384 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.381+0000 7f7f51c93640 1 -- 172.21.15.154:0/1844677220 shutdown_connections 2024-04-15T15:19:03.384 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.381+0000 7f7f51c93640 1 --2- 172.21.15.154:0/1844677220 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f4c15de50 0x7f7f4c153990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.384 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.381+0000 7f7f51c93640 1 --2- 172.21.15.154:0/1844677220 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f4c0b1dd0 0x7f7f4c0ce3c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.384 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.381+0000 7f7f51c93640 1 --2- 172.21.15.154:0/1844677220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f4c09aa40 0x7f7f4c0b6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.384 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.382+0000 7f7f51c93640 1 -- 172.21.15.154:0/1844677220 >> 172.21.15.154:0/1844677220 conn(0x7f7f4c0d25e0 msgr2=0x7f7f4c0710a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:03.384 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.382+0000 7f7f51c93640 1 -- 172.21.15.154:0/1844677220 shutdown_connections 2024-04-15T15:19:03.384 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.382+0000 7f7f51c93640 1 -- 172.21.15.154:0/1844677220 wait complete. 2024-04-15T15:19:03.394 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd new 267d02ea-675f-428f-aae8-e0375e4d4feb 6 2024-04-15T15:19:03.498 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.497+0000 7f6b14e35640 1 Processor -- start 2024-04-15T15:19:03.499 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.497+0000 7f6b14e35640 1 -- start start 2024-04-15T15:19:03.500 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.498+0000 7f6b14e35640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6b1015b970 0x7f6b100cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:03.500 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.498+0000 7f6b14e35640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6b100ce2a0 0x7f6b1006f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:03.500 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.498+0000 7f6b14e35640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b100cbb20 con 0x7f6b1006fa80 2024-04-15T15:19:03.500 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.498+0000 7f6b14e35640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f6b100b3ac0 con 0x7f6b1015b970 2024-04-15T15:19:03.500 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.498+0000 7f6b14e35640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f6b1006fef0 con 0x7f6b100ce2a0 2024-04-15T15:19:03.500 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.498+0000 7f6b07fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6b100ce2a0 0x7f6b1006f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:03.500 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.498+0000 7f6b07fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6b100ce2a0 0x7f6b1006f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50848/0 (socket says 172.21.15.154:50848) 2024-04-15T15:19:03.500 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.498+0000 7f6b07fff640 1 -- 172.21.15.154:0/3339575324 learned_addr learned my addr 172.21.15.154:0/3339575324 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-04-15T15:19:03.500 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.498+0000 7f6b0e59c640 1 --2- 172.21.15.154:0/3339575324 >> v2:172.21.15.105:3300/0 conn(0x7f6b1015b970 0x7f6b100cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:03.501 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.498+0000 7f6b07fff640 1 -- 172.21.15.154:0/3339575324 >> v2:172.21.15.105:3300/0 conn(0x7f6b1015b970 msgr2=0x7f6b100cdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:03.501 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.498+0000 7f6b07fff640 1 --2- 172.21.15.154:0/3339575324 >> v2:172.21.15.105:3300/0 conn(0x7f6b1015b970 0x7f6b100cdd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.501 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.498+0000 7f6b07fff640 1 -- 172.21.15.154:0/3339575324 >> v1:172.21.15.154:6789/0 conn(0x7f6b1006fa80 legacy=0x7f6b100cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:03.501 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.498+0000 7f6b07fff640 1 -- 172.21.15.154:0/3339575324 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b100cbe00 con 0x7f6b100ce2a0 2024-04-15T15:19:03.502 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.499+0000 7f6b07fff640 1 --2- 172.21.15.154:0/3339575324 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b100ce2a0 0x7f6b1006f540 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6b10073440 tx=0x7f6af802e0f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a75a9c533e3261d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:03.502 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.499+0000 7f6b0dd9b640 1 -- 172.21.15.154:0/3339575324 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af803b070 con 0x7f6b100ce2a0 2024-04-15T15:19:03.502 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.499+0000 7f6b0dd9b640 1 -- 172.21.15.154:0/3339575324 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6af8036440 con 0x7f6b100ce2a0 2024-04-15T15:19:03.502 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.499+0000 7f6b0dd9b640 1 -- 172.21.15.154:0/3339575324 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af8036810 con 0x7f6b100ce2a0 2024-04-15T15:19:03.502 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.500+0000 7f6b14e35640 1 -- 172.21.15.154:0/3339575324 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b100ce2a0 msgr2=0x7f6b1006f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:03.502 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.500+0000 7f6b14e35640 1 --2- 172.21.15.154:0/3339575324 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b100ce2a0 0x7f6b1006f540 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6b10073440 tx=0x7f6af802e0f0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.502 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.500+0000 7f6b14e35640 1 -- 172.21.15.154:0/3339575324 shutdown_connections 2024-04-15T15:19:03.502 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.500+0000 7f6b14e35640 1 --2- 172.21.15.154:0/3339575324 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b100ce2a0 0x7f6b1006f540 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.500+0000 7f6b14e35640 1 --2- 172.21.15.154:0/3339575324 >> v2:172.21.15.105:3300/0 conn(0x7f6b1015b970 0x7f6b100cdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.500+0000 7f6b14e35640 1 -- 172.21.15.154:0/3339575324 >> 172.21.15.154:0/3339575324 conn(0x7f6b1009aa40 msgr2=0x7f6b100b3060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:03.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.500+0000 7f6b14e35640 1 -- 172.21.15.154:0/3339575324 shutdown_connections 2024-04-15T15:19:03.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.500+0000 7f6b14e35640 1 -- 172.21.15.154:0/3339575324 wait complete. 2024-04-15T15:19:03.505 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.501+0000 7f6b14e35640 1 Processor -- start 2024-04-15T15:19:03.505 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.501+0000 7f6b14e35640 1 -- start start 2024-04-15T15:19:03.505 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.502+0000 7f6b14e35640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b1006fa80 0x7f6b100cad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:03.505 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.502+0000 7f6b14e35640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6b100ce2a0 0x7f6b1014e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:03.505 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.502+0000 7f6b14e35640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b1015b970 0x7f6b1015e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:03.505 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.502+0000 7f6b14e35640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6b100c62d0 con 0x7f6b1006fa80 2024-04-15T15:19:03.506 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.502+0000 7f6b14e35640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6b100cd8f0 con 0x7f6b1015b970 2024-04-15T15:19:03.506 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.502+0000 7f6b14e35640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f6b1015e100 con 0x7f6b100ce2a0 2024-04-15T15:19:03.506 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.502+0000 7f6b0ed9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b1015b970 0x7f6b1015e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:03.506 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.502+0000 7f6b07fff640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6b100ce2a0 0x7f6b1014e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:03.506 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.502+0000 7f6b0ed9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b1015b970 0x7f6b1015e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50862/0 (socket says 172.21.15.154:50862) 2024-04-15T15:19:03.506 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.502+0000 7f6b0ed9d640 1 -- 172.21.15.154:0/2902372796 learned_addr learned my addr 172.21.15.154:0/2902372796 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-04-15T15:19:03.506 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.502+0000 7f6b0e59c640 1 --2- 172.21.15.154:0/2902372796 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b1006fa80 0x7f6b100cad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:03.506 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.502+0000 7f6b0ed9d640 1 -- 172.21.15.154:0/2902372796 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6b100ce2a0 msgr2=0x7f6b1014e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:03.506 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.502+0000 7f6b0ed9d640 1 --2- 172.21.15.154:0/2902372796 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6b100ce2a0 0x7f6b1014e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.506 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.502+0000 7f6b0ed9d640 1 -- 172.21.15.154:0/2902372796 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b1006fa80 msgr2=0x7f6b100cad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:03.506 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.502+0000 7f6b0ed9d640 1 --2- 172.21.15.154:0/2902372796 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b1006fa80 0x7f6b100cad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.506 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.502+0000 7f6b0ed9d640 1 -- 172.21.15.154:0/2902372796 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b10154a50 con 0x7f6b1015b970 2024-04-15T15:19:03.506 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.502+0000 7f6b07fff640 1 --2- 172.21.15.154:0/2902372796 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6b100ce2a0 0x7f6b1014e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:19:03.506 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.502+0000 7f6b0ed9d640 1 --2- 172.21.15.154:0/2902372796 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b1015b970 0x7f6b1015e590 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6b0800c7a0 tx=0x7f6b0800cc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:03.507 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.502+0000 7f6b06ffd640 1 -- 172.21.15.154:0/2902372796 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b08003b70 con 0x7f6b1015b970 2024-04-15T15:19:03.507 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.503+0000 7f6b06ffd640 1 -- 172.21.15.154:0/2902372796 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b08003d30 con 0x7f6b1015b970 2024-04-15T15:19:03.507 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.503+0000 7f6b14e35640 1 -- 172.21.15.154:0/2902372796 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b1015d4f0 con 0x7f6b1015b970 2024-04-15T15:19:03.507 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.503+0000 7f6b06ffd640 1 -- 172.21.15.154:0/2902372796 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0800aa90 con 0x7f6b1015b970 2024-04-15T15:19:03.507 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.503+0000 7f6b14e35640 1 -- 172.21.15.154:0/2902372796 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b1015dab0 con 0x7f6b1015b970 2024-04-15T15:19:03.507 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.503+0000 7f6b06ffd640 1 -- 172.21.15.154:0/2902372796 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6b0800ac30 con 0x7f6b1015b970 2024-04-15T15:19:03.507 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.503+0000 7f6b06ffd640 1 -- 172.21.15.154:0/2902372796 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f6b08012bf0 con 0x7f6b1015b970 2024-04-15T15:19:03.507 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.503+0000 7f6b14e35640 1 -- 172.21.15.154:0/2902372796 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b10069150 con 0x7f6b1015b970 2024-04-15T15:19:03.509 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.507+0000 7f6b06ffd640 1 -- 172.21.15.154:0/2902372796 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f6b10154a50 con 0x7f6b1015b970 2024-04-15T15:19:03.520 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-04-15T15:19:03.519+0000 7ff178e16040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-15T15:19:03.584 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.582+0000 7f6b14e35640 1 -- 172.21.15.154:0/2902372796 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd new", "uuid": "267d02ea-675f-428f-aae8-e0375e4d4feb", "id": 6} v 0) -- 0x7f6b100c80b0 con 0x7f6b1015b970 2024-04-15T15:19:03.590 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.588+0000 7f6b06ffd640 1 -- 172.21.15.154:0/2902372796 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "267d02ea-675f-428f-aae8-e0375e4d4feb", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f6b08012600 con 0x7f6b1015b970 2024-04-15T15:19:03.590 INFO:teuthology.orchestra.run.smithi154.stdout:6 2024-04-15T15:19:03.591 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.589+0000 7f6b14e35640 1 -- 172.21.15.154:0/2902372796 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b1015b970 msgr2=0x7f6b1015e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:03.591 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.589+0000 7f6b14e35640 1 --2- 172.21.15.154:0/2902372796 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b1015b970 0x7f6b1015e590 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6b0800c7a0 tx=0x7f6b0800cc60 comp rx=0 tx=0).stop 2024-04-15T15:19:03.591 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.589+0000 7f6b14e35640 1 -- 172.21.15.154:0/2902372796 shutdown_connections 2024-04-15T15:19:03.591 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.589+0000 7f6b14e35640 1 --2- 172.21.15.154:0/2902372796 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6b100ce2a0 0x7f6b1014e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.591 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.589+0000 7f6b14e35640 1 --2- 172.21.15.154:0/2902372796 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b1015b970 0x7f6b1015e590 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.591 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.589+0000 7f6b14e35640 1 --2- 172.21.15.154:0/2902372796 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b1006fa80 0x7f6b100cad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.592 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.589+0000 7f6b14e35640 1 -- 172.21.15.154:0/2902372796 >> 172.21.15.154:0/2902372796 conn(0x7f6b1009aa40 msgr2=0x7f6b1006c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:03.592 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.589+0000 7f6b14e35640 1 -- 172.21.15.154:0/2902372796 shutdown_connections 2024-04-15T15:19:03.592 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.589+0000 7f6b14e35640 1 -- 172.21.15.154:0/2902372796 wait complete. 2024-04-15T15:19:03.593 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-04-15T15:19:03.592+0000 7f64fa816040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-15T15:19:03.601 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd new 759ac6cc-896a-423c-885f-9fe90d8dcb14 7 2024-04-15T15:19:03.655 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-04-15T15:19:03.654+0000 7ff178e16040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-15T15:19:03.705 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.703+0000 7fd0a5dba640 1 Processor -- start 2024-04-15T15:19:03.705 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.703+0000 7fd0a5dba640 1 -- start start 2024-04-15T15:19:03.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.704+0000 7fd0a5dba640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd0a00cf000 0x7fd0a00c7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:03.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.704+0000 7fd0a5dba640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0a00b0ad0 con 0x7fd0a015eac0 2024-04-15T15:19:03.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.704+0000 7fd0a5dba640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0a006bbd0 con 0x7fd0a00c7d90 2024-04-15T15:19:03.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.704+0000 7fd0a5dba640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fd0a015eea0 con 0x7fd0a00cf000 2024-04-15T15:19:03.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.704+0000 7fd096ffd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd0a00cf000 0x7fd0a00c7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:03.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.704+0000 7fd096ffd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd0a00cf000 0x7fd0a00c7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50866/0 (socket says 172.21.15.154:50866) 2024-04-15T15:19:03.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.704+0000 7fd096ffd640 1 -- 172.21.15.154:0/1090813134 learned_addr learned my addr 172.21.15.154:0/1090813134 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-04-15T15:19:03.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.705+0000 7fd096ffd640 1 -- 172.21.15.154:0/1090813134 >> v1:172.21.15.154:6789/0 conn(0x7fd0a00c7d90 legacy=0x7fd0a00c67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:03.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.705+0000 7fd096ffd640 1 -- 172.21.15.154:0/1090813134 >> v1:172.21.15.105:6789/0 conn(0x7fd0a015eac0 legacy=0x7fd0a00b3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:03.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.705+0000 7fd096ffd640 1 -- 172.21.15.154:0/1090813134 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a006beb0 con 0x7fd0a00cf000 2024-04-15T15:19:03.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.705+0000 7fd096ffd640 1 --2- 172.21.15.154:0/1090813134 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0a00cf000 0x7fd0a00c7850 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd088009870 tx=0x7fd08802ba20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77336862c3489822 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:03.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.705+0000 7fd09effd640 1 -- 172.21.15.154:0/1090813134 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08802f840 con 0x7fd0a00cf000 2024-04-15T15:19:03.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.706+0000 7fd09effd640 1 -- 172.21.15.154:0/1090813134 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd08802f9e0 con 0x7fd0a00cf000 2024-04-15T15:19:03.709 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.706+0000 7fd09effd640 1 -- 172.21.15.154:0/1090813134 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08802fdb0 con 0x7fd0a00cf000 2024-04-15T15:19:03.709 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.706+0000 7fd0a5dba640 1 -- 172.21.15.154:0/1090813134 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0a00cf000 msgr2=0x7fd0a00c7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:03.709 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.706+0000 7fd0a5dba640 1 --2- 172.21.15.154:0/1090813134 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0a00cf000 0x7fd0a00c7850 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd088009870 tx=0x7fd08802ba20 comp rx=0 tx=0).stop 2024-04-15T15:19:03.709 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.706+0000 7fd0a5dba640 1 -- 172.21.15.154:0/1090813134 shutdown_connections 2024-04-15T15:19:03.709 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.706+0000 7fd0a5dba640 1 --2- 172.21.15.154:0/1090813134 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0a00cf000 0x7fd0a00c7850 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.709 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.706+0000 7fd0a5dba640 1 -- 172.21.15.154:0/1090813134 >> 172.21.15.154:0/1090813134 conn(0x7fd0a009aa40 msgr2=0x7fd0a00b3060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:03.709 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.706+0000 7fd0a5dba640 1 -- 172.21.15.154:0/1090813134 shutdown_connections 2024-04-15T15:19:03.709 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.706+0000 7fd0a5dba640 1 -- 172.21.15.154:0/1090813134 wait complete. 2024-04-15T15:19:03.710 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.708+0000 7fd0a5dba640 1 Processor -- start 2024-04-15T15:19:03.710 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.708+0000 7fd0a5dba640 1 -- start start 2024-04-15T15:19:03.713 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.709+0000 7fd0a5dba640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0a00c7d90 0x7fd0a0150420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:03.713 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.709+0000 7fd0a5dba640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd0a00cf000 0x7fd0a01f45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:03.713 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.709+0000 7fd0a5dba640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0a015eac0 0x7fd0a01f4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:03.713 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.709+0000 7fd0a5dba640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a00c6360 con 0x7fd0a00c7d90 2024-04-15T15:19:03.713 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.709+0000 7fd0a5dba640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a00c81e0 con 0x7fd0a015eac0 2024-04-15T15:19:03.713 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.709+0000 7fd0a5dba640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fd0a006c470 con 0x7fd0a00cf000 2024-04-15T15:19:03.713 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.710+0000 7fd09ffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0a015eac0 0x7fd0a01f4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:03.713 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.710+0000 7fd09f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0a00c7d90 0x7fd0a0150420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:03.713 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.710+0000 7fd09ffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0a015eac0 0x7fd0a01f4b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50880/0 (socket says 172.21.15.154:50880) 2024-04-15T15:19:03.713 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.710+0000 7fd096ffd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd0a00cf000 0x7fd0a01f45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:03.713 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.710+0000 7fd09ffff640 1 -- 172.21.15.154:0/127632062 learned_addr learned my addr 172.21.15.154:0/127632062 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-04-15T15:19:03.713 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.710+0000 7fd09ffff640 1 -- 172.21.15.154:0/127632062 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd0a00cf000 msgr2=0x7fd0a01f45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:03.714 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.710+0000 7fd09ffff640 1 --2- 172.21.15.154:0/127632062 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd0a00cf000 0x7fd0a01f45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.714 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.710+0000 7fd09ffff640 1 -- 172.21.15.154:0/127632062 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0a00c7d90 msgr2=0x7fd0a0150420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:03.714 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.710+0000 7fd09ffff640 1 --2- 172.21.15.154:0/127632062 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0a00c7d90 0x7fd0a0150420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.714 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.710+0000 7fd09ffff640 1 -- 172.21.15.154:0/127632062 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a0158020 con 0x7fd0a015eac0 2024-04-15T15:19:03.714 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.710+0000 7fd09f7fe640 1 --2- 172.21.15.154:0/127632062 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0a00c7d90 0x7fd0a0150420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:19:03.714 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.710+0000 7fd09ffff640 1 --2- 172.21.15.154:0/127632062 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0a015eac0 0x7fd0a01f4b10 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd09800c6e0 tx=0x7fd09800cba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:03.714 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.710+0000 7fd09cff9640 1 -- 172.21.15.154:0/127632062 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd098003b70 con 0x7fd0a015eac0 2024-04-15T15:19:03.714 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.710+0000 7fd09cff9640 1 -- 172.21.15.154:0/127632062 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd098003d30 con 0x7fd0a015eac0 2024-04-15T15:19:03.714 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.710+0000 7fd0a5dba640 1 -- 172.21.15.154:0/127632062 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0a0158300 con 0x7fd0a015eac0 2024-04-15T15:19:03.714 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.710+0000 7fd09cff9640 1 -- 172.21.15.154:0/127632062 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd098005310 con 0x7fd0a015eac0 2024-04-15T15:19:03.714 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.710+0000 7fd0a5dba640 1 -- 172.21.15.154:0/127632062 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0a01588c0 con 0x7fd0a015eac0 2024-04-15T15:19:03.714 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.710+0000 7fd09cff9640 1 -- 172.21.15.154:0/127632062 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd0980054b0 con 0x7fd0a015eac0 2024-04-15T15:19:03.714 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.711+0000 7fd09cff9640 1 -- 172.21.15.154:0/127632062 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7fd09800ac40 con 0x7fd0a015eac0 2024-04-15T15:19:03.714 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.711+0000 7fd0a5dba640 1 -- 172.21.15.154:0/127632062 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0a0069150 con 0x7fd0a015eac0 2024-04-15T15:19:03.715 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.713+0000 7fd09cff9640 1 -- 172.21.15.154:0/127632062 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fd0980057a0 con 0x7fd0a015eac0 2024-04-15T15:19:03.738 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-04-15T15:19:03.736+0000 7f64fa816040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-15T15:19:03.775 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-04-15T15:19:03.773+0000 7ff178e16040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-15T15:19:03.787 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.786+0000 7fd0a5dba640 1 -- 172.21.15.154:0/127632062 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd new", "uuid": "759ac6cc-896a-423c-885f-9fe90d8dcb14", "id": 7} v 0) -- 0x7fd0a00ae650 con 0x7fd0a015eac0 2024-04-15T15:19:03.796 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.794+0000 7fd09cff9640 1 -- 172.21.15.154:0/127632062 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "759ac6cc-896a-423c-885f-9fe90d8dcb14", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7fd09800a5e0 con 0x7fd0a015eac0 2024-04-15T15:19:03.796 INFO:teuthology.orchestra.run.smithi154.stdout:7 2024-04-15T15:19:03.798 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.795+0000 7fd0a5dba640 1 -- 172.21.15.154:0/127632062 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0a015eac0 msgr2=0x7fd0a01f4b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:03.798 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.795+0000 7fd0a5dba640 1 --2- 172.21.15.154:0/127632062 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0a015eac0 0x7fd0a01f4b10 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd09800c6e0 tx=0x7fd09800cba0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.798 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.795+0000 7fd0a5dba640 1 -- 172.21.15.154:0/127632062 shutdown_connections 2024-04-15T15:19:03.798 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.795+0000 7fd0a5dba640 1 --2- 172.21.15.154:0/127632062 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd0a00cf000 0x7fd0a01f45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.798 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.795+0000 7fd0a5dba640 1 --2- 172.21.15.154:0/127632062 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0a015eac0 0x7fd0a01f4b10 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.798 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.795+0000 7fd0a5dba640 1 --2- 172.21.15.154:0/127632062 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0a00c7d90 0x7fd0a0150420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.798 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.795+0000 7fd0a5dba640 1 -- 172.21.15.154:0/127632062 >> 172.21.15.154:0/127632062 conn(0x7fd0a009aa40 msgr2=0x7fd0a015bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:03.798 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.795+0000 7fd0a5dba640 1 -- 172.21.15.154:0/127632062 shutdown_connections 2024-04-15T15:19:03.798 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:03.795+0000 7fd0a5dba640 1 -- 172.21.15.154:0/127632062 wait complete. 2024-04-15T15:19:03.808 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-15T15:19:03.808 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-15T15:19:03.810 INFO:tasks.ceph.osd.0:Started 2024-04-15T15:19:03.810 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-15T15:19:03.811 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-15T15:19:03.813 INFO:tasks.ceph.osd.1:Started 2024-04-15T15:19:03.813 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-15T15:19:03.813 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-15T15:19:03.815 INFO:tasks.ceph.osd.2:Started 2024-04-15T15:19:03.815 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-15T15:19:03.815 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-15T15:19:03.817 INFO:tasks.ceph.osd.3:Started 2024-04-15T15:19:03.817 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-15T15:19:03.818 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-15T15:19:03.820 INFO:tasks.ceph.osd.4:Started 2024-04-15T15:19:03.820 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-15T15:19:03.820 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-15T15:19:03.822 INFO:tasks.ceph.osd.5:Started 2024-04-15T15:19:03.822 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-15T15:19:03.822 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-15T15:19:03.823 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-04-15T15:19:03.822+0000 7ff178e16040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-15T15:19:03.824 INFO:tasks.ceph.osd.6:Started 2024-04-15T15:19:03.824 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-15T15:19:03.824 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-15T15:19:03.829 INFO:tasks.ceph.osd.7:Started 2024-04-15T15:19:03.829 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:19:03.874 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-04-15T15:19:03.872+0000 7f64fa816040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-15T15:19:03.933 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-04-15T15:19:03.932+0000 7f64fa816040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-15T15:19:03.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:03.976+0000 7f93d5b8f640 1 Processor -- start 2024-04-15T15:19:03.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:03.976+0000 7f93d5b8f640 1 -- start start 2024-04-15T15:19:03.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:03.977+0000 7f93d5b8f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f93d00d18e0 0x7f93d0162630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:03.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:03.977+0000 7f93d5b8f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f93d0162b70 0x7f93d00d3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:03.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:03.977+0000 7f93d5b8f640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93d015ef40 con 0x7f93d015eb60 2024-04-15T15:19:03.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:03.977+0000 7f93d5b8f640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f93d0163540 con 0x7f93d00d18e0 2024-04-15T15:19:03.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:03.977+0000 7f93d5b8f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f93d00d2fc0 con 0x7f93d0162b70 2024-04-15T15:19:03.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:03.977+0000 7f93c7fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f93d00d18e0 0x7f93d0162630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:03.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:03.977+0000 7f93cf7fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f93d015eb60 0x7f93d00d11d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:35490/0 (socket says 172.21.15.105:35490) 2024-04-15T15:19:03.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:03.977+0000 7f93cf7fe640 1 -- 172.21.15.105:0/737647420 learned_addr learned my addr 172.21.15.105:0/737647420 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:19:03.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:03.977+0000 7f93cffff640 1 --2- 172.21.15.105:0/737647420 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f93d0162b70 0x7f93d00d3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:03.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:03.977+0000 7f93c7fff640 1 -- 172.21.15.105:0/737647420 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f93d0162b70 msgr2=0x7f93d00d3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:03.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:03.977+0000 7f93c7fff640 1 --2- 172.21.15.105:0/737647420 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f93d0162b70 0x7f93d00d3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:03.977+0000 7f93c7fff640 1 -- 172.21.15.105:0/737647420 >> v1:172.21.15.105:6789/0 conn(0x7f93d015eb60 legacy=0x7f93d00d11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:03.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:03.977+0000 7f93ceffd640 1 -- 172.21.15.105:0/737647420 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1082946893 0 0) 0x7f93d015ef40 con 0x7f93d015eb60 2024-04-15T15:19:03.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:03.977+0000 7f93c7fff640 1 -- 172.21.15.105:0/737647420 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93d0163260 con 0x7f93d00d18e0 2024-04-15T15:19:03.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:03.977+0000 7f93cffff640 1 --2- 172.21.15.105:0/737647420 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f93d0162b70 0x7f93d00d3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:19:03.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:03.978+0000 7f93c7fff640 1 --2- 172.21.15.105:0/737647420 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f93d00d18e0 0x7f93d0162630 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f93b80097a0 tx=0x7f93b802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3aa0b71d3051860 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:03.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:03.978+0000 7f93ceffd640 1 -- 172.21.15.105:0/737647420 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b8035020 con 0x7f93d00d18e0 2024-04-15T15:19:03.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:03.978+0000 7f93ceffd640 1 -- 172.21.15.105:0/737647420 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93b8031830 con 0x7f93d00d18e0 2024-04-15T15:19:03.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:03.978+0000 7f93ceffd640 1 -- 172.21.15.105:0/737647420 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b8031c00 con 0x7f93d00d18e0 2024-04-15T15:19:03.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:03.979+0000 7f93d5b8f640 1 -- 172.21.15.105:0/737647420 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f93d00d18e0 msgr2=0x7f93d0162630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:03.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:03.979+0000 7f93d5b8f640 1 --2- 172.21.15.105:0/737647420 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f93d00d18e0 0x7f93d0162630 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f93b80097a0 tx=0x7f93b802b8d0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:03.979+0000 7f93d5b8f640 1 -- 172.21.15.105:0/737647420 shutdown_connections 2024-04-15T15:19:03.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:03.979+0000 7f93d5b8f640 1 --2- 172.21.15.105:0/737647420 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f93d0162b70 0x7f93d00d3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:03.979+0000 7f93d5b8f640 1 --2- 172.21.15.105:0/737647420 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f93d00d18e0 0x7f93d0162630 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:03.979+0000 7f93d5b8f640 1 -- 172.21.15.105:0/737647420 >> 172.21.15.105:0/737647420 conn(0x7f93d0067720 msgr2=0x7f93d0067b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:03.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:03.979+0000 7f93d5b8f640 1 -- 172.21.15.105:0/737647420 shutdown_connections 2024-04-15T15:19:03.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:03.979+0000 7f93d5b8f640 1 -- 172.21.15.105:0/737647420 wait complete. 2024-04-15T15:19:03.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:03.980+0000 7f93d5b8f640 1 Processor -- start 2024-04-15T15:19:03.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:03.980+0000 7f93d5b8f640 1 -- start start 2024-04-15T15:19:03.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:03.981+0000 7f93d5b8f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f93d00d18e0 0x7f93d00b4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:03.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:03.981+0000 7f93d5b8f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93d015eb60 0x7f93d01f4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:03.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:03.981+0000 7f93d5b8f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f93d0162b70 0x7f93d00b9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:03.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:03.981+0000 7f93d5b8f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f93d0162f80 con 0x7f93d0162b70 2024-04-15T15:19:03.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:03.981+0000 7f93d5b8f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f93d00d2c50 con 0x7f93d015eb60 2024-04-15T15:19:03.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:03.981+0000 7f93d5b8f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f93d00b78f0 con 0x7f93d00d18e0 2024-04-15T15:19:03.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:03.981+0000 7f93cffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f93d0162b70 0x7f93d00b9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:03.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:03.981+0000 7f93cffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f93d0162b70 0x7f93d00b9560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55586/0 (socket says 172.21.15.105:55586) 2024-04-15T15:19:03.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:03.981+0000 7f93cffff640 1 -- 172.21.15.105:0/4078891305 learned_addr learned my addr 172.21.15.105:0/4078891305 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:03.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:03.981+0000 7f93cf7fe640 1 --2- 172.21.15.105:0/4078891305 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f93d00d18e0 0x7f93d00b4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:03.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:03.981+0000 7f93c7fff640 1 --2- 172.21.15.105:0/4078891305 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93d015eb60 0x7f93d01f4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:03.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:03.981+0000 7f93cffff640 1 -- 172.21.15.105:0/4078891305 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f93d00d18e0 msgr2=0x7f93d00b4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:03.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:03.981+0000 7f93cffff640 1 --2- 172.21.15.105:0/4078891305 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f93d00d18e0 0x7f93d00b4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:03.981+0000 7f93cffff640 1 -- 172.21.15.105:0/4078891305 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93d015eb60 msgr2=0x7f93d01f4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:03.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:03.981+0000 7f93cffff640 1 --2- 172.21.15.105:0/4078891305 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93d015eb60 0x7f93d01f4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:03.981+0000 7f93cffff640 1 -- 172.21.15.105:0/4078891305 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93d00b4820 con 0x7f93d0162b70 2024-04-15T15:19:03.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:03.981+0000 7f93cffff640 1 --2- 172.21.15.105:0/4078891305 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f93d0162b70 0x7f93d00b9560 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f93c8007df0 tx=0x7f93c800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:03.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:03.982+0000 7f93ccff9640 1 -- 172.21.15.105:0/4078891305 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c801a070 con 0x7f93d0162b70 2024-04-15T15:19:03.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:03.982+0000 7f93ccff9640 1 -- 172.21.15.105:0/4078891305 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93c8003990 con 0x7f93d0162b70 2024-04-15T15:19:03.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:03.982+0000 7f93d5b8f640 1 -- 172.21.15.105:0/4078891305 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93d00b6230 con 0x7f93d0162b70 2024-04-15T15:19:03.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:03.982+0000 7f93ccff9640 1 -- 172.21.15.105:0/4078891305 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c8003d80 con 0x7f93d0162b70 2024-04-15T15:19:03.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:03.982+0000 7f93d5b8f640 1 -- 172.21.15.105:0/4078891305 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93d00b67f0 con 0x7f93d0162b70 2024-04-15T15:19:03.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:03.982+0000 7f93ccff9640 1 -- 172.21.15.105:0/4078891305 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f93c801d070 con 0x7f93d0162b70 2024-04-15T15:19:03.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:03.982+0000 7f93ccff9640 1 -- 172.21.15.105:0/4078891305 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f93c8015b10 con 0x7f93d0162b70 2024-04-15T15:19:03.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:03.983+0000 7f93d5b8f640 1 -- 172.21.15.105:0/4078891305 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93d0068a40 con 0x7f93d0162b70 2024-04-15T15:19:03.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:03.985+0000 7f93ccff9640 1 -- 172.21.15.105:0/4078891305 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f93c8024080 con 0x7f93d0162b70 2024-04-15T15:19:03.987 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-04-15T15:19:03.986+0000 7ff178e16040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-15T15:19:04.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:04.045+0000 7f93d5b8f640 1 -- 172.21.15.105:0/4078891305 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f93d00d23f0 con 0x7f93d0162b70 2024-04-15T15:19:04.047 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-04-15T15:19:04.045+0000 7f64fa816040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-15T15:19:04.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:04.046+0000 7f93ccff9640 1 -- 172.21.15.105:0/4078891305 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f93c8010070 con 0x7f93d0162b70 2024-04-15T15:19:04.048 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:19:04.048 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":10,"fsid":"09e4d2a4-a324-4793-af77-644b25e520e1","created":"2024-04-15T15:19:01.524543+0000","modified":"2024-04-15T15:19:03.788126+0000","last_up_change":"0.000000","last_in_change":"2024-04-15T15:19:03.788126+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"b6f6292b-2d7e-4aa0-9d1c-6ddd0f97287c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"8900966a-b393-41e3-8f93-e5a43bd23622","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"81a5dc7a-9325-4ff5-941d-a2d5fba21f1d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"df766ed2-5c71-4805-96f4-3d4970edc960","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"e1039e03-28b8-49df-b951-c91930b93c9a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"a9f5eaf1-66ff-405a-a9e4-ec3340812a13","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"267d02ea-675f-428f-aae8-e0375e4d4feb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"759ac6cc-896a-423c-885f-9fe90d8dcb14","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:19:04.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:04.046+0000 7f93d5b8f640 1 -- 172.21.15.105:0/4078891305 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f93d0162b70 msgr2=0x7f93d00b9560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:04.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:04.046+0000 7f93d5b8f640 1 --2- 172.21.15.105:0/4078891305 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f93d0162b70 0x7f93d00b9560 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f93c8007df0 tx=0x7f93c800d4a0 comp rx=0 tx=0).stop 2024-04-15T15:19:04.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:04.047+0000 7f93d5b8f640 1 -- 172.21.15.105:0/4078891305 shutdown_connections 2024-04-15T15:19:04.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:04.047+0000 7f93d5b8f640 1 --2- 172.21.15.105:0/4078891305 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f93d0162b70 0x7f93d00b9560 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:04.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:04.047+0000 7f93d5b8f640 1 --2- 172.21.15.105:0/4078891305 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93d015eb60 0x7f93d01f4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:04.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:04.047+0000 7f93d5b8f640 1 --2- 172.21.15.105:0/4078891305 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f93d00d18e0 0x7f93d00b4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:04.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:04.047+0000 7f93d5b8f640 1 -- 172.21.15.105:0/4078891305 >> 172.21.15.105:0/4078891305 conn(0x7f93d0067720 msgr2=0x7f93d00b3740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:04.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:04.047+0000 7f93d5b8f640 1 -- 172.21.15.105:0/4078891305 shutdown_connections 2024-04-15T15:19:04.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:04.047+0000 7f93d5b8f640 1 -- 172.21.15.105:0/4078891305 wait complete. 2024-04-15T15:19:04.060 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-15T15:19:04.060 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-15T15:19:04.086 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-04-15T15:19:04.084+0000 7ff178e16040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-15T15:19:04.141 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-04-15T15:19:04.139+0000 7f64fa816040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-15T15:19:04.242 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-04-15T15:19:04.241+0000 7ff178e16040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-15T15:19:04.284 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-04-15T15:19:04.283+0000 7ff178e16040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-15T15:19:04.296 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-04-15T15:19:04.294+0000 7f64fa816040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-15T15:19:04.337 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-04-15T15:19:04.336+0000 7f64fa816040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-15T15:19:04.416 INFO:tasks.ceph.osd.0.smithi105.stderr:2024-04-15T15:19:04.414+0000 7ff830014640 -1 Falling back to public interface 2024-04-15T15:19:04.423 INFO:tasks.ceph.osd.2.smithi105.stderr:2024-04-15T15:19:04.421+0000 7f5434217640 -1 Falling back to public interface 2024-04-15T15:19:04.429 INFO:tasks.ceph.osd.5.smithi154.stderr:2024-04-15T15:19:04.427+0000 7f8d2320d640 -1 Falling back to public interface 2024-04-15T15:19:04.440 INFO:tasks.ceph.osd.3.smithi105.stderr:2024-04-15T15:19:04.438+0000 7f4a99048640 -1 Falling back to public interface 2024-04-15T15:19:04.440 INFO:tasks.ceph.osd.1.smithi105.stderr:2024-04-15T15:19:04.438+0000 7f094d60f640 -1 Falling back to public interface 2024-04-15T15:19:04.441 INFO:tasks.ceph.osd.4.smithi154.stderr:2024-04-15T15:19:04.439+0000 7f860ca08640 -1 Falling back to public interface 2024-04-15T15:19:04.441 INFO:tasks.ceph.osd.7.smithi154.stderr:2024-04-15T15:19:04.439+0000 7f897b41f640 -1 Falling back to public interface 2024-04-15T15:19:04.442 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-04-15T15:19:04.440+0000 7ff178e16040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-15T15:19:04.443 INFO:tasks.ceph.osd.7.smithi154.stderr:2024-04-15T15:19:04.441+0000 7f8979e10640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6818/0: (98) Address already in use 2024-04-15T15:19:04.443 INFO:tasks.ceph.osd.3.smithi105.stderr:2024-04-15T15:19:04.441+0000 7f4a97ff7640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6826/0: (98) Address already in use 2024-04-15T15:19:04.448 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-04-15T15:19:04.446+0000 7f0165e24640 -1 Falling back to public interface 2024-04-15T15:19:04.496 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-04-15T15:19:04.494+0000 7f64fa816040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-15T15:19:04.866 DEBUG:teuthology.orchestra.run.smithi105:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-15T15:19:04.960 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:04.958+0000 7f1adf19d640 1 Processor -- start 2024-04-15T15:19:04.960 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:04.958+0000 7f1adf19d640 1 -- start start 2024-04-15T15:19:04.961 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:04.959+0000 7f1adf19d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1ad80cd620 0x7f1ad80d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:04.961 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:04.959+0000 7f1adf19d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ad806b9e0 0x7f1ad806be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:04.961 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:04.959+0000 7f1adf19d640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ad80ced60 con 0x7f1ad80b1d70 2024-04-15T15:19:04.961 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:04.959+0000 7f1adf19d640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f1ad80ce1c0 con 0x7f1ad80cd620 2024-04-15T15:19:04.961 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:04.959+0000 7f1adf19d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f1ad80b2150 con 0x7f1ad806b9e0 2024-04-15T15:19:04.961 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:04.959+0000 7f1add73a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ad806b9e0 0x7f1ad806be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:04.961 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:04.959+0000 7f1ad7fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1ad80cd620 0x7f1ad80d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:04.962 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:04.959+0000 7f1ad7fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1ad80cd620 0x7f1ad80d0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.105:56772/0 (socket says 172.21.15.105:56772) 2024-04-15T15:19:04.962 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:04.959+0000 7f1ad7fff640 1 -- 172.21.15.105:0/920496500 learned_addr learned my addr 172.21.15.105:0/920496500 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:04.962 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:04.959+0000 7f1add73a640 1 -- 172.21.15.105:0/920496500 >> v2:172.21.15.154:3300/0 conn(0x7f1ad80cd620 msgr2=0x7f1ad80d0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:04.962 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:04.959+0000 7f1add73a640 1 --2- 172.21.15.105:0/920496500 >> v2:172.21.15.154:3300/0 conn(0x7f1ad80cd620 0x7f1ad80d0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:04.962 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:04.959+0000 7f1add73a640 1 -- 172.21.15.105:0/920496500 >> v1:172.21.15.154:6789/0 conn(0x7f1ad80b1d70 legacy=0x7f1ad80ce650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:04.962 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:04.959+0000 7f1add73a640 1 -- 172.21.15.105:0/920496500 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ad80d0d20 con 0x7f1ad806b9e0 2024-04-15T15:19:04.962 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:04.959+0000 7f1ad7fff640 1 --2- 172.21.15.105:0/920496500 >> v2:172.21.15.154:3300/0 conn(0x7f1ad80cd620 0x7f1ad80d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:19:04.963 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:04.960+0000 7f1add73a640 1 --2- 172.21.15.105:0/920496500 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ad806b9e0 0x7f1ad806be90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0009d90 tx=0x7f1ad002bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=872d4afa4077f077 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:04.963 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:04.960+0000 7f1ad77fe640 1 -- 172.21.15.105:0/920496500 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad0039070 con 0x7f1ad806b9e0 2024-04-15T15:19:04.963 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:04.960+0000 7f1ad77fe640 1 -- 172.21.15.105:0/920496500 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ad002cc50 con 0x7f1ad806b9e0 2024-04-15T15:19:04.963 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:04.960+0000 7f1ad77fe640 1 -- 172.21.15.105:0/920496500 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad0034440 con 0x7f1ad806b9e0 2024-04-15T15:19:04.963 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:04.961+0000 7f1adf19d640 1 -- 172.21.15.105:0/920496500 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ad806b9e0 msgr2=0x7f1ad806be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:04.964 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:04.961+0000 7f1adf19d640 1 --2- 172.21.15.105:0/920496500 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ad806b9e0 0x7f1ad806be90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0009d90 tx=0x7f1ad002bea0 comp rx=0 tx=0).stop 2024-04-15T15:19:04.964 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:04.961+0000 7f1adf19d640 1 -- 172.21.15.105:0/920496500 shutdown_connections 2024-04-15T15:19:04.964 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:04.961+0000 7f1adf19d640 1 --2- 172.21.15.105:0/920496500 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ad806b9e0 0x7f1ad806be90 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:04.964 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:04.961+0000 7f1adf19d640 1 --2- 172.21.15.105:0/920496500 >> v2:172.21.15.154:3300/0 conn(0x7f1ad80cd620 0x7f1ad80d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:04.964 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:04.961+0000 7f1adf19d640 1 -- 172.21.15.105:0/920496500 >> 172.21.15.105:0/920496500 conn(0x7f1ad809a9d0 msgr2=0x7f1ad80b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:04.964 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:04.961+0000 7f1adf19d640 1 -- 172.21.15.105:0/920496500 shutdown_connections 2024-04-15T15:19:04.964 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:04.961+0000 7f1adf19d640 1 -- 172.21.15.105:0/920496500 wait complete. 2024-04-15T15:19:04.966 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:04.962+0000 7f1adf19d640 1 Processor -- start 2024-04-15T15:19:04.966 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:04.962+0000 7f1adf19d640 1 -- start start 2024-04-15T15:19:04.967 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:04.963+0000 7f1adf19d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ad806b9e0 0x7f1ad80ae2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:04.967 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:04.963+0000 7f1adf19d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ad80b1d70 0x7f1ad8155e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:04.967 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:04.963+0000 7f1adf19d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ad80cd620 0x7f1ad80afb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:04.967 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:04.963+0000 7f1adf19d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad809ade0 con 0x7f1ad806b9e0 2024-04-15T15:19:04.967 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:04.963+0000 7f1adf19d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad80cde50 con 0x7f1ad80cd620 2024-04-15T15:19:04.967 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:04.963+0000 7f1adf19d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f1ad809af60 con 0x7f1ad80b1d70 2024-04-15T15:19:04.967 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:04.963+0000 7f1adcf39640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ad806b9e0 0x7f1ad80ae2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:04.967 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:04.963+0000 7f1adcf39640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ad806b9e0 0x7f1ad80ae2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55600/0 (socket says 172.21.15.105:55600) 2024-04-15T15:19:04.967 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:04.963+0000 7f1adcf39640 1 -- 172.21.15.105:0/2566424292 learned_addr learned my addr 172.21.15.105:0/2566424292 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:04.967 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:04.963+0000 7f1add73a640 1 --2- 172.21.15.105:0/2566424292 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ad80cd620 0x7f1ad80afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:04.967 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:04.963+0000 7f1ad7fff640 1 --2- 172.21.15.105:0/2566424292 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ad80b1d70 0x7f1ad8155e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:04.967 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:04.963+0000 7f1adcf39640 1 -- 172.21.15.105:0/2566424292 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ad80b1d70 msgr2=0x7f1ad8155e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:04.967 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:04.964+0000 7f1adcf39640 1 --2- 172.21.15.105:0/2566424292 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ad80b1d70 0x7f1ad8155e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:04.967 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:04.964+0000 7f1adcf39640 1 -- 172.21.15.105:0/2566424292 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ad80cd620 msgr2=0x7f1ad80afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:04.968 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:04.964+0000 7f1adcf39640 1 --2- 172.21.15.105:0/2566424292 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ad80cd620 0x7f1ad80afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:04.968 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:04.964+0000 7f1adcf39640 1 -- 172.21.15.105:0/2566424292 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ad8156f00 con 0x7f1ad806b9e0 2024-04-15T15:19:04.968 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:04.964+0000 7f1adcf39640 1 --2- 172.21.15.105:0/2566424292 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ad806b9e0 0x7f1ad80ae2a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1acc00c400 tx=0x7f1acc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:04.968 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:04.964+0000 7f1ad57fa640 1 -- 172.21.15.105:0/2566424292 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1acc0052d0 con 0x7f1ad806b9e0 2024-04-15T15:19:04.968 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:04.964+0000 7f1ad57fa640 1 -- 172.21.15.105:0/2566424292 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1acc005490 con 0x7f1ad806b9e0 2024-04-15T15:19:04.968 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:04.964+0000 7f1adf19d640 1 -- 172.21.15.105:0/2566424292 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ad80b0860 con 0x7f1ad806b9e0 2024-04-15T15:19:04.968 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:04.964+0000 7f1ad57fa640 1 -- 172.21.15.105:0/2566424292 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1acc007260 con 0x7f1ad806b9e0 2024-04-15T15:19:04.968 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:04.964+0000 7f1adf19d640 1 -- 172.21.15.105:0/2566424292 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ad80b0e20 con 0x7f1ad806b9e0 2024-04-15T15:19:04.968 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:04.964+0000 7f1ad57fa640 1 -- 172.21.15.105:0/2566424292 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1acc007400 con 0x7f1ad806b9e0 2024-04-15T15:19:04.968 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:04.964+0000 7f1ad57fa640 1 -- 172.21.15.105:0/2566424292 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f1acc011c60 con 0x7f1ad806b9e0 2024-04-15T15:19:04.968 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:04.965+0000 7f1adf19d640 1 -- 172.21.15.105:0/2566424292 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ad804fd50 con 0x7f1ad806b9e0 2024-04-15T15:19:04.969 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:04.967+0000 7f1ad57fa640 1 -- 172.21.15.105:0/2566424292 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f1ad80ced60 con 0x7f1ad806b9e0 2024-04-15T15:19:05.031 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:05.029+0000 7f1adf19d640 1 -- 172.21.15.105:0/2566424292 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1ad80b7430 con 0x7f1ad806b9e0 2024-04-15T15:19:05.031 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:05.030+0000 7f1ad57fa640 1 -- 172.21.15.105:0/2566424292 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f1acc0035f0 con 0x7f1ad806b9e0 2024-04-15T15:19:05.031 INFO:teuthology.misc.health.smithi105.stdout: 2024-04-15T15:19:05.032 INFO:teuthology.misc.health.smithi105.stdout:{"epoch":10,"fsid":"09e4d2a4-a324-4793-af77-644b25e520e1","created":"2024-04-15T15:19:01.524543+0000","modified":"2024-04-15T15:19:03.788126+0000","last_up_change":"0.000000","last_in_change":"2024-04-15T15:19:03.788126+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"b6f6292b-2d7e-4aa0-9d1c-6ddd0f97287c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"8900966a-b393-41e3-8f93-e5a43bd23622","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"81a5dc7a-9325-4ff5-941d-a2d5fba21f1d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"df766ed2-5c71-4805-96f4-3d4970edc960","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"e1039e03-28b8-49df-b951-c91930b93c9a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"a9f5eaf1-66ff-405a-a9e4-ec3340812a13","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"267d02ea-675f-428f-aae8-e0375e4d4feb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"759ac6cc-896a-423c-885f-9fe90d8dcb14","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:19:05.032 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:05.030+0000 7f1adf19d640 1 -- 172.21.15.105:0/2566424292 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ad806b9e0 msgr2=0x7f1ad80ae2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:05.032 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:05.030+0000 7f1adf19d640 1 --2- 172.21.15.105:0/2566424292 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ad806b9e0 0x7f1ad80ae2a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1acc00c400 tx=0x7f1acc00c8c0 comp rx=0 tx=0).stop 2024-04-15T15:19:05.033 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:05.031+0000 7f1adf19d640 1 -- 172.21.15.105:0/2566424292 shutdown_connections 2024-04-15T15:19:05.033 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:05.031+0000 7f1adf19d640 1 --2- 172.21.15.105:0/2566424292 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ad80b1d70 0x7f1ad8155e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:05.033 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:05.031+0000 7f1adf19d640 1 --2- 172.21.15.105:0/2566424292 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ad80cd620 0x7f1ad80afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:05.033 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:05.031+0000 7f1adf19d640 1 --2- 172.21.15.105:0/2566424292 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ad806b9e0 0x7f1ad80ae2a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:05.033 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:05.031+0000 7f1adf19d640 1 -- 172.21.15.105:0/2566424292 >> 172.21.15.105:0/2566424292 conn(0x7f1ad809a9d0 msgr2=0x7f1ad81607d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:05.033 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:05.031+0000 7f1adf19d640 1 -- 172.21.15.105:0/2566424292 shutdown_connections 2024-04-15T15:19:05.033 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:05.031+0000 7f1adf19d640 1 -- 172.21.15.105:0/2566424292 wait complete. 2024-04-15T15:19:05.042 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-15T15:19:05.285 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-04-15T15:19:05.282+0000 7ff178e16040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-15T15:19:05.328 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-04-15T15:19:05.326+0000 7f64fa816040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-15T15:19:05.560 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-04-15T15:19:05.558+0000 7ff178e16040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-15T15:19:05.606 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-04-15T15:19:05.605+0000 7f64fa816040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-15T15:19:05.611 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-04-15T15:19:05.609+0000 7ff178e16040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-15T15:19:05.658 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-04-15T15:19:05.656+0000 7f64fa816040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-15T15:19:05.725 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-04-15T15:19:05.724+0000 7ff178e16040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-15T15:19:05.776 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-04-15T15:19:05.774+0000 7f64fa816040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-15T15:19:06.739 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-04-15T15:19:06.737+0000 7ff11967d640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-15T15:19:06.739 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-04-15T15:19:06.737+0000 7ff11967d640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-15T15:19:07.899 INFO:tasks.ceph.osd.0.smithi105.stderr:2024-04-15T15:19:07.898+0000 7ff830014640 -1 osd.0 0 log_to_monitors true 2024-04-15T15:19:07.918 INFO:tasks.ceph.osd.7.smithi154.stderr:2024-04-15T15:19:07.917+0000 7f897b41f640 -1 osd.7 0 log_to_monitors true 2024-04-15T15:19:07.949 INFO:tasks.ceph.osd.2.smithi105.stderr:2024-04-15T15:19:07.947+0000 7f5434217640 -1 osd.2 0 log_to_monitors true 2024-04-15T15:19:07.969 INFO:tasks.ceph.osd.3.smithi105.stderr:2024-04-15T15:19:07.967+0000 7f4a99048640 -1 osd.3 0 log_to_monitors true 2024-04-15T15:19:07.969 INFO:tasks.ceph.osd.1.smithi105.stderr:2024-04-15T15:19:07.967+0000 7f094d60f640 -1 osd.1 0 log_to_monitors true 2024-04-15T15:19:07.972 INFO:tasks.ceph.osd.5.smithi154.stderr:2024-04-15T15:19:07.971+0000 7f8d2320d640 -1 osd.5 0 log_to_monitors true 2024-04-15T15:19:07.981 INFO:tasks.ceph.osd.4.smithi154.stderr:2024-04-15T15:19:07.979+0000 7f860ca08640 -1 osd.4 0 log_to_monitors true 2024-04-15T15:19:07.981 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-04-15T15:19:07.979+0000 7f0165e24640 -1 osd.6 0 log_to_monitors true 2024-04-15T15:19:11.824 INFO:tasks.ceph.osd.2.smithi105.stderr:2024-04-15T15:19:11.822+0000 7f542fc7d640 -1 osd.2 0 waiting for initial osdmap 2024-04-15T15:19:11.832 INFO:tasks.ceph.osd.2.smithi105.stderr:2024-04-15T15:19:11.831+0000 7f5427a9d640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-15T15:19:11.838 INFO:tasks.ceph.osd.3.smithi105.stderr:2024-04-15T15:19:11.836+0000 7f4a94fde640 -1 osd.3 0 waiting for initial osdmap 2024-04-15T15:19:11.847 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-04-15T15:19:11.846+0000 7f01616fb640 -1 osd.6 0 waiting for initial osdmap 2024-04-15T15:19:11.847 INFO:tasks.ceph.osd.5.smithi154.stderr:2024-04-15T15:19:11.846+0000 7f8d1eb7d640 -1 osd.5 0 waiting for initial osdmap 2024-04-15T15:19:11.849 DEBUG:teuthology.orchestra.run.smithi105:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-15T15:19:11.852 INFO:tasks.ceph.osd.3.smithi105.stderr:2024-04-15T15:19:11.851+0000 7f4a8cdfe640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-15T15:19:11.856 INFO:tasks.ceph.osd.4.smithi154.stderr:2024-04-15T15:19:11.855+0000 7f860899e640 -1 osd.4 0 waiting for initial osdmap 2024-04-15T15:19:11.857 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-04-15T15:19:11.856+0000 7f015951b640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-15T15:19:11.858 INFO:tasks.ceph.osd.5.smithi154.stderr:2024-04-15T15:19:11.857+0000 7f8d1699d640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-15T15:19:11.865 INFO:tasks.ceph.osd.1.smithi105.stderr:2024-04-15T15:19:11.864+0000 7f0948fc0640 -1 osd.1 0 waiting for initial osdmap 2024-04-15T15:19:11.869 INFO:tasks.ceph.osd.4.smithi154.stderr:2024-04-15T15:19:11.868+0000 7f86007be640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-15T15:19:11.870 INFO:tasks.ceph.osd.7.smithi154.stderr:2024-04-15T15:19:11.868+0000 7f897760b640 -1 osd.7 0 waiting for initial osdmap 2024-04-15T15:19:11.870 INFO:tasks.ceph.osd.0.smithi105.stderr:2024-04-15T15:19:11.869+0000 7ff82b961640 -1 osd.0 0 waiting for initial osdmap 2024-04-15T15:19:11.877 INFO:tasks.ceph.osd.1.smithi105.stderr:2024-04-15T15:19:11.876+0000 7f0940de0640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-15T15:19:11.880 INFO:tasks.ceph.osd.0.smithi105.stderr:2024-04-15T15:19:11.879+0000 7ff823781640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-15T15:19:11.885 INFO:tasks.ceph.osd.7.smithi154.stderr:2024-04-15T15:19:11.884+0000 7f896ec17640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-15T15:19:11.943 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:11.941+0000 7f1e252df640 1 Processor -- start 2024-04-15T15:19:11.943 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:11.942+0000 7f1e252df640 1 -- start start 2024-04-15T15:19:11.945 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:11.942+0000 7f1e252df640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1e200ce090 0x7f1e200cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:11.945 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:11.942+0000 7f1e252df640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e200cf8a0 con 0x7f1e200ce470 2024-04-15T15:19:11.945 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:11.942+0000 7f1e252df640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e200cfb40 con 0x7f1e2015ba90 2024-04-15T15:19:11.946 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:11.942+0000 7f1e252df640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f1e200d03e0 con 0x7f1e200ce090 2024-04-15T15:19:11.946 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:11.943+0000 7f1e1e7fc640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f1e200ce470 0x7f1e2015b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:53152/0 (socket says 172.21.15.105:53152) 2024-04-15T15:19:11.946 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:11.943+0000 7f1e1e7fc640 1 -- 172.21.15.105:0/3178305616 learned_addr learned my addr 172.21.15.105:0/3178305616 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:19:11.946 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:11.943+0000 7f1e1effd640 1 --2- 172.21.15.105:0/3178305616 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1e200ce090 0x7f1e200cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:11.946 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:11.943+0000 7f1e1dffb640 1 -- 172.21.15.105:0/3178305616 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3946667056 0 0) 0x7f1e200cf8a0 con 0x7f1e200ce470 2024-04-15T15:19:11.946 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:11.943+0000 7f1e1dffb640 1 -- 172.21.15.105:0/3178305616 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e08003520 con 0x7f1e200ce470 2024-04-15T15:19:11.946 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:11.943+0000 7f1e1dffb640 1 -- 172.21.15.105:0/3178305616 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1040945879 0 0) 0x7f1e200cfb40 con 0x7f1e2015ba90 2024-04-15T15:19:11.946 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:11.943+0000 7f1e1dffb640 1 -- 172.21.15.105:0/3178305616 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e200cf8a0 con 0x7f1e2015ba90 2024-04-15T15:19:11.946 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:11.943+0000 7f1e1effd640 1 -- 172.21.15.105:0/3178305616 >> v1:172.21.15.154:6789/0 conn(0x7f1e2015ba90 legacy=0x7f1e200cbfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:11.946 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:11.943+0000 7f1e1effd640 1 -- 172.21.15.105:0/3178305616 >> v1:172.21.15.105:6789/0 conn(0x7f1e200ce470 legacy=0x7f1e2015b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:11.946 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:11.943+0000 7f1e1effd640 1 -- 172.21.15.105:0/3178305616 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e200cfe20 con 0x7f1e200ce090 2024-04-15T15:19:11.946 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:11.944+0000 7f1e1effd640 1 --2- 172.21.15.105:0/3178305616 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1e200ce090 0x7f1e200cf0d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1e180097d0 tx=0x7f1e1802b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=714b3553bc0922bd server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:11.946 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:11.944+0000 7f1e1dffb640 1 -- 172.21.15.105:0/3178305616 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e18035020 con 0x7f1e200ce090 2024-04-15T15:19:11.947 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:11.944+0000 7f1e1dffb640 1 -- 172.21.15.105:0/3178305616 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e18031800 con 0x7f1e200ce090 2024-04-15T15:19:11.947 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:11.944+0000 7f1e1dffb640 1 -- 172.21.15.105:0/3178305616 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e18031bb0 con 0x7f1e200ce090 2024-04-15T15:19:11.948 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:11.944+0000 7f1e252df640 1 -- 172.21.15.105:0/3178305616 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1e200ce090 msgr2=0x7f1e200cf0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:11.948 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:11.944+0000 7f1e252df640 1 --2- 172.21.15.105:0/3178305616 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1e200ce090 0x7f1e200cf0d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1e180097d0 tx=0x7f1e1802b8a0 comp rx=0 tx=0).stop 2024-04-15T15:19:11.948 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:11.944+0000 7f1e252df640 1 -- 172.21.15.105:0/3178305616 shutdown_connections 2024-04-15T15:19:11.948 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:11.944+0000 7f1e252df640 1 --2- 172.21.15.105:0/3178305616 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1e200ce090 0x7f1e200cf0d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:11.948 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:11.944+0000 7f1e252df640 1 -- 172.21.15.105:0/3178305616 >> 172.21.15.105:0/3178305616 conn(0x7f1e2009a9d0 msgr2=0x7f1e200d1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:11.948 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:11.944+0000 7f1e252df640 1 -- 172.21.15.105:0/3178305616 shutdown_connections 2024-04-15T15:19:11.948 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:11.944+0000 7f1e252df640 1 -- 172.21.15.105:0/3178305616 wait complete. 2024-04-15T15:19:11.948 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:11.946+0000 7f1e252df640 1 Processor -- start 2024-04-15T15:19:11.948 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:11.946+0000 7f1e252df640 1 -- start start 2024-04-15T15:19:11.951 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:11.946+0000 7f1e252df640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1e200ce090 0x7f1e200b5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:11.951 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:11.946+0000 7f1e252df640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1e200ce470 0x7f1e2006bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:11.951 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:11.946+0000 7f1e252df640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e2015ba90 0x7f1e2014e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:11.951 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:11.946+0000 7f1e252df640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1e200af680 con 0x7f1e200ce090 2024-04-15T15:19:11.951 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:11.946+0000 7f1e252df640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1e200af500 con 0x7f1e2015ba90 2024-04-15T15:19:11.951 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:11.946+0000 7f1e252df640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f1e200af800 con 0x7f1e200ce470 2024-04-15T15:19:11.951 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:11.947+0000 7f1e1effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1e200ce090 0x7f1e200b5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:11.951 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:11.947+0000 7f1e1effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1e200ce090 0x7f1e200b5e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39020/0 (socket says 172.21.15.105:39020) 2024-04-15T15:19:11.951 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:11.947+0000 7f1e1effd640 1 -- 172.21.15.105:0/2498869840 learned_addr learned my addr 172.21.15.105:0/2498869840 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:11.951 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:11.947+0000 7f1e1e7fc640 1 --2- 172.21.15.105:0/2498869840 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1e200ce470 0x7f1e2006bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:11.951 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:11.947+0000 7f1e1f7fe640 1 --2- 172.21.15.105:0/2498869840 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e2015ba90 0x7f1e2014e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:11.951 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:11.947+0000 7f1e1effd640 1 -- 172.21.15.105:0/2498869840 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1e200ce470 msgr2=0x7f1e2006bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:11.952 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:11.947+0000 7f1e1effd640 1 --2- 172.21.15.105:0/2498869840 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1e200ce470 0x7f1e2006bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:11.952 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:11.947+0000 7f1e1effd640 1 -- 172.21.15.105:0/2498869840 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e2015ba90 msgr2=0x7f1e2014e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:11.952 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:11.947+0000 7f1e1effd640 1 --2- 172.21.15.105:0/2498869840 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e2015ba90 0x7f1e2014e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:11.952 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:11.947+0000 7f1e1effd640 1 -- 172.21.15.105:0/2498869840 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e2014c2e0 con 0x7f1e200ce090 2024-04-15T15:19:11.952 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:11.947+0000 7f1e1e7fc640 1 --2- 172.21.15.105:0/2498869840 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1e200ce470 0x7f1e2006bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:19:11.952 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:11.947+0000 7f1e1effd640 1 --2- 172.21.15.105:0/2498869840 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1e200ce090 0x7f1e200b5e00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1e1802b8e0 tx=0x7f1e180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:11.952 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:11.947+0000 7f1e077fe640 1 -- 172.21.15.105:0/2498869840 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e18040070 con 0x7f1e200ce090 2024-04-15T15:19:11.952 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:11.947+0000 7f1e077fe640 1 -- 172.21.15.105:0/2498869840 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e18031e30 con 0x7f1e200ce090 2024-04-15T15:19:11.952 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:11.947+0000 7f1e077fe640 1 -- 172.21.15.105:0/2498869840 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e18035040 con 0x7f1e200ce090 2024-04-15T15:19:11.952 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:11.947+0000 7f1e252df640 1 -- 172.21.15.105:0/2498869840 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e200b66c0 con 0x7f1e200ce090 2024-04-15T15:19:11.952 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:11.947+0000 7f1e252df640 1 -- 172.21.15.105:0/2498869840 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e2014c5a0 con 0x7f1e200ce090 2024-04-15T15:19:11.952 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:11.948+0000 7f1e252df640 1 -- 172.21.15.105:0/2498869840 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e2004fd50 con 0x7f1e200ce090 2024-04-15T15:19:11.952 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:11.949+0000 7f1e077fe640 1 -- 172.21.15.105:0/2498869840 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7f1e1804b060 con 0x7f1e200ce090 2024-04-15T15:19:11.952 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:11.949+0000 7f1e077fe640 1 --2- 172.21.15.105:0/2498869840 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f1e0c06c860 0x7f1e0c06ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:11.953 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:11.949+0000 7f1e1e7fc640 1 --2- 172.21.15.105:0/2498869840 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f1e0c06c860 0x7f1e0c06ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:11.953 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:11.949+0000 7f1e077fe640 1 -- 172.21.15.105:0/2498869840 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7f1e180ac510 con 0x7f1e200ce090 2024-04-15T15:19:11.953 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:11.950+0000 7f1e1e7fc640 1 --2- 172.21.15.105:0/2498869840 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f1e0c06c860 0x7f1e0c06ed20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1e20154cb0 tx=0x7f1e14009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:11.955 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:11.954+0000 7f1e077fe640 1 -- 172.21.15.105:0/2498869840 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1e1807b1f0 con 0x7f1e200ce090 2024-04-15T15:19:12.132 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:12.130+0000 7f1e252df640 1 -- 172.21.15.105:0/2498869840 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1e2015b760 con 0x7f1e200ce090 2024-04-15T15:19:12.132 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:12.131+0000 7f1e077fe640 1 -- 172.21.15.105:0/2498869840 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+7058 (secure 0 0 0) 0x7f1e1807f4f0 con 0x7f1e200ce090 2024-04-15T15:19:12.132 INFO:teuthology.misc.health.smithi105.stdout: 2024-04-15T15:19:12.133 INFO:teuthology.misc.health.smithi105.stdout:{"epoch":12,"fsid":"09e4d2a4-a324-4793-af77-644b25e520e1","created":"2024-04-15T15:19:01.524543+0000","modified":"2024-04-15T15:19:09.741584+0000","last_up_change":"0.000000","last_in_change":"2024-04-15T15:19:03.788126+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"b6f6292b-2d7e-4aa0-9d1c-6ddd0f97287c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"8900966a-b393-41e3-8f93-e5a43bd23622","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"81a5dc7a-9325-4ff5-941d-a2d5fba21f1d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"df766ed2-5c71-4805-96f4-3d4970edc960","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"e1039e03-28b8-49df-b951-c91930b93c9a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"a9f5eaf1-66ff-405a-a9e4-ec3340812a13","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"267d02ea-675f-428f-aae8-e0375e4d4feb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"759ac6cc-896a-423c-885f-9fe90d8dcb14","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:19:12.135 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:12.132+0000 7f1e252df640 1 -- 172.21.15.105:0/2498869840 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f1e0c06c860 msgr2=0x7f1e0c06ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:12.135 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:12.132+0000 7f1e252df640 1 --2- 172.21.15.105:0/2498869840 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f1e0c06c860 0x7f1e0c06ed20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1e20154cb0 tx=0x7f1e14009040 comp rx=0 tx=0).stop 2024-04-15T15:19:12.135 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:12.132+0000 7f1e252df640 1 -- 172.21.15.105:0/2498869840 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1e200ce090 msgr2=0x7f1e200b5e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:12.135 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:12.132+0000 7f1e252df640 1 --2- 172.21.15.105:0/2498869840 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1e200ce090 0x7f1e200b5e00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1e1802b8e0 tx=0x7f1e180046c0 comp rx=0 tx=0).stop 2024-04-15T15:19:12.135 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:12.132+0000 7f1e252df640 1 -- 172.21.15.105:0/2498869840 shutdown_connections 2024-04-15T15:19:12.135 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:12.133+0000 7f1e252df640 1 --2- 172.21.15.105:0/2498869840 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1e200ce470 0x7f1e2006bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:12.135 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:12.133+0000 7f1e252df640 1 --2- 172.21.15.105:0/2498869840 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f1e0c06c860 0x7f1e0c06ed20 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:12.135 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:12.133+0000 7f1e252df640 1 --2- 172.21.15.105:0/2498869840 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e2015ba90 0x7f1e2014e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:12.135 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:12.133+0000 7f1e252df640 1 --2- 172.21.15.105:0/2498869840 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1e200ce090 0x7f1e200b5e00 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:12.135 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:12.133+0000 7f1e252df640 1 -- 172.21.15.105:0/2498869840 >> 172.21.15.105:0/2498869840 conn(0x7f1e2009a9d0 msgr2=0x7f1e2015c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:12.135 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:12.133+0000 7f1e252df640 1 -- 172.21.15.105:0/2498869840 shutdown_connections 2024-04-15T15:19:12.135 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:12.133+0000 7f1e252df640 1 -- 172.21.15.105:0/2498869840 wait complete. 2024-04-15T15:19:12.144 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-15T15:19:18.951 DEBUG:teuthology.orchestra.run.smithi105:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-15T15:19:19.054 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.052+0000 7f1ba3faa640 1 Processor -- start 2024-04-15T15:19:19.054 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.052+0000 7f1ba3faa640 1 -- start start 2024-04-15T15:19:19.055 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.053+0000 7f1ba3faa640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1b9c0ce5a0 0x7f1b9c0cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:19.055 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.053+0000 7f1ba3faa640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b9c0cbb50 0x7f1b9c0cbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:19.055 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.053+0000 7f1ba3faa640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b9c06c4a0 con 0x7f1b9c15eb00 2024-04-15T15:19:19.055 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.053+0000 7f1ba3faa640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f1b9c0cd870 con 0x7f1b9c0ce5a0 2024-04-15T15:19:19.055 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.053+0000 7f1ba3faa640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f1b9c0cdac0 con 0x7f1b9c0cbb50 2024-04-15T15:19:19.055 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.053+0000 7f1ba1545640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1b9c0ce5a0 0x7f1b9c0cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:19.055 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.053+0000 7f1ba2547640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b9c0cbb50 0x7f1b9c0cbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:19.055 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.053+0000 7f1ba1545640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1b9c0ce5a0 0x7f1b9c0cea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.105:56320/0 (socket says 172.21.15.105:56320) 2024-04-15T15:19:19.055 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.053+0000 7f1ba1545640 1 -- 172.21.15.105:0/1071766094 learned_addr learned my addr 172.21.15.105:0/1071766094 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:19.055 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.053+0000 7f1ba1d46640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f1b9c15eb00 0x7f1b9c06b9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.105:49942/0 (socket says 172.21.15.105:49942) 2024-04-15T15:19:19.056 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.054+0000 7f1ba1545640 1 -- 172.21.15.105:0/1071766094 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b9c0cbb50 msgr2=0x7f1b9c0cbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:19.056 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.054+0000 7f1ba1545640 1 --2- 172.21.15.105:0/1071766094 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b9c0cbb50 0x7f1b9c0cbfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:19.056 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.054+0000 7f1ba1545640 1 -- 172.21.15.105:0/1071766094 >> v1:172.21.15.154:6789/0 conn(0x7f1b9c15eb00 legacy=0x7f1b9c06b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:19.057 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.054+0000 7f1ba1545640 1 -- 172.21.15.105:0/1071766094 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b9c0cdda0 con 0x7f1b9c0ce5a0 2024-04-15T15:19:19.057 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.054+0000 7f1ba2547640 1 --2- 172.21.15.105:0/1071766094 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b9c0cbb50 0x7f1b9c0cbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:19:19.057 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.054+0000 7f1ba1545640 1 --2- 172.21.15.105:0/1071766094 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b9c0ce5a0 0x7f1b9c0cea00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c009870 tx=0x7f1b8c02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd4a4af9dbdafd55 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:19.057 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.055+0000 7f1ba0d44640 1 -- 172.21.15.105:0/1071766094 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c034070 con 0x7f1b9c0ce5a0 2024-04-15T15:19:19.057 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.055+0000 7f1ba0d44640 1 -- 172.21.15.105:0/1071766094 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b8c030840 con 0x7f1b9c0ce5a0 2024-04-15T15:19:19.057 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.055+0000 7f1ba0d44640 1 -- 172.21.15.105:0/1071766094 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c030c10 con 0x7f1b9c0ce5a0 2024-04-15T15:19:19.058 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.055+0000 7f1ba3faa640 1 -- 172.21.15.105:0/1071766094 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b9c0ce5a0 msgr2=0x7f1b9c0cea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:19.058 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.055+0000 7f1ba3faa640 1 --2- 172.21.15.105:0/1071766094 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b9c0ce5a0 0x7f1b9c0cea00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c009870 tx=0x7f1b8c02b980 comp rx=0 tx=0).stop 2024-04-15T15:19:19.058 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.055+0000 7f1ba3faa640 1 -- 172.21.15.105:0/1071766094 shutdown_connections 2024-04-15T15:19:19.058 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.055+0000 7f1ba3faa640 1 --2- 172.21.15.105:0/1071766094 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b9c0cbb50 0x7f1b9c0cbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:19.058 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.055+0000 7f1ba3faa640 1 --2- 172.21.15.105:0/1071766094 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b9c0ce5a0 0x7f1b9c0cea00 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:19.058 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.055+0000 7f1ba3faa640 1 -- 172.21.15.105:0/1071766094 >> 172.21.15.105:0/1071766094 conn(0x7f1b9c09a9d0 msgr2=0x7f1b9c0d1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:19.058 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.055+0000 7f1ba3faa640 1 -- 172.21.15.105:0/1071766094 shutdown_connections 2024-04-15T15:19:19.058 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.055+0000 7f1ba3faa640 1 -- 172.21.15.105:0/1071766094 wait complete. 2024-04-15T15:19:19.061 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.056+0000 7f1ba3faa640 1 Processor -- start 2024-04-15T15:19:19.061 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.057+0000 7f1ba3faa640 1 -- start start 2024-04-15T15:19:19.061 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.057+0000 7f1ba3faa640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b9c0cbb50 0x7f1b9c152660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:19.061 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.057+0000 7f1ba3faa640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1b9c0ce5a0 0x7f1b9c0c8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:19.061 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.057+0000 7f1ba3faa640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b9c15eb00 0x7f1b9c0aeff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:19.061 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.057+0000 7f1ba3faa640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1b9c06c0f0 con 0x7f1b9c0ce5a0 2024-04-15T15:19:19.061 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.057+0000 7f1ba3faa640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1b9c0cf300 con 0x7f1b9c15eb00 2024-04-15T15:19:19.061 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.057+0000 7f1ba3faa640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f1b9c0b0f40 con 0x7f1b9c0cbb50 2024-04-15T15:19:19.061 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.057+0000 7f1ba1545640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1b9c0ce5a0 0x7f1b9c0c8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:19.061 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.057+0000 7f1ba1545640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1b9c0ce5a0 0x7f1b9c0c8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39074/0 (socket says 172.21.15.105:39074) 2024-04-15T15:19:19.061 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.057+0000 7f1ba1545640 1 -- 172.21.15.105:0/517436906 learned_addr learned my addr 172.21.15.105:0/517436906 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:19.061 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.058+0000 7f1ba1d46640 1 --2- 172.21.15.105:0/517436906 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b9c0cbb50 0x7f1b9c152660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:19.062 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.058+0000 7f1ba2547640 1 --2- 172.21.15.105:0/517436906 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b9c15eb00 0x7f1b9c0aeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:19.062 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.058+0000 7f1ba1545640 1 -- 172.21.15.105:0/517436906 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b9c0cbb50 msgr2=0x7f1b9c152660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:19.062 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.058+0000 7f1ba1545640 1 --2- 172.21.15.105:0/517436906 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b9c0cbb50 0x7f1b9c152660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:19.062 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.058+0000 7f1ba1545640 1 -- 172.21.15.105:0/517436906 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b9c15eb00 msgr2=0x7f1b9c0aeff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:19.062 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.058+0000 7f1ba1545640 1 --2- 172.21.15.105:0/517436906 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b9c15eb00 0x7f1b9c0aeff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:19.062 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.058+0000 7f1ba1545640 1 -- 172.21.15.105:0/517436906 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b9c0ae230 con 0x7f1b9c0ce5a0 2024-04-15T15:19:19.062 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.058+0000 7f1ba1d46640 1 --2- 172.21.15.105:0/517436906 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b9c0cbb50 0x7f1b9c152660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:19:19.062 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.058+0000 7f1ba2547640 1 --2- 172.21.15.105:0/517436906 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b9c15eb00 0x7f1b9c0aeff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:19:19.062 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.058+0000 7f1ba1545640 1 --2- 172.21.15.105:0/517436906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1b9c0ce5a0 0x7f1b9c0c8870 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c000c00 tx=0x7f1b8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:19.062 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.058+0000 7f1b927fc640 1 -- 172.21.15.105:0/517436906 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c034040 con 0x7f1b9c0ce5a0 2024-04-15T15:19:19.062 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.058+0000 7f1b927fc640 1 -- 172.21.15.105:0/517436906 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b8c039d90 con 0x7f1b9c0ce5a0 2024-04-15T15:19:19.062 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.058+0000 7f1ba3faa640 1 -- 172.21.15.105:0/517436906 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b9c0ae510 con 0x7f1b9c0ce5a0 2024-04-15T15:19:19.062 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.058+0000 7f1b927fc640 1 -- 172.21.15.105:0/517436906 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c043630 con 0x7f1b9c0ce5a0 2024-04-15T15:19:19.062 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.058+0000 7f1ba3faa640 1 -- 172.21.15.105:0/517436906 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b9c0aead0 con 0x7f1b9c0ce5a0 2024-04-15T15:19:19.063 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.059+0000 7f1ba3faa640 1 -- 172.21.15.105:0/517436906 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b9c04fd50 con 0x7f1b9c0ce5a0 2024-04-15T15:19:19.063 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.060+0000 7f1b927fc640 1 -- 172.21.15.105:0/517436906 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1b8c04a060 con 0x7f1b9c0ce5a0 2024-04-15T15:19:19.063 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.060+0000 7f1b927fc640 1 --2- 172.21.15.105:0/517436906 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f1b8006c8f0 0x7f1b8006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:19.063 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.060+0000 7f1ba1d46640 1 --2- 172.21.15.105:0/517436906 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f1b8006c8f0 0x7f1b8006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:19.064 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.060+0000 7f1b927fc640 1 -- 172.21.15.105:0/517436906 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f1b8c0ad6d0 con 0x7f1b9c0ce5a0 2024-04-15T15:19:19.064 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.061+0000 7f1ba1d46640 1 --2- 172.21.15.105:0/517436906 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f1b8006c8f0 0x7f1b8006edb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1b98007fd0 tx=0x7f1b98007d60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:19.066 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.065+0000 7f1b927fc640 1 -- 172.21.15.105:0/517436906 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1b8c03e070 con 0x7f1b9c0ce5a0 2024-04-15T15:19:19.241 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.239+0000 7f1ba3faa640 1 -- 172.21.15.105:0/517436906 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1b9c0aeca0 con 0x7f1b9c0ce5a0 2024-04-15T15:19:19.242 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.240+0000 7f1b927fc640 1 -- 172.21.15.105:0/517436906 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12932 (secure 0 0 0) 0x7f1b8c07bee0 con 0x7f1b9c0ce5a0 2024-04-15T15:19:19.242 INFO:teuthology.misc.health.smithi105.stdout: 2024-04-15T15:19:19.242 INFO:teuthology.misc.health.smithi105.stdout:{"epoch":16,"fsid":"09e4d2a4-a324-4793-af77-644b25e520e1","created":"2024-04-15T15:19:01.524543+0000","modified":"2024-04-15T15:19:15.830502+0000","last_up_change":"2024-04-15T15:19:12.825815+0000","last_in_change":"2024-04-15T15:19:03.788126+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:19:13.748585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6f6292b-2d7e-4aa0-9d1c-6ddd0f97287c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6801","nonce":2873040054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6803","nonce":2873040054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6807","nonce":2873040054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6805","nonce":2873040054}]},"public_addr":"172.21.15.105:6801/2873040054","cluster_addr":"172.21.15.105:6803/2873040054","heartbeat_back_addr":"172.21.15.105:6807/2873040054","heartbeat_front_addr":"172.21.15.105:6805/2873040054","state":["exists","up"]},{"osd":1,"uuid":"8900966a-b393-41e3-8f93-e5a43bd23622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6818","nonce":2494994479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6821","nonce":2494994479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6828","nonce":2494994479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6823","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6825","nonce":2494994479}]},"public_addr":"172.21.15.105:6818/2494994479","cluster_addr":"172.21.15.105:6821/2494994479","heartbeat_back_addr":"172.21.15.105:6828/2494994479","heartbeat_front_addr":"172.21.15.105:6825/2494994479","state":["exists","up"]},{"osd":2,"uuid":"81a5dc7a-9325-4ff5-941d-a2d5fba21f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6809","nonce":3798408699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6811","nonce":3798408699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6815","nonce":3798408699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6813","nonce":3798408699}]},"public_addr":"172.21.15.105:6809/3798408699","cluster_addr":"172.21.15.105:6811/3798408699","heartbeat_back_addr":"172.21.15.105:6815/3798408699","heartbeat_front_addr":"172.21.15.105:6813/3798408699","state":["exists","up"]},{"osd":3,"uuid":"df766ed2-5c71-4805-96f4-3d4970edc960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6819","nonce":1920720186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6824","nonce":1920720186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6831","nonce":1920720186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6827","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6829","nonce":1920720186}]},"public_addr":"172.21.15.105:6819/1920720186","cluster_addr":"172.21.15.105:6824/1920720186","heartbeat_back_addr":"172.21.15.105:6831/1920720186","heartbeat_front_addr":"172.21.15.105:6829/1920720186","state":["exists","up"]},{"osd":4,"uuid":"e1039e03-28b8-49df-b951-c91930b93c9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6810","nonce":3516494456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6813","nonce":3516494456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6820","nonce":3516494456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6817","nonce":3516494456}]},"public_addr":"172.21.15.154:6810/3516494456","cluster_addr":"172.21.15.154:6813/3516494456","heartbeat_back_addr":"172.21.15.154:6820/3516494456","heartbeat_front_addr":"172.21.15.154:6817/3516494456","state":["exists","up"]},{"osd":5,"uuid":"a9f5eaf1-66ff-405a-a9e4-ec3340812a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6801","nonce":540367751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6803","nonce":540367751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6807","nonce":540367751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6805","nonce":540367751}]},"public_addr":"172.21.15.154:6801/540367751","cluster_addr":"172.21.15.154:6803/540367751","heartbeat_back_addr":"172.21.15.154:6807/540367751","heartbeat_front_addr":"172.21.15.154:6805/540367751","state":["exists","up"]},{"osd":6,"uuid":"267d02ea-675f-428f-aae8-e0375e4d4feb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6825","nonce":1550829005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6827","nonce":1550829005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6831","nonce":1550829005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6829","nonce":1550829005}]},"public_addr":"172.21.15.154:6825/1550829005","cluster_addr":"172.21.15.154:6827/1550829005","heartbeat_back_addr":"172.21.15.154:6831/1550829005","heartbeat_front_addr":"172.21.15.154:6829/1550829005","state":["exists","up"]},{"osd":7,"uuid":"759ac6cc-896a-423c-885f-9fe90d8dcb14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6809","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6811","nonce":3920089903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6816","nonce":3920089903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6823","nonce":3920089903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6819","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6821","nonce":3920089903}]},"public_addr":"172.21.15.154:6811/3920089903","cluster_addr":"172.21.15.154:6816/3920089903","heartbeat_back_addr":"172.21.15.154:6823/3920089903","heartbeat_front_addr":"172.21.15.154:6821/3920089903","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.949509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.984072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.907278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.947391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.975447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.998896+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:09.028739+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.894119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:19:19.244 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.242+0000 7f1ba3faa640 1 -- 172.21.15.105:0/517436906 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f1b8006c8f0 msgr2=0x7f1b8006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:19.244 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.242+0000 7f1ba3faa640 1 --2- 172.21.15.105:0/517436906 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f1b8006c8f0 0x7f1b8006edb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1b98007fd0 tx=0x7f1b98007d60 comp rx=0 tx=0).stop 2024-04-15T15:19:19.244 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.242+0000 7f1ba3faa640 1 -- 172.21.15.105:0/517436906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1b9c0ce5a0 msgr2=0x7f1b9c0c8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:19.244 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.242+0000 7f1ba3faa640 1 --2- 172.21.15.105:0/517436906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1b9c0ce5a0 0x7f1b9c0c8870 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c000c00 tx=0x7f1b8c0046c0 comp rx=0 tx=0).stop 2024-04-15T15:19:19.244 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.242+0000 7f1ba3faa640 1 -- 172.21.15.105:0/517436906 shutdown_connections 2024-04-15T15:19:19.245 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.242+0000 7f1ba3faa640 1 --2- 172.21.15.105:0/517436906 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f1b8006c8f0 0x7f1b8006edb0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:19.245 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.242+0000 7f1ba3faa640 1 --2- 172.21.15.105:0/517436906 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b9c15eb00 0x7f1b9c0aeff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:19.245 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.242+0000 7f1ba3faa640 1 --2- 172.21.15.105:0/517436906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1b9c0ce5a0 0x7f1b9c0c8870 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:19.245 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.242+0000 7f1ba3faa640 1 --2- 172.21.15.105:0/517436906 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b9c0cbb50 0x7f1b9c152660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:19.245 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.242+0000 7f1ba3faa640 1 -- 172.21.15.105:0/517436906 >> 172.21.15.105:0/517436906 conn(0x7f1b9c09a9d0 msgr2=0x7f1b9c15dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:19.245 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.242+0000 7f1ba3faa640 1 -- 172.21.15.105:0/517436906 shutdown_connections 2024-04-15T15:19:19.245 INFO:teuthology.misc.health.smithi105.stderr:2024-04-15T15:19:19.242+0000 7f1ba3faa640 1 -- 172.21.15.105:0/517436906 wait complete. 2024-04-15T15:19:19.254 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-15T15:19:19.254 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-15T15:19:19.254 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-15T15:19:19.254 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-15T15:19:19.257 INFO:tasks.ceph.mds.a:Started 2024-04-15T15:19:19.257 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-15T15:19:19.257 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-15T15:19:19.259 INFO:tasks.ceph.mds.c:Started 2024-04-15T15:19:19.259 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-15T15:19:19.259 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-15T15:19:19.262 INFO:tasks.ceph.mds.b:Started 2024-04-15T15:19:19.262 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-15T15:19:19.262 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-15T15:19:19.264 INFO:tasks.ceph.mds.d:Started 2024-04-15T15:19:19.265 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-15T15:19:19.265 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:19:19.336 INFO:tasks.ceph.mds.a.smithi105.stdout:starting mds.a at 2024-04-15T15:19:19.339 INFO:tasks.ceph.mds.c.smithi105.stdout:starting mds.c at 2024-04-15T15:19:19.350 INFO:tasks.ceph.mds.b.smithi154.stdout:starting mds.b at 2024-04-15T15:19:19.350 INFO:tasks.ceph.mds.d.smithi154.stdout:starting mds.d at 2024-04-15T15:19:19.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.370+0000 7f369e4a0640 1 Processor -- start 2024-04-15T15:19:19.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.370+0000 7f369e4a0640 1 -- start start 2024-04-15T15:19:19.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.371+0000 7f369e4a0640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f36980d18e0 0x7f3698162630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:19.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.371+0000 7f369e4a0640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3698162b70 0x7f36980d3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:19.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.371+0000 7f369e4a0640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f369815ef40 con 0x7f369815eb60 2024-04-15T15:19:19.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.371+0000 7f369e4a0640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f3698163540 con 0x7f36980d18e0 2024-04-15T15:19:19.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.371+0000 7f369e4a0640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f36980d2fc0 con 0x7f3698162b70 2024-04-15T15:19:19.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.371+0000 7f36977fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f36980d18e0 0x7f3698162630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:19.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.371+0000 7f36977fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f36980d18e0 0x7f3698162630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39134/0 (socket says 172.21.15.105:39134) 2024-04-15T15:19:19.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.371+0000 7f36977fe640 1 -- 172.21.15.105:0/3561399416 learned_addr learned my addr 172.21.15.105:0/3561399416 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:19.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.371+0000 7f369ca3d640 1 --2- 172.21.15.105:0/3561399416 >> v2:172.21.15.154:3300/0 conn(0x7f3698162b70 0x7f36980d3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:19.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.372+0000 7f36977fe640 1 -- 172.21.15.105:0/3561399416 >> v2:172.21.15.154:3300/0 conn(0x7f3698162b70 msgr2=0x7f36980d3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:19.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.372+0000 7f36977fe640 1 --2- 172.21.15.105:0/3561399416 >> v2:172.21.15.154:3300/0 conn(0x7f3698162b70 0x7f36980d3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:19.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.372+0000 7f36977fe640 1 -- 172.21.15.105:0/3561399416 >> v1:172.21.15.154:6789/0 conn(0x7f369815eb60 legacy=0x7f36980d11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:19.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.372+0000 7f36977fe640 1 -- 172.21.15.105:0/3561399416 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3698163260 con 0x7f36980d18e0 2024-04-15T15:19:19.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.372+0000 7f36977fe640 1 --2- 172.21.15.105:0/3561399416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36980d18e0 0x7f3698162630 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f36980d1eb0 tx=0x7f368c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1c68a551c30d2d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:19.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.373+0000 7f3696ffd640 1 -- 172.21.15.105:0/3561399416 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368c034070 con 0x7f36980d18e0 2024-04-15T15:19:19.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.373+0000 7f3696ffd640 1 -- 172.21.15.105:0/3561399416 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f368c030860 con 0x7f36980d18e0 2024-04-15T15:19:19.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.373+0000 7f3696ffd640 1 -- 172.21.15.105:0/3561399416 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368c030c30 con 0x7f36980d18e0 2024-04-15T15:19:19.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.373+0000 7f369e4a0640 1 -- 172.21.15.105:0/3561399416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36980d18e0 msgr2=0x7f3698162630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:19.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.373+0000 7f369e4a0640 1 --2- 172.21.15.105:0/3561399416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36980d18e0 0x7f3698162630 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f36980d1eb0 tx=0x7f368c02b9a0 comp rx=0 tx=0).stop 2024-04-15T15:19:19.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.373+0000 7f369e4a0640 1 -- 172.21.15.105:0/3561399416 shutdown_connections 2024-04-15T15:19:19.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.373+0000 7f369e4a0640 1 --2- 172.21.15.105:0/3561399416 >> v2:172.21.15.154:3300/0 conn(0x7f3698162b70 0x7f36980d3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:19.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.373+0000 7f369e4a0640 1 --2- 172.21.15.105:0/3561399416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36980d18e0 0x7f3698162630 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:19.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.373+0000 7f369e4a0640 1 -- 172.21.15.105:0/3561399416 >> 172.21.15.105:0/3561399416 conn(0x7f3698067720 msgr2=0x7f3698067b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:19.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.373+0000 7f369e4a0640 1 -- 172.21.15.105:0/3561399416 shutdown_connections 2024-04-15T15:19:19.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.373+0000 7f369e4a0640 1 -- 172.21.15.105:0/3561399416 wait complete. 2024-04-15T15:19:19.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.374+0000 7f369e4a0640 1 Processor -- start 2024-04-15T15:19:19.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.374+0000 7f369e4a0640 1 -- start start 2024-04-15T15:19:19.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.375+0000 7f369e4a0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36980d18e0 0x7f369814fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:19.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.375+0000 7f369e4a0640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f369815eb60 0x7f36981f4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:19.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.375+0000 7f369e4a0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3698162b70 0x7f36980b58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:19.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.375+0000 7f369e4a0640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3698162f80 con 0x7f3698162b70 2024-04-15T15:19:19.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.375+0000 7f369e4a0640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f36980d2c50 con 0x7f36980d18e0 2024-04-15T15:19:19.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.375+0000 7f369e4a0640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f36980b78f0 con 0x7f369815eb60 2024-04-15T15:19:19.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.375+0000 7f369ca3d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3698162b70 0x7f36980b58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:19.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.375+0000 7f369ca3d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3698162b70 0x7f36980b58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39146/0 (socket says 172.21.15.105:39146) 2024-04-15T15:19:19.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.375+0000 7f369ca3d640 1 -- 172.21.15.105:0/2366952226 learned_addr learned my addr 172.21.15.105:0/2366952226 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:19.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.375+0000 7f3697fff640 1 --2- 172.21.15.105:0/2366952226 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36980d18e0 0x7f369814fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:19.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.375+0000 7f36977fe640 1 --2- 172.21.15.105:0/2366952226 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f369815eb60 0x7f36981f4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:19.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.376+0000 7f369ca3d640 1 -- 172.21.15.105:0/2366952226 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f369815eb60 msgr2=0x7f36981f4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:19.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.376+0000 7f369ca3d640 1 --2- 172.21.15.105:0/2366952226 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f369815eb60 0x7f36981f4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:19.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.376+0000 7f369ca3d640 1 -- 172.21.15.105:0/2366952226 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36980d18e0 msgr2=0x7f369814fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:19.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.376+0000 7f369ca3d640 1 --2- 172.21.15.105:0/2366952226 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36980d18e0 0x7f369814fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:19.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.376+0000 7f369ca3d640 1 -- 172.21.15.105:0/2366952226 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36980b41c0 con 0x7f3698162b70 2024-04-15T15:19:19.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.376+0000 7f369ca3d640 1 --2- 172.21.15.105:0/2366952226 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3698162b70 0x7f36980b58e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f369000efc0 tx=0x7f369000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:19.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.376+0000 7f3694ff9640 1 -- 172.21.15.105:0/2366952226 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3690018020 con 0x7f3698162b70 2024-04-15T15:19:19.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.376+0000 7f3694ff9640 1 -- 172.21.15.105:0/2366952226 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3690014480 con 0x7f3698162b70 2024-04-15T15:19:19.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.376+0000 7f369e4a0640 1 -- 172.21.15.105:0/2366952226 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36980b44a0 con 0x7f3698162b70 2024-04-15T15:19:19.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.376+0000 7f3694ff9640 1 -- 172.21.15.105:0/2366952226 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3690014870 con 0x7f3698162b70 2024-04-15T15:19:19.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.376+0000 7f369e4a0640 1 -- 172.21.15.105:0/2366952226 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36980b4a60 con 0x7f3698162b70 2024-04-15T15:19:19.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.377+0000 7f369e4a0640 1 -- 172.21.15.105:0/2366952226 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3698068a40 con 0x7f3698162b70 2024-04-15T15:19:19.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.377+0000 7f3694ff9640 1 -- 172.21.15.105:0/2366952226 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3690005d10 con 0x7f3698162b70 2024-04-15T15:19:19.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.378+0000 7f3694ff9640 1 --2- 172.21.15.105:0/2366952226 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f368406c8f0 0x7f368406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:19.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.378+0000 7f3694ff9640 1 -- 172.21.15.105:0/2366952226 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f369008c150 con 0x7f3698162b70 2024-04-15T15:19:19.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.378+0000 7f3697fff640 1 --2- 172.21.15.105:0/2366952226 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f368406c8f0 0x7f368406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:19.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.378+0000 7f3697fff640 1 --2- 172.21.15.105:0/2366952226 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f368406c8f0 0x7f368406edb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f369815df00 tx=0x7f3688009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:19.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.382+0000 7f3694ff9640 1 -- 172.21.15.105:0/2366952226 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3690022070 con 0x7f3698162b70 2024-04-15T15:19:19.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.559+0000 7f369e4a0640 1 -- 172.21.15.105:0/2366952226 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f36980d23f0 con 0x7f3698162b70 2024-04-15T15:19:19.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.560+0000 7f3694ff9640 1 -- 172.21.15.105:0/2366952226 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12932 (secure 0 0 0) 0x7f3690010040 con 0x7f3698162b70 2024-04-15T15:19:19.561 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:19:19.562 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":16,"fsid":"09e4d2a4-a324-4793-af77-644b25e520e1","created":"2024-04-15T15:19:01.524543+0000","modified":"2024-04-15T15:19:15.830502+0000","last_up_change":"2024-04-15T15:19:12.825815+0000","last_in_change":"2024-04-15T15:19:03.788126+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:19:13.748585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6f6292b-2d7e-4aa0-9d1c-6ddd0f97287c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6801","nonce":2873040054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6803","nonce":2873040054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6807","nonce":2873040054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6805","nonce":2873040054}]},"public_addr":"172.21.15.105:6801/2873040054","cluster_addr":"172.21.15.105:6803/2873040054","heartbeat_back_addr":"172.21.15.105:6807/2873040054","heartbeat_front_addr":"172.21.15.105:6805/2873040054","state":["exists","up"]},{"osd":1,"uuid":"8900966a-b393-41e3-8f93-e5a43bd23622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6818","nonce":2494994479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6821","nonce":2494994479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6828","nonce":2494994479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6823","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6825","nonce":2494994479}]},"public_addr":"172.21.15.105:6818/2494994479","cluster_addr":"172.21.15.105:6821/2494994479","heartbeat_back_addr":"172.21.15.105:6828/2494994479","heartbeat_front_addr":"172.21.15.105:6825/2494994479","state":["exists","up"]},{"osd":2,"uuid":"81a5dc7a-9325-4ff5-941d-a2d5fba21f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6809","nonce":3798408699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6811","nonce":3798408699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6815","nonce":3798408699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6813","nonce":3798408699}]},"public_addr":"172.21.15.105:6809/3798408699","cluster_addr":"172.21.15.105:6811/3798408699","heartbeat_back_addr":"172.21.15.105:6815/3798408699","heartbeat_front_addr":"172.21.15.105:6813/3798408699","state":["exists","up"]},{"osd":3,"uuid":"df766ed2-5c71-4805-96f4-3d4970edc960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6819","nonce":1920720186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6824","nonce":1920720186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6831","nonce":1920720186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6827","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6829","nonce":1920720186}]},"public_addr":"172.21.15.105:6819/1920720186","cluster_addr":"172.21.15.105:6824/1920720186","heartbeat_back_addr":"172.21.15.105:6831/1920720186","heartbeat_front_addr":"172.21.15.105:6829/1920720186","state":["exists","up"]},{"osd":4,"uuid":"e1039e03-28b8-49df-b951-c91930b93c9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6810","nonce":3516494456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6813","nonce":3516494456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6820","nonce":3516494456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6817","nonce":3516494456}]},"public_addr":"172.21.15.154:6810/3516494456","cluster_addr":"172.21.15.154:6813/3516494456","heartbeat_back_addr":"172.21.15.154:6820/3516494456","heartbeat_front_addr":"172.21.15.154:6817/3516494456","state":["exists","up"]},{"osd":5,"uuid":"a9f5eaf1-66ff-405a-a9e4-ec3340812a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6801","nonce":540367751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6803","nonce":540367751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6807","nonce":540367751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6805","nonce":540367751}]},"public_addr":"172.21.15.154:6801/540367751","cluster_addr":"172.21.15.154:6803/540367751","heartbeat_back_addr":"172.21.15.154:6807/540367751","heartbeat_front_addr":"172.21.15.154:6805/540367751","state":["exists","up"]},{"osd":6,"uuid":"267d02ea-675f-428f-aae8-e0375e4d4feb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6825","nonce":1550829005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6827","nonce":1550829005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6831","nonce":1550829005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6829","nonce":1550829005}]},"public_addr":"172.21.15.154:6825/1550829005","cluster_addr":"172.21.15.154:6827/1550829005","heartbeat_back_addr":"172.21.15.154:6831/1550829005","heartbeat_front_addr":"172.21.15.154:6829/1550829005","state":["exists","up"]},{"osd":7,"uuid":"759ac6cc-896a-423c-885f-9fe90d8dcb14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6809","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6811","nonce":3920089903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6816","nonce":3920089903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6823","nonce":3920089903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6819","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6821","nonce":3920089903}]},"public_addr":"172.21.15.154:6811/3920089903","cluster_addr":"172.21.15.154:6816/3920089903","heartbeat_back_addr":"172.21.15.154:6823/3920089903","heartbeat_front_addr":"172.21.15.154:6821/3920089903","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.949509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.984072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.907278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.947391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.975447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.998896+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:09.028739+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.894119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:19:19.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.561+0000 7f369e4a0640 1 -- 172.21.15.105:0/2366952226 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f368406c8f0 msgr2=0x7f368406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:19.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.562+0000 7f369e4a0640 1 --2- 172.21.15.105:0/2366952226 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f368406c8f0 0x7f368406edb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f369815df00 tx=0x7f3688009040 comp rx=0 tx=0).stop 2024-04-15T15:19:19.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.562+0000 7f369e4a0640 1 -- 172.21.15.105:0/2366952226 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3698162b70 msgr2=0x7f36980b58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:19.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.562+0000 7f369e4a0640 1 --2- 172.21.15.105:0/2366952226 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3698162b70 0x7f36980b58e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f369000efc0 tx=0x7f369000c490 comp rx=0 tx=0).stop 2024-04-15T15:19:19.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.562+0000 7f369e4a0640 1 -- 172.21.15.105:0/2366952226 shutdown_connections 2024-04-15T15:19:19.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.562+0000 7f369e4a0640 1 --2- 172.21.15.105:0/2366952226 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f369815eb60 0x7f36981f4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:19.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.562+0000 7f369e4a0640 1 --2- 172.21.15.105:0/2366952226 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f368406c8f0 0x7f368406edb0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:19.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.562+0000 7f369e4a0640 1 --2- 172.21.15.105:0/2366952226 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3698162b70 0x7f36980b58e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:19.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.562+0000 7f369e4a0640 1 --2- 172.21.15.105:0/2366952226 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36980d18e0 0x7f369814fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:19.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.562+0000 7f369e4a0640 1 -- 172.21.15.105:0/2366952226 >> 172.21.15.105:0/2366952226 conn(0x7f3698067720 msgr2=0x7f36980cfa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:19.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.562+0000 7f369e4a0640 1 -- 172.21.15.105:0/2366952226 shutdown_connections 2024-04-15T15:19:19.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.562+0000 7f369e4a0640 1 -- 172.21.15.105:0/2366952226 wait complete. 2024-04-15T15:19:19.574 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:19:13.748585+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:19:19.575 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:19:19.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.679+0000 7fbf687cd640 1 Processor -- start 2024-04-15T15:19:19.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.679+0000 7fbf687cd640 1 -- start start 2024-04-15T15:19:19.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.680+0000 7fbf687cd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbf60072570 0x7fbf6006de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:19.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.680+0000 7fbf687cd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbf6006e3c0 0x7fbf6015d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:19.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.680+0000 7fbf687cd640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf6006ebb0 con 0x7fbf60145f00 2024-04-15T15:19:19.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.680+0000 7fbf687cd640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fbf601462e0 con 0x7fbf60072570 2024-04-15T15:19:19.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.680+0000 7fbf687cd640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fbf6006ee90 con 0x7fbf6006e3c0 2024-04-15T15:19:19.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.680+0000 7fbf65d68640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbf60072570 0x7fbf6006de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:19.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.680+0000 7fbf65d68640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbf60072570 0x7fbf6006de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39150/0 (socket says 172.21.15.105:39150) 2024-04-15T15:19:19.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.680+0000 7fbf65d68640 1 -- 172.21.15.105:0/1972746548 learned_addr learned my addr 172.21.15.105:0/1972746548 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:19.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.680+0000 7fbf66d6a640 1 --2- 172.21.15.105:0/1972746548 >> v2:172.21.15.154:3300/0 conn(0x7fbf6006e3c0 0x7fbf6015d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:19.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.680+0000 7fbf65d68640 1 -- 172.21.15.105:0/1972746548 >> v2:172.21.15.154:3300/0 conn(0x7fbf6006e3c0 msgr2=0x7fbf6015d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:19.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.680+0000 7fbf65d68640 1 --2- 172.21.15.105:0/1972746548 >> v2:172.21.15.154:3300/0 conn(0x7fbf6006e3c0 0x7fbf6015d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:19.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.680+0000 7fbf65d68640 1 -- 172.21.15.105:0/1972746548 >> v1:172.21.15.154:6789/0 conn(0x7fbf60145f00 legacy=0x7fbf60071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:19.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.680+0000 7fbf65d68640 1 -- 172.21.15.105:0/1972746548 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf6015cde0 con 0x7fbf60072570 2024-04-15T15:19:19.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.681+0000 7fbf65d68640 1 --2- 172.21.15.105:0/1972746548 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf60072570 0x7fbf6006de80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbf600cff30 tx=0x7fbf5002b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3c5ccf82f13f3df server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:19.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.681+0000 7fbf65567640 1 -- 172.21.15.105:0/1972746548 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf50038070 con 0x7fbf60072570 2024-04-15T15:19:19.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.681+0000 7fbf65567640 1 -- 172.21.15.105:0/1972746548 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf5002f050 con 0x7fbf60072570 2024-04-15T15:19:19.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.681+0000 7fbf65567640 1 -- 172.21.15.105:0/1972746548 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf50031970 con 0x7fbf60072570 2024-04-15T15:19:19.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.682+0000 7fbf687cd640 1 -- 172.21.15.105:0/1972746548 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf60072570 msgr2=0x7fbf6006de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:19.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.682+0000 7fbf687cd640 1 --2- 172.21.15.105:0/1972746548 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf60072570 0x7fbf6006de80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbf600cff30 tx=0x7fbf5002b7f0 comp rx=0 tx=0).stop 2024-04-15T15:19:19.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.682+0000 7fbf687cd640 1 -- 172.21.15.105:0/1972746548 shutdown_connections 2024-04-15T15:19:19.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.682+0000 7fbf687cd640 1 --2- 172.21.15.105:0/1972746548 >> v2:172.21.15.154:3300/0 conn(0x7fbf6006e3c0 0x7fbf6015d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:19.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.682+0000 7fbf687cd640 1 --2- 172.21.15.105:0/1972746548 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf60072570 0x7fbf6006de80 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:19.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.682+0000 7fbf687cd640 1 -- 172.21.15.105:0/1972746548 >> 172.21.15.105:0/1972746548 conn(0x7fbf60067d50 msgr2=0x7fbf6006b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:19.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.682+0000 7fbf687cd640 1 -- 172.21.15.105:0/1972746548 shutdown_connections 2024-04-15T15:19:19.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.682+0000 7fbf687cd640 1 -- 172.21.15.105:0/1972746548 wait complete. 2024-04-15T15:19:19.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.683+0000 7fbf687cd640 1 Processor -- start 2024-04-15T15:19:19.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.683+0000 7fbf687cd640 1 -- start start 2024-04-15T15:19:19.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.684+0000 7fbf687cd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf6006e3c0 0x7fbf601f22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:19.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.684+0000 7fbf687cd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbf60072570 0x7fbf601f4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:19.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.684+0000 7fbf687cd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbf60145f00 0x7fbf600a95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:19.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.684+0000 7fbf687cd640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6015db20 con 0x7fbf6006e3c0 2024-04-15T15:19:19.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.684+0000 7fbf687cd640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6006e830 con 0x7fbf60072570 2024-04-15T15:19:19.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.684+0000 7fbf687cd640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fbf6015dca0 con 0x7fbf60145f00 2024-04-15T15:19:19.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.684+0000 7fbf66569640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf6006e3c0 0x7fbf601f22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:19.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.684+0000 7fbf66569640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf6006e3c0 0x7fbf601f22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39156/0 (socket says 172.21.15.105:39156) 2024-04-15T15:19:19.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.684+0000 7fbf66569640 1 -- 172.21.15.105:0/3600470554 learned_addr learned my addr 172.21.15.105:0/3600470554 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:19.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.684+0000 7fbf66569640 1 -- 172.21.15.105:0/3600470554 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbf60145f00 msgr2=0x7fbf600a95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:19.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.684+0000 7fbf66569640 1 --2- 172.21.15.105:0/3600470554 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbf60145f00 0x7fbf600a95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:19.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.684+0000 7fbf66569640 1 -- 172.21.15.105:0/3600470554 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbf60072570 msgr2=0x7fbf601f4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:19.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.684+0000 7fbf66569640 1 --2- 172.21.15.105:0/3600470554 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbf60072570 0x7fbf601f4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:19.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.684+0000 7fbf66569640 1 -- 172.21.15.105:0/3600470554 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf600ca2a0 con 0x7fbf6006e3c0 2024-04-15T15:19:19.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.685+0000 7fbf66569640 1 --2- 172.21.15.105:0/3600470554 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf6006e3c0 0x7fbf601f22c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c00b420 tx=0x7fbf5c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:19.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.685+0000 7fbf56ffd640 1 -- 172.21.15.105:0/3600470554 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c011830 con 0x7fbf6006e3c0 2024-04-15T15:19:19.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.685+0000 7fbf56ffd640 1 -- 172.21.15.105:0/3600470554 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf5c016070 con 0x7fbf6006e3c0 2024-04-15T15:19:19.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.685+0000 7fbf687cd640 1 -- 172.21.15.105:0/3600470554 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf600ca580 con 0x7fbf6006e3c0 2024-04-15T15:19:19.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.685+0000 7fbf56ffd640 1 -- 172.21.15.105:0/3600470554 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c011c10 con 0x7fbf6006e3c0 2024-04-15T15:19:19.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.685+0000 7fbf687cd640 1 -- 172.21.15.105:0/3600470554 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf600cab40 con 0x7fbf6006e3c0 2024-04-15T15:19:19.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.686+0000 7fbf687cd640 1 -- 172.21.15.105:0/3600470554 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf60068ab0 con 0x7fbf6006e3c0 2024-04-15T15:19:19.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.686+0000 7fbf56ffd640 1 -- 172.21.15.105:0/3600470554 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbf5c01b460 con 0x7fbf6006e3c0 2024-04-15T15:19:19.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.686+0000 7fbf56ffd640 1 --2- 172.21.15.105:0/3600470554 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fbf4406c8f0 0x7fbf4406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:19.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.687+0000 7fbf56ffd640 1 -- 172.21.15.105:0/3600470554 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fbf5c08b2b0 con 0x7fbf6006e3c0 2024-04-15T15:19:19.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.687+0000 7fbf65d68640 1 --2- 172.21.15.105:0/3600470554 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fbf4406c8f0 0x7fbf4406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:19.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.687+0000 7fbf65d68640 1 --2- 172.21.15.105:0/3600470554 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fbf4406c8f0 0x7fbf4406edb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbf50035000 tx=0x7fbf50030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:19.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.691+0000 7fbf56ffd640 1 -- 172.21.15.105:0/3600470554 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbf5c020020 con 0x7fbf6006e3c0 2024-04-15T15:19:19.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.867+0000 7fbf687cd640 1 -- 172.21.15.105:0/3600470554 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbf6006ce40 con 0x7fbf6006e3c0 2024-04-15T15:19:19.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.867+0000 7fbf56ffd640 1 -- 172.21.15.105:0/3600470554 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7fbf5c059ac0 con 0x7fbf6006e3c0 2024-04-15T15:19:19.869 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-04-15T15:19:19.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.869+0000 7fbf687cd640 1 -- 172.21.15.105:0/3600470554 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fbf4406c8f0 msgr2=0x7fbf4406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:19.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.869+0000 7fbf687cd640 1 --2- 172.21.15.105:0/3600470554 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fbf4406c8f0 0x7fbf4406edb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbf50035000 tx=0x7fbf50030000 comp rx=0 tx=0).stop 2024-04-15T15:19:19.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.869+0000 7fbf687cd640 1 -- 172.21.15.105:0/3600470554 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf6006e3c0 msgr2=0x7fbf601f22c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:19.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.869+0000 7fbf687cd640 1 --2- 172.21.15.105:0/3600470554 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf6006e3c0 0x7fbf601f22c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c00b420 tx=0x7fbf5c00b8e0 comp rx=0 tx=0).stop 2024-04-15T15:19:19.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.869+0000 7fbf687cd640 1 -- 172.21.15.105:0/3600470554 shutdown_connections 2024-04-15T15:19:19.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.869+0000 7fbf687cd640 1 --2- 172.21.15.105:0/3600470554 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbf60145f00 0x7fbf600a95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:19.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.869+0000 7fbf687cd640 1 --2- 172.21.15.105:0/3600470554 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fbf4406c8f0 0x7fbf4406edb0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:19.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.869+0000 7fbf687cd640 1 --2- 172.21.15.105:0/3600470554 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbf60072570 0x7fbf601f4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:19.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.869+0000 7fbf687cd640 1 --2- 172.21.15.105:0/3600470554 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf6006e3c0 0x7fbf601f22c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:19.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.869+0000 7fbf687cd640 1 -- 172.21.15.105:0/3600470554 >> 172.21.15.105:0/3600470554 conn(0x7fbf60067d50 msgr2=0x7fbf600cebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:19.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.869+0000 7fbf687cd640 1 -- 172.21.15.105:0/3600470554 shutdown_connections 2024-04-15T15:19:19.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.869+0000 7fbf687cd640 1 -- 172.21.15.105:0/3600470554 wait complete. 2024-04-15T15:19:19.881 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-15T15:19:19.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.983+0000 7fbadecc8640 1 Processor -- start 2024-04-15T15:19:19.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.983+0000 7fbadecc8640 1 -- start start 2024-04-15T15:19:19.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.983+0000 7fbadecc8640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbad80725f0 0x7fbad8072a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:19.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.983+0000 7fbadecc8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbad80b3dc0 0x7fbad80b4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:19.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.983+0000 7fbadecc8640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbad815efd0 con 0x7fbad815ebf0 2024-04-15T15:19:19.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.983+0000 7fbadecc8640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fbad8072f90 con 0x7fbad80725f0 2024-04-15T15:19:19.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.983+0000 7fbadecc8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fbad806baf0 con 0x7fbad80b3dc0 2024-04-15T15:19:19.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.984+0000 7fbad7fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbad80725f0 0x7fbad8072a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:19.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.984+0000 7fbadca64640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fbad815ebf0 0x7fbad80b58c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.105:49986/0 (socket says 172.21.15.105:49986) 2024-04-15T15:19:19.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.984+0000 7fbadca64640 1 -- 172.21.15.105:0/500930528 learned_addr learned my addr 172.21.15.105:0/500930528 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:19:19.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.984+0000 7fbadd265640 1 --2- 172.21.15.105:0/500930528 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbad80b3dc0 0x7fbad80b4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:19.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.984+0000 7fbad77fe640 1 -- 172.21.15.105:0/500930528 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3728019764 0 0) 0x7fbad815efd0 con 0x7fbad815ebf0 2024-04-15T15:19:19.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.984+0000 7fbad77fe640 1 -- 172.21.15.105:0/500930528 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbac4003520 con 0x7fbad815ebf0 2024-04-15T15:19:19.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.984+0000 7fbadd265640 1 -- 172.21.15.105:0/500930528 >> v2:172.21.15.154:3300/0 conn(0x7fbad80725f0 msgr2=0x7fbad8072a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:19.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.984+0000 7fbadd265640 1 --2- 172.21.15.105:0/500930528 >> v2:172.21.15.154:3300/0 conn(0x7fbad80725f0 0x7fbad8072a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:19.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.984+0000 7fbadd265640 1 -- 172.21.15.105:0/500930528 >> v1:172.21.15.154:6789/0 conn(0x7fbad815ebf0 legacy=0x7fbad80b58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:19.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.984+0000 7fbadd265640 1 -- 172.21.15.105:0/500930528 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbad806bd90 con 0x7fbad80b3dc0 2024-04-15T15:19:19.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.984+0000 7fbad7fff640 1 --2- 172.21.15.105:0/500930528 >> v2:172.21.15.154:3300/0 conn(0x7fbad80725f0 0x7fbad8072a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:19:19.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.985+0000 7fbadd265640 1 --2- 172.21.15.105:0/500930528 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbad80b3dc0 0x7fbad80b4270 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbac80097a0 tx=0x7fbac802b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d425a92a9893039 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:19.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.985+0000 7fbad77fe640 1 -- 172.21.15.105:0/500930528 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac80315a0 con 0x7fbad80b3dc0 2024-04-15T15:19:19.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.985+0000 7fbad77fe640 1 -- 172.21.15.105:0/500930528 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbac8037070 con 0x7fbad80b3dc0 2024-04-15T15:19:19.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.985+0000 7fbad77fe640 1 -- 172.21.15.105:0/500930528 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac8031940 con 0x7fbad80b3dc0 2024-04-15T15:19:19.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.986+0000 7fbadecc8640 1 -- 172.21.15.105:0/500930528 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbad80b3dc0 msgr2=0x7fbad80b4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:19.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.986+0000 7fbadecc8640 1 --2- 172.21.15.105:0/500930528 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbad80b3dc0 0x7fbad80b4270 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbac80097a0 tx=0x7fbac802b840 comp rx=0 tx=0).stop 2024-04-15T15:19:19.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.986+0000 7fbadecc8640 1 -- 172.21.15.105:0/500930528 shutdown_connections 2024-04-15T15:19:19.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.986+0000 7fbadecc8640 1 --2- 172.21.15.105:0/500930528 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbad80b3dc0 0x7fbad80b4270 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:19.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.986+0000 7fbadecc8640 1 --2- 172.21.15.105:0/500930528 >> v2:172.21.15.154:3300/0 conn(0x7fbad80725f0 0x7fbad8072a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:19.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.986+0000 7fbadecc8640 1 -- 172.21.15.105:0/500930528 >> 172.21.15.105:0/500930528 conn(0x7fbad80d0d00 msgr2=0x7fbad80d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:19.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.986+0000 7fbadecc8640 1 -- 172.21.15.105:0/500930528 shutdown_connections 2024-04-15T15:19:19.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.986+0000 7fbadecc8640 1 -- 172.21.15.105:0/500930528 wait complete. 2024-04-15T15:19:19.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.987+0000 7fbadecc8640 1 Processor -- start 2024-04-15T15:19:19.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.987+0000 7fbadecc8640 1 -- start start 2024-04-15T15:19:19.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.988+0000 7fbadecc8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbad80725f0 0x7fbad81508f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:19.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.988+0000 7fbadecc8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbad80b3dc0 0x7fbad80c5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:19.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.988+0000 7fbadecc8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbad815ebf0 0x7fbad80aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:19.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.988+0000 7fbadecc8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbad806c630 con 0x7fbad80725f0 2024-04-15T15:19:19.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.988+0000 7fbadecc8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbad806c350 con 0x7fbad80b3dc0 2024-04-15T15:19:19.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.988+0000 7fbadecc8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fbad8073870 con 0x7fbad815ebf0 2024-04-15T15:19:19.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.988+0000 7fbadca64640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbad80725f0 0x7fbad81508f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:19.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.988+0000 7fbadca64640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbad80725f0 0x7fbad81508f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39172/0 (socket says 172.21.15.105:39172) 2024-04-15T15:19:19.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.988+0000 7fbadca64640 1 -- 172.21.15.105:0/2590552935 learned_addr learned my addr 172.21.15.105:0/2590552935 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:19.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.988+0000 7fbadd265640 1 --2- 172.21.15.105:0/2590552935 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbad815ebf0 0x7fbad80aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:19.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.988+0000 7fbad7fff640 1 --2- 172.21.15.105:0/2590552935 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbad80b3dc0 0x7fbad80c5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:19.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.988+0000 7fbadca64640 1 -- 172.21.15.105:0/2590552935 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbad815ebf0 msgr2=0x7fbad80aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:19.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.988+0000 7fbadca64640 1 --2- 172.21.15.105:0/2590552935 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbad815ebf0 0x7fbad80aedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:19.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.988+0000 7fbadca64640 1 -- 172.21.15.105:0/2590552935 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbad80b3dc0 msgr2=0x7fbad80c5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:19.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.988+0000 7fbadca64640 1 --2- 172.21.15.105:0/2590552935 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbad80b3dc0 0x7fbad80c5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:19.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.988+0000 7fbadca64640 1 -- 172.21.15.105:0/2590552935 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbad80c61e0 con 0x7fbad80725f0 2024-04-15T15:19:19.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.988+0000 7fbadca64640 1 --2- 172.21.15.105:0/2590552935 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbad80725f0 0x7fbad81508f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbad000c460 tx=0x7fbad000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:19.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.989+0000 7fbad57fa640 1 -- 172.21.15.105:0/2590552935 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad0017070 con 0x7fbad80725f0 2024-04-15T15:19:19.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.989+0000 7fbad57fa640 1 -- 172.21.15.105:0/2590552935 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbad0005ce0 con 0x7fbad80725f0 2024-04-15T15:19:19.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.989+0000 7fbadecc8640 1 -- 172.21.15.105:0/2590552935 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbad81f8180 con 0x7fbad80725f0 2024-04-15T15:19:19.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.989+0000 7fbad57fa640 1 -- 172.21.15.105:0/2590552935 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad0007290 con 0x7fbad80725f0 2024-04-15T15:19:19.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.989+0000 7fbadecc8640 1 -- 172.21.15.105:0/2590552935 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbad81f8740 con 0x7fbad80725f0 2024-04-15T15:19:19.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.990+0000 7fbadecc8640 1 -- 172.21.15.105:0/2590552935 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbad8068a40 con 0x7fbad80725f0 2024-04-15T15:19:19.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.990+0000 7fbad57fa640 1 -- 172.21.15.105:0/2590552935 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbad001a070 con 0x7fbad80725f0 2024-04-15T15:19:19.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.991+0000 7fbad57fa640 1 --2- 172.21.15.105:0/2590552935 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fbab806c920 0x7fbab806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:19.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.991+0000 7fbad57fa640 1 -- 172.21.15.105:0/2590552935 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fbad008abe0 con 0x7fbad80725f0 2024-04-15T15:19:19.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.991+0000 7fbad7fff640 1 --2- 172.21.15.105:0/2590552935 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fbab806c920 0x7fbab806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:19.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.991+0000 7fbad7fff640 1 --2- 172.21.15.105:0/2590552935 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fbab806c920 0x7fbab806ede0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbad81f7d00 tx=0x7fbac000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:19.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:19.995+0000 7fbad57fa640 1 -- 172.21.15.105:0/2590552935 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbad00592c0 con 0x7fbad80725f0 2024-04-15T15:19:20.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:20.205+0000 7fbadecc8640 1 -- 172.21.15.105:0/2590552935 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbad80731f0 con 0x7fbad80725f0 2024-04-15T15:19:20.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:20.205+0000 7fbad57fa640 1 -- 172.21.15.105:0/2590552935 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1395 (secure 0 0 0) 0x7fbad0058bc0 con 0x7fbad80725f0 2024-04-15T15:19:20.207 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:19:20.207 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2677199890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2677199890},{"type":"v1","addr":"172.21.15.105:6835","nonce":2677199890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-15T15:19:20.208 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 2 2024-04-15T15:19:20.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:20.207+0000 7fbadecc8640 1 -- 172.21.15.105:0/2590552935 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fbab806c920 msgr2=0x7fbab806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:20.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:20.207+0000 7fbadecc8640 1 --2- 172.21.15.105:0/2590552935 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fbab806c920 0x7fbab806ede0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbad81f7d00 tx=0x7fbac000b040 comp rx=0 tx=0).stop 2024-04-15T15:19:20.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:20.207+0000 7fbadecc8640 1 -- 172.21.15.105:0/2590552935 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbad80725f0 msgr2=0x7fbad81508f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:20.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:20.207+0000 7fbadecc8640 1 --2- 172.21.15.105:0/2590552935 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbad80725f0 0x7fbad81508f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbad000c460 tx=0x7fbad000c920 comp rx=0 tx=0).stop 2024-04-15T15:19:20.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:20.207+0000 7fbadecc8640 1 -- 172.21.15.105:0/2590552935 shutdown_connections 2024-04-15T15:19:20.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:20.207+0000 7fbadecc8640 1 --2- 172.21.15.105:0/2590552935 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbad815ebf0 0x7fbad80aedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:20.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:20.207+0000 7fbadecc8640 1 --2- 172.21.15.105:0/2590552935 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fbab806c920 0x7fbab806ede0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:20.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:20.207+0000 7fbadecc8640 1 --2- 172.21.15.105:0/2590552935 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbad80b3dc0 0x7fbad80c5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:20.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:20.207+0000 7fbadecc8640 1 --2- 172.21.15.105:0/2590552935 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbad80725f0 0x7fbad81508f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:20.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:20.207+0000 7fbadecc8640 1 -- 172.21.15.105:0/2590552935 >> 172.21.15.105:0/2590552935 conn(0x7fbad80d0d00 msgr2=0x7fbad80d0020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:20.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:20.207+0000 7fbadecc8640 1 -- 172.21.15.105:0/2590552935 shutdown_connections 2024-04-15T15:19:20.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:20.207+0000 7fbadecc8640 1 -- 172.21.15.105:0/2590552935 wait complete. 2024-04-15T15:19:22.220 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-15T15:19:22.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.333+0000 7f580fe93640 1 Processor -- start 2024-04-15T15:19:22.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.333+0000 7f580fe93640 1 -- start start 2024-04-15T15:19:22.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.334+0000 7f580fe93640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f580809b150 0x7f58080d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:22.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.334+0000 7f580fe93640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58080714f0 con 0x7f580815ec60 2024-04-15T15:19:22.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.334+0000 7f580fe93640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58080d1940 con 0x7f580809b780 2024-04-15T15:19:22.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.334+0000 7f580fe93640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f580806cad0 con 0x7f580809b150 2024-04-15T15:19:22.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.334+0000 7f580dc2f640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f580815ec60 0x7f580809aa40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:40288/0 (socket says 172.21.15.105:40288) 2024-04-15T15:19:22.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.334+0000 7f580dc2f640 1 -- 172.21.15.105:0/1563825614 learned_addr learned my addr 172.21.15.105:0/1563825614 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:19:22.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.334+0000 7f580cc2d640 1 -- 172.21.15.105:0/1563825614 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3151970812 0 0) 0x7f58080714f0 con 0x7f580815ec60 2024-04-15T15:19:22.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.334+0000 7f580cc2d640 1 -- 172.21.15.105:0/1563825614 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f57f4003520 con 0x7f580815ec60 2024-04-15T15:19:22.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.334+0000 7f580d42e640 1 --2- 172.21.15.105:0/1563825614 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f580809b150 0x7f58080d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:22.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.334+0000 7f580cc2d640 1 -- 172.21.15.105:0/1563825614 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1481979407 0 0) 0x7f57f4003520 con 0x7f580815ec60 2024-04-15T15:19:22.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.334+0000 7f580cc2d640 1 -- 172.21.15.105:0/1563825614 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f580809b150 msgr2=0x7f58080d1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:22.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.334+0000 7f580cc2d640 1 --2- 172.21.15.105:0/1563825614 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f580809b150 0x7f58080d1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:22.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.334+0000 7f580cc2d640 1 -- 172.21.15.105:0/1563825614 >> v1:172.21.15.154:6789/0 conn(0x7f580809b780 legacy=0x7f58080cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:22.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.334+0000 7f580cc2d640 1 -- 172.21.15.105:0/1563825614 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58080ccd60 con 0x7f580815ec60 2024-04-15T15:19:22.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.334+0000 7f580cc2d640 1 -- 172.21.15.105:0/1563825614 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7f5804001670 con 0x7f580815ec60 2024-04-15T15:19:22.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.334+0000 7f580cc2d640 1 -- 172.21.15.105:0/1563825614 >> v1:172.21.15.105:6789/0 conn(0x7f580815ec60 legacy=0x7f580809aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:22.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.334+0000 7f580cc2d640 1 --2- 172.21.15.105:0/1563825614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57f40039e0 0x7f57f4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:22.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.334+0000 7f580cc2d640 1 --2- 172.21.15.105:0/1563825614 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f580809b150 0x7f57f4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:22.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.334+0000 7f580cc2d640 1 --2- 172.21.15.105:0/1563825614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57f4004d40 0x7f57f40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:22.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.334+0000 7f580cc2d640 1 -- 172.21.15.105:0/1563825614 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f57f400ba70 con 0x7f57f40039e0 2024-04-15T15:19:22.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.334+0000 7f580cc2d640 1 -- 172.21.15.105:0/1563825614 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f57f400bbf0 con 0x7f57f4004d40 2024-04-15T15:19:22.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.334+0000 7f580cc2d640 1 -- 172.21.15.105:0/1563825614 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f57f400bdf0 con 0x7f580809b150 2024-04-15T15:19:22.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.335+0000 7f580dc2f640 1 --2- 172.21.15.105:0/1563825614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57f40039e0 0x7f57f4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:22.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.335+0000 7f580d42e640 1 --2- 172.21.15.105:0/1563825614 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f580809b150 0x7f57f4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:22.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.335+0000 7f580e430640 1 --2- 172.21.15.105:0/1563825614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57f4004d40 0x7f57f40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:22.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.335+0000 7f580dc2f640 1 -- 172.21.15.105:0/1563825614 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f580809b150 msgr2=0x7f57f4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:22.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.335+0000 7f580dc2f640 1 --2- 172.21.15.105:0/1563825614 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f580809b150 0x7f57f4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:22.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.335+0000 7f580dc2f640 1 -- 172.21.15.105:0/1563825614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57f4004d40 msgr2=0x7f57f40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:22.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.335+0000 7f580dc2f640 1 --2- 172.21.15.105:0/1563825614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57f4004d40 0x7f57f40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:22.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.335+0000 7f580dc2f640 1 -- 172.21.15.105:0/1563825614 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57f4003520 con 0x7f57f40039e0 2024-04-15T15:19:22.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.336+0000 7f580dc2f640 1 --2- 172.21.15.105:0/1563825614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57f40039e0 0x7f57f4003e00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5804001eb0 tx=0x7f57f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4087aaf393b34dad server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:22.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.336+0000 7f580cc2d640 1 -- 172.21.15.105:0/1563825614 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5804007d30 con 0x7f57f40039e0 2024-04-15T15:19:22.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.336+0000 7f580cc2d640 1 -- 172.21.15.105:0/1563825614 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5804030070 con 0x7f57f40039e0 2024-04-15T15:19:22.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.336+0000 7f580cc2d640 1 -- 172.21.15.105:0/1563825614 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5804005d30 con 0x7f57f40039e0 2024-04-15T15:19:22.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.336+0000 7f580fe93640 1 -- 172.21.15.105:0/1563825614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57f40039e0 msgr2=0x7f57f4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:22.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.336+0000 7f580fe93640 1 --2- 172.21.15.105:0/1563825614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57f40039e0 0x7f57f4003e00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5804001eb0 tx=0x7f57f4005830 comp rx=0 tx=0).stop 2024-04-15T15:19:22.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.336+0000 7f580e430640 1 -- 172.21.15.105:0/1563825614 reap_dead start 2024-04-15T15:19:22.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.336+0000 7f580fe93640 1 -- 172.21.15.105:0/1563825614 shutdown_connections 2024-04-15T15:19:22.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.336+0000 7f580fe93640 1 -- 172.21.15.105:0/1563825614 >> 172.21.15.105:0/1563825614 conn(0x7f58080d2570 msgr2=0x7f58080d2980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:22.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.336+0000 7f580fe93640 1 -- 172.21.15.105:0/1563825614 shutdown_connections 2024-04-15T15:19:22.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.336+0000 7f580fe93640 1 -- 172.21.15.105:0/1563825614 wait complete. 2024-04-15T15:19:22.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.338+0000 7f580fe93640 1 Processor -- start 2024-04-15T15:19:22.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.338+0000 7f580fe93640 1 -- start start 2024-04-15T15:19:22.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.338+0000 7f580fe93640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57f40039e0 0x7f58081549a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:22.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.338+0000 7f580fe93640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f580815abc0 0x7f58080c60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:22.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.338+0000 7f580fe93640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58080b7be0 0x7f58080b03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:22.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.338+0000 7f580fe93640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f58080d1210 con 0x7f57f40039e0 2024-04-15T15:19:22.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.338+0000 7f580fe93640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f58080cd3b0 con 0x7f58080b7be0 2024-04-15T15:19:22.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.338+0000 7f580fe93640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f58080d1390 con 0x7f580815abc0 2024-04-15T15:19:22.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.338+0000 7f580dc2f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57f40039e0 0x7f58081549a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:22.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.338+0000 7f580dc2f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57f40039e0 0x7f58081549a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57002/0 (socket says 172.21.15.105:57002) 2024-04-15T15:19:22.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.338+0000 7f580dc2f640 1 -- 172.21.15.105:0/1313523669 learned_addr learned my addr 172.21.15.105:0/1313523669 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:22.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.339+0000 7f580d42e640 1 --2- 172.21.15.105:0/1313523669 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f580815abc0 0x7f58080c60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:22.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.339+0000 7f580dc2f640 1 -- 172.21.15.105:0/1313523669 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f580815abc0 msgr2=0x7f58080c60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:22.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.339+0000 7f580dc2f640 1 --2- 172.21.15.105:0/1313523669 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f580815abc0 0x7f58080c60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:22.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.339+0000 7f580dc2f640 1 -- 172.21.15.105:0/1313523669 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58080b7be0 msgr2=0x7f58080b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:22.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.339+0000 7f580e430640 1 --2- 172.21.15.105:0/1313523669 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58080b7be0 0x7f58080b03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:22.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.339+0000 7f580dc2f640 1 --2- 172.21.15.105:0/1313523669 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58080b7be0 0x7f58080b03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:22.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.339+0000 7f580dc2f640 1 -- 172.21.15.105:0/1313523669 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58080b6b40 con 0x7f57f40039e0 2024-04-15T15:19:22.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.339+0000 7f580dc2f640 1 --2- 172.21.15.105:0/1313523669 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57f40039e0 0x7f58081549a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f580402c540 tx=0x7f580402e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:22.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.339+0000 7f57fe7fc640 1 -- 172.21.15.105:0/1313523669 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5804046070 con 0x7f57f40039e0 2024-04-15T15:19:22.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.339+0000 7f57fe7fc640 1 -- 172.21.15.105:0/1313523669 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5804004c10 con 0x7f57f40039e0 2024-04-15T15:19:22.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.339+0000 7f580fe93640 1 -- 172.21.15.105:0/1313523669 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58080b6e20 con 0x7f57f40039e0 2024-04-15T15:19:22.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.339+0000 7f57fe7fc640 1 -- 172.21.15.105:0/1313523669 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5804007d10 con 0x7f57f40039e0 2024-04-15T15:19:22.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.339+0000 7f580fe93640 1 -- 172.21.15.105:0/1313523669 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58080b73e0 con 0x7f57f40039e0 2024-04-15T15:19:22.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.340+0000 7f580fe93640 1 -- 172.21.15.105:0/1313523669 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5808068a40 con 0x7f57f40039e0 2024-04-15T15:19:22.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.340+0000 7f57fe7fc640 1 -- 172.21.15.105:0/1313523669 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5804007070 con 0x7f57f40039e0 2024-04-15T15:19:22.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.341+0000 7f57fe7fc640 1 --2- 172.21.15.105:0/1313523669 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f57ec06c8f0 0x7f57ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:22.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.341+0000 7f57fe7fc640 1 -- 172.21.15.105:0/1313523669 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f58040acc30 con 0x7f57f40039e0 2024-04-15T15:19:22.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.341+0000 7f580d42e640 1 --2- 172.21.15.105:0/1313523669 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f57ec06c8f0 0x7f57ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:22.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.341+0000 7f580d42e640 1 --2- 172.21.15.105:0/1313523669 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f57ec06c8f0 0x7f57ec06edb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f58080b1570 tx=0x7f57f8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:22.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.345+0000 7f57fe7fc640 1 -- 172.21.15.105:0/1313523669 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f580407b320 con 0x7f57f40039e0 2024-04-15T15:19:22.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.557+0000 7f580fe93640 1 -- 172.21.15.105:0/1313523669 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f58080c65e0 con 0x7f57f40039e0 2024-04-15T15:19:22.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.558+0000 7f57fe7fc640 1 -- 172.21.15.105:0/1313523669 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3876 (secure 0 0 0) 0x7f580407ac20 con 0x7f57f40039e0 2024-04-15T15:19:22.560 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:19:22.560 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4259,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/3319657830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":3319657830},{"type":"v1","addr":"172.21.15.154:6835","nonce":3319657830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2677199890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2677199890},{"type":"v1","addr":"172.21.15.105:6835","nonce":2677199890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2867739743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2867739743},{"type":"v1","addr":"172.21.15.105:6837","nonce":2867739743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6834/1529675504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1529675504},{"type":"v1","addr":"172.21.15.154:6834","nonce":1529675504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-15T15:19:22.560 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 3 2024-04-15T15:19:22.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.560+0000 7f580fe93640 1 -- 172.21.15.105:0/1313523669 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f57ec06c8f0 msgr2=0x7f57ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:22.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.560+0000 7f580fe93640 1 --2- 172.21.15.105:0/1313523669 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f57ec06c8f0 0x7f57ec06edb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f58080b1570 tx=0x7f57f8008040 comp rx=0 tx=0).stop 2024-04-15T15:19:22.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.560+0000 7f580fe93640 1 -- 172.21.15.105:0/1313523669 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57f40039e0 msgr2=0x7f58081549a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:22.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.560+0000 7f580fe93640 1 --2- 172.21.15.105:0/1313523669 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57f40039e0 0x7f58081549a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f580402c540 tx=0x7f580402e070 comp rx=0 tx=0).stop 2024-04-15T15:19:22.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.560+0000 7f580fe93640 1 -- 172.21.15.105:0/1313523669 shutdown_connections 2024-04-15T15:19:22.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.560+0000 7f580fe93640 1 --2- 172.21.15.105:0/1313523669 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f580815abc0 0x7f58080c60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:22.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.560+0000 7f580fe93640 1 --2- 172.21.15.105:0/1313523669 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f57ec06c8f0 0x7f57ec06edb0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:22.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.560+0000 7f580fe93640 1 --2- 172.21.15.105:0/1313523669 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58080b7be0 0x7f58080b03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:22.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.560+0000 7f580fe93640 1 --2- 172.21.15.105:0/1313523669 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57f40039e0 0x7f58081549a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:22.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.560+0000 7f580fe93640 1 -- 172.21.15.105:0/1313523669 >> 172.21.15.105:0/1313523669 conn(0x7f58080d2570 msgr2=0x7f580806d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:22.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.560+0000 7f580fe93640 1 -- 172.21.15.105:0/1313523669 shutdown_connections 2024-04-15T15:19:22.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.560+0000 7f580fe93640 1 -- 172.21.15.105:0/1313523669 wait complete. 2024-04-15T15:19:22.573 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:19:22.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.674+0000 7f622b752640 1 Processor -- start 2024-04-15T15:19:22.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.674+0000 7f622b752640 1 -- start start 2024-04-15T15:19:22.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.675+0000 7f622b752640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f622415ecf0 0x7f622406e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:22.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.675+0000 7f622b752640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f62240d0f10 0x7f62240ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:22.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.675+0000 7f622b752640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f62240cd190 0x7f62240cd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:22.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.675+0000 7f622b752640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f622406d2a0 con 0x7f62240cd190 2024-04-15T15:19:22.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.675+0000 7f622b752640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f622406d580 con 0x7f62240d0f10 2024-04-15T15:19:22.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.675+0000 7f622b752640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f622406d860 con 0x7f622415ecf0 2024-04-15T15:19:22.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.675+0000 7f6229cef640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f62240cd190 0x7f62240cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:22.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.675+0000 7f6229cef640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f62240cd190 0x7f62240cd620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57008/0 (socket says 172.21.15.105:57008) 2024-04-15T15:19:22.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.675+0000 7f6229cef640 1 -- 172.21.15.105:0/1886246266 learned_addr learned my addr 172.21.15.105:0/1886246266 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:22.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.675+0000 7f62294ee640 1 --2- 172.21.15.105:0/1886246266 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f622415ecf0 0x7f622406e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:22.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.675+0000 7f6228ced640 1 --2- 172.21.15.105:0/1886246266 >> v2:172.21.15.154:3300/0 conn(0x7f62240d0f10 0x7f62240ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:22.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.676+0000 7f6229cef640 1 -- 172.21.15.105:0/1886246266 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f622415ecf0 msgr2=0x7f622406e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:22.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.676+0000 7f6229cef640 1 --2- 172.21.15.105:0/1886246266 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f622415ecf0 0x7f622406e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:22.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.676+0000 7f6229cef640 1 -- 172.21.15.105:0/1886246266 >> v2:172.21.15.154:3300/0 conn(0x7f62240d0f10 msgr2=0x7f62240ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:22.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.676+0000 7f6229cef640 1 --2- 172.21.15.105:0/1886246266 >> v2:172.21.15.154:3300/0 conn(0x7f62240d0f10 0x7f62240ccc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:22.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.676+0000 7f6229cef640 1 -- 172.21.15.105:0/1886246266 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f622406db40 con 0x7f62240cd190 2024-04-15T15:19:22.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.676+0000 7f6229cef640 1 --2- 172.21.15.105:0/1886246266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62240cd190 0x7f62240cd620 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f62240739c0 tx=0x7f621c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c63e020688f94537 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:22.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.676+0000 7f621bfff640 1 -- 172.21.15.105:0/1886246266 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621c035020 con 0x7f62240cd190 2024-04-15T15:19:22.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.676+0000 7f621bfff640 1 -- 172.21.15.105:0/1886246266 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f621c031870 con 0x7f62240cd190 2024-04-15T15:19:22.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.676+0000 7f621bfff640 1 -- 172.21.15.105:0/1886246266 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621c031c40 con 0x7f62240cd190 2024-04-15T15:19:22.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.677+0000 7f622b752640 1 -- 172.21.15.105:0/1886246266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62240cd190 msgr2=0x7f62240cd620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:22.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.677+0000 7f622b752640 1 --2- 172.21.15.105:0/1886246266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62240cd190 0x7f62240cd620 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f62240739c0 tx=0x7f621c02b910 comp rx=0 tx=0).stop 2024-04-15T15:19:22.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.677+0000 7f622b752640 1 -- 172.21.15.105:0/1886246266 shutdown_connections 2024-04-15T15:19:22.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.677+0000 7f622b752640 1 --2- 172.21.15.105:0/1886246266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62240cd190 0x7f62240cd620 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:22.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.677+0000 7f622b752640 1 --2- 172.21.15.105:0/1886246266 >> v2:172.21.15.154:3300/0 conn(0x7f62240d0f10 0x7f62240ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:22.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.677+0000 7f622b752640 1 --2- 172.21.15.105:0/1886246266 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f622415ecf0 0x7f622406e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:22.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.677+0000 7f622b752640 1 -- 172.21.15.105:0/1886246266 >> 172.21.15.105:0/1886246266 conn(0x7f62240d2600 msgr2=0x7f62240d2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:22.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.677+0000 7f622b752640 1 -- 172.21.15.105:0/1886246266 shutdown_connections 2024-04-15T15:19:22.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.677+0000 7f622b752640 1 -- 172.21.15.105:0/1886246266 wait complete. 2024-04-15T15:19:22.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.678+0000 7f622b752640 1 Processor -- start 2024-04-15T15:19:22.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.678+0000 7f622b752640 1 -- start start 2024-04-15T15:19:22.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.679+0000 7f622b752640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62240cd190 0x7f6224150b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:22.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.679+0000 7f622b752640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62240d0f10 0x7f6224161c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:22.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.679+0000 7f622b752640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f622415ecf0 0x7f62240b5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:22.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.679+0000 7f622b752640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f622415f0d0 con 0x7f62240d0f10 2024-04-15T15:19:22.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.679+0000 7f622b752640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f62240d1370 con 0x7f622415ecf0 2024-04-15T15:19:22.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.679+0000 7f622b752640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f622415f250 con 0x7f62240cd190 2024-04-15T15:19:22.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.679+0000 7f6228ced640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62240d0f10 0x7f6224161c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:22.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.679+0000 7f6228ced640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62240d0f10 0x7f6224161c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57022/0 (socket says 172.21.15.105:57022) 2024-04-15T15:19:22.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.679+0000 7f6228ced640 1 -- 172.21.15.105:0/8708324 learned_addr learned my addr 172.21.15.105:0/8708324 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:22.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.679+0000 7f62294ee640 1 --2- 172.21.15.105:0/8708324 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62240cd190 0x7f6224150b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:22.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.679+0000 7f6229cef640 1 --2- 172.21.15.105:0/8708324 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f622415ecf0 0x7f62240b5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:22.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.679+0000 7f6228ced640 1 -- 172.21.15.105:0/8708324 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62240cd190 msgr2=0x7f6224150b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:22.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.679+0000 7f6228ced640 1 --2- 172.21.15.105:0/8708324 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62240cd190 0x7f6224150b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:22.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.679+0000 7f6228ced640 1 -- 172.21.15.105:0/8708324 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f622415ecf0 msgr2=0x7f62240b5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:22.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.679+0000 7f6228ced640 1 --2- 172.21.15.105:0/8708324 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f622415ecf0 0x7f62240b5220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:22.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.679+0000 7f6228ced640 1 -- 172.21.15.105:0/8708324 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6224151050 con 0x7f62240d0f10 2024-04-15T15:19:22.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.679+0000 7f62294ee640 1 --2- 172.21.15.105:0/8708324 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62240cd190 0x7f6224150b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:22.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.680+0000 7f6228ced640 1 --2- 172.21.15.105:0/8708324 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62240d0f10 0x7f6224161c50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f621400d800 tx=0x7f621400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:22.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.680+0000 7f6219ffb640 1 -- 172.21.15.105:0/8708324 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6214018020 con 0x7f62240d0f10 2024-04-15T15:19:22.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.680+0000 7f6219ffb640 1 -- 172.21.15.105:0/8708324 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62140096a0 con 0x7f62240d0f10 2024-04-15T15:19:22.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.680+0000 7f622b752640 1 -- 172.21.15.105:0/8708324 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6224160bb0 con 0x7f62240d0f10 2024-04-15T15:19:22.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.680+0000 7f6219ffb640 1 -- 172.21.15.105:0/8708324 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6214009a70 con 0x7f62240d0f10 2024-04-15T15:19:22.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.680+0000 7f622b752640 1 -- 172.21.15.105:0/8708324 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6224161170 con 0x7f62240d0f10 2024-04-15T15:19:22.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.681+0000 7f622b752640 1 -- 172.21.15.105:0/8708324 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6224068a40 con 0x7f62240d0f10 2024-04-15T15:19:22.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.681+0000 7f6219ffb640 1 -- 172.21.15.105:0/8708324 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f621401b020 con 0x7f62240d0f10 2024-04-15T15:19:22.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.682+0000 7f6219ffb640 1 --2- 172.21.15.105:0/8708324 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f620806c920 0x7f620806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:22.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.682+0000 7f6219ffb640 1 -- 172.21.15.105:0/8708324 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f621408b930 con 0x7f62240d0f10 2024-04-15T15:19:22.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.682+0000 7f62294ee640 1 --2- 172.21.15.105:0/8708324 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f620806c920 0x7f620806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:22.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.682+0000 7f62294ee640 1 --2- 172.21.15.105:0/8708324 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f620806c920 0x7f620806ede0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6220007330 tx=0x7f622000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:22.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.686+0000 7f6219ffb640 1 -- 172.21.15.105:0/8708324 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6214022030 con 0x7f62240d0f10 2024-04-15T15:19:22.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.863+0000 7f622b752640 1 -- 172.21.15.105:0/8708324 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f62240b8580 con 0x7f62240d0f10 2024-04-15T15:19:22.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.864+0000 7f6219ffb640 1 -- 172.21.15.105:0/8708324 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12932 (secure 0 0 0) 0x7f621405a140 con 0x7f62240d0f10 2024-04-15T15:19:22.865 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:19:22.866 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":16,"fsid":"09e4d2a4-a324-4793-af77-644b25e520e1","created":"2024-04-15T15:19:01.524543+0000","modified":"2024-04-15T15:19:15.830502+0000","last_up_change":"2024-04-15T15:19:12.825815+0000","last_in_change":"2024-04-15T15:19:03.788126+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:19:13.748585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6f6292b-2d7e-4aa0-9d1c-6ddd0f97287c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6801","nonce":2873040054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6803","nonce":2873040054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6807","nonce":2873040054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6805","nonce":2873040054}]},"public_addr":"172.21.15.105:6801/2873040054","cluster_addr":"172.21.15.105:6803/2873040054","heartbeat_back_addr":"172.21.15.105:6807/2873040054","heartbeat_front_addr":"172.21.15.105:6805/2873040054","state":["exists","up"]},{"osd":1,"uuid":"8900966a-b393-41e3-8f93-e5a43bd23622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6818","nonce":2494994479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6821","nonce":2494994479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6828","nonce":2494994479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6823","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6825","nonce":2494994479}]},"public_addr":"172.21.15.105:6818/2494994479","cluster_addr":"172.21.15.105:6821/2494994479","heartbeat_back_addr":"172.21.15.105:6828/2494994479","heartbeat_front_addr":"172.21.15.105:6825/2494994479","state":["exists","up"]},{"osd":2,"uuid":"81a5dc7a-9325-4ff5-941d-a2d5fba21f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6809","nonce":3798408699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6811","nonce":3798408699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6815","nonce":3798408699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6813","nonce":3798408699}]},"public_addr":"172.21.15.105:6809/3798408699","cluster_addr":"172.21.15.105:6811/3798408699","heartbeat_back_addr":"172.21.15.105:6815/3798408699","heartbeat_front_addr":"172.21.15.105:6813/3798408699","state":["exists","up"]},{"osd":3,"uuid":"df766ed2-5c71-4805-96f4-3d4970edc960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6819","nonce":1920720186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6824","nonce":1920720186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6831","nonce":1920720186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6827","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6829","nonce":1920720186}]},"public_addr":"172.21.15.105:6819/1920720186","cluster_addr":"172.21.15.105:6824/1920720186","heartbeat_back_addr":"172.21.15.105:6831/1920720186","heartbeat_front_addr":"172.21.15.105:6829/1920720186","state":["exists","up"]},{"osd":4,"uuid":"e1039e03-28b8-49df-b951-c91930b93c9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6810","nonce":3516494456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6813","nonce":3516494456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6820","nonce":3516494456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6817","nonce":3516494456}]},"public_addr":"172.21.15.154:6810/3516494456","cluster_addr":"172.21.15.154:6813/3516494456","heartbeat_back_addr":"172.21.15.154:6820/3516494456","heartbeat_front_addr":"172.21.15.154:6817/3516494456","state":["exists","up"]},{"osd":5,"uuid":"a9f5eaf1-66ff-405a-a9e4-ec3340812a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6801","nonce":540367751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6803","nonce":540367751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6807","nonce":540367751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6805","nonce":540367751}]},"public_addr":"172.21.15.154:6801/540367751","cluster_addr":"172.21.15.154:6803/540367751","heartbeat_back_addr":"172.21.15.154:6807/540367751","heartbeat_front_addr":"172.21.15.154:6805/540367751","state":["exists","up"]},{"osd":6,"uuid":"267d02ea-675f-428f-aae8-e0375e4d4feb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6825","nonce":1550829005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6827","nonce":1550829005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6831","nonce":1550829005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6829","nonce":1550829005}]},"public_addr":"172.21.15.154:6825/1550829005","cluster_addr":"172.21.15.154:6827/1550829005","heartbeat_back_addr":"172.21.15.154:6831/1550829005","heartbeat_front_addr":"172.21.15.154:6829/1550829005","state":["exists","up"]},{"osd":7,"uuid":"759ac6cc-896a-423c-885f-9fe90d8dcb14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6809","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6811","nonce":3920089903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6816","nonce":3920089903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6823","nonce":3920089903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6819","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6821","nonce":3920089903}]},"public_addr":"172.21.15.154:6811/3920089903","cluster_addr":"172.21.15.154:6816/3920089903","heartbeat_back_addr":"172.21.15.154:6823/3920089903","heartbeat_front_addr":"172.21.15.154:6821/3920089903","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.949509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.984072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.907278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.947391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.975447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.998896+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:09.028739+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.894119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:19:22.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.865+0000 7f622b752640 1 -- 172.21.15.105:0/8708324 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f620806c920 msgr2=0x7f620806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:22.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.865+0000 7f622b752640 1 --2- 172.21.15.105:0/8708324 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f620806c920 0x7f620806ede0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6220007330 tx=0x7f622000b040 comp rx=0 tx=0).stop 2024-04-15T15:19:22.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.865+0000 7f622b752640 1 -- 172.21.15.105:0/8708324 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62240d0f10 msgr2=0x7f6224161c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:22.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.865+0000 7f622b752640 1 --2- 172.21.15.105:0/8708324 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62240d0f10 0x7f6224161c50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f621400d800 tx=0x7f621400dcc0 comp rx=0 tx=0).stop 2024-04-15T15:19:22.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.866+0000 7f622b752640 1 -- 172.21.15.105:0/8708324 shutdown_connections 2024-04-15T15:19:22.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.866+0000 7f622b752640 1 --2- 172.21.15.105:0/8708324 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f620806c920 0x7f620806ede0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:22.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.866+0000 7f622b752640 1 --2- 172.21.15.105:0/8708324 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f622415ecf0 0x7f62240b5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:22.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.866+0000 7f622b752640 1 --2- 172.21.15.105:0/8708324 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62240d0f10 0x7f6224161c50 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:22.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.866+0000 7f622b752640 1 --2- 172.21.15.105:0/8708324 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62240cd190 0x7f6224150b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:22.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.866+0000 7f622b752640 1 -- 172.21.15.105:0/8708324 >> 172.21.15.105:0/8708324 conn(0x7f62240d2600 msgr2=0x7f62240d2c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:22.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.866+0000 7f622b752640 1 -- 172.21.15.105:0/8708324 shutdown_connections 2024-04-15T15:19:22.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.866+0000 7f622b752640 1 -- 172.21.15.105:0/8708324 wait complete. 2024-04-15T15:19:22.878 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:19:13.748585+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:19:22.879 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:19:22.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.984+0000 7fbe94532640 1 Processor -- start 2024-04-15T15:19:22.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.984+0000 7fbe94532640 1 -- start start 2024-04-15T15:19:22.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.985+0000 7fbe94532640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbe8c15ea10 0x7fbe8c0b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:22.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.985+0000 7fbe94532640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbe8c0734b0 0x7fbe8c06b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:22.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.985+0000 7fbe94532640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe8c15edf0 con 0x7fbe8c06bf00 2024-04-15T15:19:22.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.985+0000 7fbe94532640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fbe8c0b0ae0 con 0x7fbe8c0734b0 2024-04-15T15:19:22.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.985+0000 7fbe94532640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fbe8c0b0dc0 con 0x7fbe8c15ea10 2024-04-15T15:19:22.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.985+0000 7fbe91acd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbe8c0734b0 0x7fbe8c06b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:22.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.985+0000 7fbe91acd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbe8c0734b0 0x7fbe8c06b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57038/0 (socket says 172.21.15.105:57038) 2024-04-15T15:19:22.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.985+0000 7fbe91acd640 1 -- 172.21.15.105:0/4227672065 learned_addr learned my addr 172.21.15.105:0/4227672065 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:22.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.986+0000 7fbe922ce640 1 --2- 172.21.15.105:0/4227672065 >> v2:172.21.15.154:3300/0 conn(0x7fbe8c15ea10 0x7fbe8c0b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:22.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.986+0000 7fbe91acd640 1 -- 172.21.15.105:0/4227672065 >> v2:172.21.15.154:3300/0 conn(0x7fbe8c15ea10 msgr2=0x7fbe8c0b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:22.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.986+0000 7fbe91acd640 1 --2- 172.21.15.105:0/4227672065 >> v2:172.21.15.154:3300/0 conn(0x7fbe8c15ea10 0x7fbe8c0b2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:22.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.986+0000 7fbe91acd640 1 -- 172.21.15.105:0/4227672065 >> v1:172.21.15.154:6789/0 conn(0x7fbe8c06bf00 legacy=0x7fbe8c0d1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:22.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.986+0000 7fbe91acd640 1 -- 172.21.15.105:0/4227672065 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe8c0b6490 con 0x7fbe8c0734b0 2024-04-15T15:19:22.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.986+0000 7fbe91acd640 1 --2- 172.21.15.105:0/4227672065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe8c0734b0 0x7fbe8c06b9c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c0b0c60 tx=0x7fbe7c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a4d93190cb664d server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:22.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.986+0000 7fbe912cc640 1 -- 172.21.15.105:0/4227672065 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c035020 con 0x7fbe8c0734b0 2024-04-15T15:19:22.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.987+0000 7fbe912cc640 1 -- 172.21.15.105:0/4227672065 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe7c031870 con 0x7fbe8c0734b0 2024-04-15T15:19:22.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.987+0000 7fbe912cc640 1 -- 172.21.15.105:0/4227672065 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c031c40 con 0x7fbe8c0734b0 2024-04-15T15:19:22.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.987+0000 7fbe94532640 1 -- 172.21.15.105:0/4227672065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe8c0734b0 msgr2=0x7fbe8c06b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:22.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.987+0000 7fbe94532640 1 --2- 172.21.15.105:0/4227672065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe8c0734b0 0x7fbe8c06b9c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c0b0c60 tx=0x7fbe7c02b910 comp rx=0 tx=0).stop 2024-04-15T15:19:22.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.987+0000 7fbe94532640 1 -- 172.21.15.105:0/4227672065 shutdown_connections 2024-04-15T15:19:22.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.987+0000 7fbe94532640 1 --2- 172.21.15.105:0/4227672065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe8c0734b0 0x7fbe8c06b9c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:22.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.987+0000 7fbe94532640 1 --2- 172.21.15.105:0/4227672065 >> v2:172.21.15.154:3300/0 conn(0x7fbe8c15ea10 0x7fbe8c0b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:22.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.987+0000 7fbe94532640 1 -- 172.21.15.105:0/4227672065 >> 172.21.15.105:0/4227672065 conn(0x7fbe8c0b6b10 msgr2=0x7fbe8c09aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:22.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.987+0000 7fbe94532640 1 -- 172.21.15.105:0/4227672065 shutdown_connections 2024-04-15T15:19:22.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.987+0000 7fbe94532640 1 -- 172.21.15.105:0/4227672065 wait complete. 2024-04-15T15:19:22.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.988+0000 7fbe94532640 1 Processor -- start 2024-04-15T15:19:22.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.988+0000 7fbe94532640 1 -- start start 2024-04-15T15:19:22.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.989+0000 7fbe94532640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbe8c06bf00 0x7fbe8c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:22.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.989+0000 7fbe94532640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe8c0734b0 0x7fbe8c160fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:22.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.989+0000 7fbe94532640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe8c15ea10 0x7fbe8c151850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:22.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.989+0000 7fbe94532640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbe8c0b05c0 con 0x7fbe8c0734b0 2024-04-15T15:19:22.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.989+0000 7fbe94532640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbe8c0b2260 con 0x7fbe8c15ea10 2024-04-15T15:19:22.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.989+0000 7fbe94532640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fbe8c0b6140 con 0x7fbe8c06bf00 2024-04-15T15:19:22.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.989+0000 7fbe91acd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe8c0734b0 0x7fbe8c160fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:22.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.989+0000 7fbe91acd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe8c0734b0 0x7fbe8c160fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57054/0 (socket says 172.21.15.105:57054) 2024-04-15T15:19:22.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.989+0000 7fbe91acd640 1 -- 172.21.15.105:0/3216088511 learned_addr learned my addr 172.21.15.105:0/3216088511 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:22.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.989+0000 7fbe922ce640 1 --2- 172.21.15.105:0/3216088511 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbe8c06bf00 0x7fbe8c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:22.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.989+0000 7fbe92acf640 1 --2- 172.21.15.105:0/3216088511 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe8c15ea10 0x7fbe8c151850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:22.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.989+0000 7fbe91acd640 1 -- 172.21.15.105:0/3216088511 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbe8c06bf00 msgr2=0x7fbe8c0706b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:22.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.989+0000 7fbe91acd640 1 --2- 172.21.15.105:0/3216088511 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbe8c06bf00 0x7fbe8c0706b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:22.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.989+0000 7fbe91acd640 1 -- 172.21.15.105:0/3216088511 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe8c15ea10 msgr2=0x7fbe8c151850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:22.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.989+0000 7fbe91acd640 1 --2- 172.21.15.105:0/3216088511 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe8c15ea10 0x7fbe8c151850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:22.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.989+0000 7fbe91acd640 1 -- 172.21.15.105:0/3216088511 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe8c152530 con 0x7fbe8c0734b0 2024-04-15T15:19:22.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.989+0000 7fbe92acf640 1 --2- 172.21.15.105:0/3216088511 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe8c15ea10 0x7fbe8c151850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:22.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.990+0000 7fbe91acd640 1 --2- 172.21.15.105:0/3216088511 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe8c0734b0 0x7fbe8c160fe0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c02b950 tx=0x7fbe7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:22.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.990+0000 7fbe82ffd640 1 -- 172.21.15.105:0/3216088511 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c040070 con 0x7fbe8c0734b0 2024-04-15T15:19:22.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.990+0000 7fbe82ffd640 1 -- 172.21.15.105:0/3216088511 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe7c03a420 con 0x7fbe8c0734b0 2024-04-15T15:19:22.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.990+0000 7fbe94532640 1 -- 172.21.15.105:0/3216088511 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe8c152830 con 0x7fbe8c0734b0 2024-04-15T15:19:22.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.990+0000 7fbe82ffd640 1 -- 172.21.15.105:0/3216088511 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c035040 con 0x7fbe8c0734b0 2024-04-15T15:19:22.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.990+0000 7fbe94532640 1 -- 172.21.15.105:0/3216088511 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe8c153070 con 0x7fbe8c0734b0 2024-04-15T15:19:22.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.991+0000 7fbe94532640 1 -- 172.21.15.105:0/3216088511 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe8c068ab0 con 0x7fbe8c0734b0 2024-04-15T15:19:22.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.991+0000 7fbe82ffd640 1 -- 172.21.15.105:0/3216088511 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbe7c038020 con 0x7fbe8c0734b0 2024-04-15T15:19:22.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.992+0000 7fbe82ffd640 1 --2- 172.21.15.105:0/3216088511 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fbe7006c8f0 0x7fbe7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:22.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.992+0000 7fbe82ffd640 1 -- 172.21.15.105:0/3216088511 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fbe7c0ad7b0 con 0x7fbe8c0734b0 2024-04-15T15:19:22.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.992+0000 7fbe922ce640 1 --2- 172.21.15.105:0/3216088511 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fbe7006c8f0 0x7fbe7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:22.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.992+0000 7fbe922ce640 1 --2- 172.21.15.105:0/3216088511 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fbe7006c8f0 0x7fbe7006edb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbe880070b0 tx=0x7fbe88007040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:22.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:22.996+0000 7fbe82ffd640 1 -- 172.21.15.105:0/3216088511 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbe8c152830 con 0x7fbe8c0734b0 2024-04-15T15:19:23.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.169+0000 7fbe94532640 1 -- 172.21.15.105:0/3216088511 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbe8c06fc40 con 0x7fbe8c0734b0 2024-04-15T15:19:23.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.169+0000 7fbe82ffd640 1 -- 172.21.15.105:0/3216088511 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7fbe7c07bfc0 con 0x7fbe8c0734b0 2024-04-15T15:19:23.172 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-04-15T15:19:23.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.172+0000 7fbe94532640 1 -- 172.21.15.105:0/3216088511 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fbe7006c8f0 msgr2=0x7fbe7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:23.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.172+0000 7fbe94532640 1 --2- 172.21.15.105:0/3216088511 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fbe7006c8f0 0x7fbe7006edb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbe880070b0 tx=0x7fbe88007040 comp rx=0 tx=0).stop 2024-04-15T15:19:23.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.172+0000 7fbe94532640 1 -- 172.21.15.105:0/3216088511 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe8c0734b0 msgr2=0x7fbe8c160fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:23.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.172+0000 7fbe94532640 1 --2- 172.21.15.105:0/3216088511 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe8c0734b0 0x7fbe8c160fe0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c02b950 tx=0x7fbe7c0046c0 comp rx=0 tx=0).stop 2024-04-15T15:19:23.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.172+0000 7fbe94532640 1 -- 172.21.15.105:0/3216088511 shutdown_connections 2024-04-15T15:19:23.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.172+0000 7fbe94532640 1 --2- 172.21.15.105:0/3216088511 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fbe7006c8f0 0x7fbe7006edb0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:23.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.172+0000 7fbe94532640 1 --2- 172.21.15.105:0/3216088511 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe8c15ea10 0x7fbe8c151850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:23.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.172+0000 7fbe94532640 1 --2- 172.21.15.105:0/3216088511 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe8c0734b0 0x7fbe8c160fe0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:23.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.172+0000 7fbe94532640 1 --2- 172.21.15.105:0/3216088511 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbe8c06bf00 0x7fbe8c0706b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:23.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.172+0000 7fbe94532640 1 -- 172.21.15.105:0/3216088511 >> 172.21.15.105:0/3216088511 conn(0x7fbe8c0b6b10 msgr2=0x7fbe8c0d19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:23.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.172+0000 7fbe94532640 1 -- 172.21.15.105:0/3216088511 shutdown_connections 2024-04-15T15:19:23.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.172+0000 7fbe94532640 1 -- 172.21.15.105:0/3216088511 wait complete. 2024-04-15T15:19:23.184 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-15T15:19:23.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.287+0000 7f3c50800640 1 Processor -- start 2024-04-15T15:19:23.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.287+0000 7f3c50800640 1 -- start start 2024-04-15T15:19:23.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.288+0000 7f3c50800640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3c4c0d1c90 0x7f3c4c0d39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:23.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.288+0000 7f3c50800640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3c4c0cf350 0x7f3c4c0cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:23.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.288+0000 7f3c50800640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c4c0d2c30 con 0x7f3c4c15eae0 2024-04-15T15:19:23.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.288+0000 7f3c50800640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f3c4c0d3560 con 0x7f3c4c0cf350 2024-04-15T15:19:23.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.288+0000 7f3c50800640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f3c4c0d2f10 con 0x7f3c4c0d1c90 2024-04-15T15:19:23.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.288+0000 7f3c4b59e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3c4c0cf350 0x7f3c4c0cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:23.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.288+0000 7f3c4b59e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3c4c0cf350 0x7f3c4c0cf7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57070/0 (socket says 172.21.15.105:57070) 2024-04-15T15:19:23.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.288+0000 7f3c4b59e640 1 -- 172.21.15.105:0/2815509015 learned_addr learned my addr 172.21.15.105:0/2815509015 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:23.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.288+0000 7f3c4b59e640 1 -- 172.21.15.105:0/2815509015 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3c4c0d1c90 msgr2=0x7f3c4c0d39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:23.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.288+0000 7f3c4a59c640 1 --2- 172.21.15.105:0/2815509015 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3c4c0d1c90 0x7f3c4c0d39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:23.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.288+0000 7f3c4b59e640 1 --2- 172.21.15.105:0/2815509015 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3c4c0d1c90 0x7f3c4c0d39f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:23.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.288+0000 7f3c4b59e640 1 -- 172.21.15.105:0/2815509015 >> v1:172.21.15.154:6789/0 conn(0x7f3c4c15eae0 legacy=0x7f3c4c0d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:23.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.288+0000 7f3c4b59e640 1 -- 172.21.15.105:0/2815509015 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c4c0d31f0 con 0x7f3c4c0cf350 2024-04-15T15:19:23.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.289+0000 7f3c4b59e640 1 --2- 172.21.15.105:0/2815509015 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3c4c0cf350 0x7f3c4c0cf7e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3c44009e50 tx=0x7f3c4402bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66a6c7f9848c2c29 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:23.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.289+0000 7f3c49d9b640 1 -- 172.21.15.105:0/2815509015 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c4402c950 con 0x7f3c4c0cf350 2024-04-15T15:19:23.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.289+0000 7f3c49d9b640 1 -- 172.21.15.105:0/2815509015 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c4402caf0 con 0x7f3c4c0cf350 2024-04-15T15:19:23.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.289+0000 7f3c49d9b640 1 -- 172.21.15.105:0/2815509015 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c440343f0 con 0x7f3c4c0cf350 2024-04-15T15:19:23.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.290+0000 7f3c50800640 1 -- 172.21.15.105:0/2815509015 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3c4c0cf350 msgr2=0x7f3c4c0cf7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:23.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.290+0000 7f3c50800640 1 --2- 172.21.15.105:0/2815509015 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3c4c0cf350 0x7f3c4c0cf7e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3c44009e50 tx=0x7f3c4402bdb0 comp rx=0 tx=0).stop 2024-04-15T15:19:23.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.290+0000 7f3c50800640 1 -- 172.21.15.105:0/2815509015 shutdown_connections 2024-04-15T15:19:23.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.290+0000 7f3c50800640 1 --2- 172.21.15.105:0/2815509015 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3c4c0cf350 0x7f3c4c0cf7e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:23.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.290+0000 7f3c50800640 1 --2- 172.21.15.105:0/2815509015 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3c4c0d1c90 0x7f3c4c0d39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:23.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.290+0000 7f3c50800640 1 -- 172.21.15.105:0/2815509015 >> 172.21.15.105:0/2815509015 conn(0x7f3c4c0b6560 msgr2=0x7f3c4c0b6970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:23.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.290+0000 7f3c50800640 1 -- 172.21.15.105:0/2815509015 shutdown_connections 2024-04-15T15:19:23.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.290+0000 7f3c50800640 1 -- 172.21.15.105:0/2815509015 wait complete. 2024-04-15T15:19:23.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.291+0000 7f3c50800640 1 Processor -- start 2024-04-15T15:19:23.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.291+0000 7f3c50800640 1 -- start start 2024-04-15T15:19:23.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.292+0000 7f3c50800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c4c0cf350 0x7f3c4c14c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:23.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.292+0000 7f3c50800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3c4c0d1c90 0x7f3c4c0cd020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:23.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.292+0000 7f3c50800640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3c4c15eae0 0x7f3c4c0cd560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:23.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.292+0000 7f3c50800640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4c0cfd20 con 0x7f3c4c0d1c90 2024-04-15T15:19:23.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.292+0000 7f3c50800640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4c15eec0 con 0x7f3c4c0cf350 2024-04-15T15:19:23.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.292+0000 7f3c50800640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f3c4c0b7ad0 con 0x7f3c4c15eae0 2024-04-15T15:19:23.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.292+0000 7f3c4a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3c4c0d1c90 0x7f3c4c0cd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:23.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.292+0000 7f3c4a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3c4c0d1c90 0x7f3c4c0cd020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57074/0 (socket says 172.21.15.105:57074) 2024-04-15T15:19:23.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.292+0000 7f3c4a59c640 1 -- 172.21.15.105:0/4007295584 learned_addr learned my addr 172.21.15.105:0/4007295584 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:23.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.292+0000 7f3c4ad9d640 1 --2- 172.21.15.105:0/4007295584 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c4c0cf350 0x7f3c4c14c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:23.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.292+0000 7f3c4b59e640 1 --2- 172.21.15.105:0/4007295584 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3c4c15eae0 0x7f3c4c0cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:23.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.292+0000 7f3c4a59c640 1 -- 172.21.15.105:0/4007295584 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3c4c15eae0 msgr2=0x7f3c4c0cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:23.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.292+0000 7f3c4a59c640 1 --2- 172.21.15.105:0/4007295584 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3c4c15eae0 0x7f3c4c0cd560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:23.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.292+0000 7f3c4a59c640 1 -- 172.21.15.105:0/4007295584 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c4c0cf350 msgr2=0x7f3c4c14c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:23.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.292+0000 7f3c4a59c640 1 --2- 172.21.15.105:0/4007295584 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c4c0cf350 0x7f3c4c14c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:23.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.292+0000 7f3c4a59c640 1 -- 172.21.15.105:0/4007295584 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c4c1f4e50 con 0x7f3c4c0d1c90 2024-04-15T15:19:23.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.293+0000 7f3c4a59c640 1 --2- 172.21.15.105:0/4007295584 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3c4c0d1c90 0x7f3c4c0cd020 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3c4000c7f0 tx=0x7f3c4000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:23.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.293+0000 7f3c337fe640 1 -- 172.21.15.105:0/4007295584 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c40015070 con 0x7f3c4c0d1c90 2024-04-15T15:19:23.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.293+0000 7f3c337fe640 1 -- 172.21.15.105:0/4007295584 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c40004240 con 0x7f3c4c0d1c90 2024-04-15T15:19:23.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.293+0000 7f3c50800640 1 -- 172.21.15.105:0/4007295584 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c4c1e9ea0 con 0x7f3c4c0d1c90 2024-04-15T15:19:23.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.293+0000 7f3c337fe640 1 -- 172.21.15.105:0/4007295584 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c40007620 con 0x7f3c4c0d1c90 2024-04-15T15:19:23.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.293+0000 7f3c50800640 1 -- 172.21.15.105:0/4007295584 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c4c14cf00 con 0x7f3c4c0d1c90 2024-04-15T15:19:23.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.294+0000 7f3c50800640 1 -- 172.21.15.105:0/4007295584 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c4c0b9d60 con 0x7f3c4c0d1c90 2024-04-15T15:19:23.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.294+0000 7f3c337fe640 1 -- 172.21.15.105:0/4007295584 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3c400077c0 con 0x7f3c4c0d1c90 2024-04-15T15:19:23.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.294+0000 7f3c337fe640 1 --2- 172.21.15.105:0/4007295584 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f3c3806c8f0 0x7f3c3806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:23.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.295+0000 7f3c337fe640 1 -- 172.21.15.105:0/4007295584 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f3c4008b660 con 0x7f3c4c0d1c90 2024-04-15T15:19:23.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.295+0000 7f3c4ad9d640 1 --2- 172.21.15.105:0/4007295584 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f3c3806c8f0 0x7f3c3806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:23.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.295+0000 7f3c4ad9d640 1 --2- 172.21.15.105:0/4007295584 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f3c3806c8f0 0x7f3c3806edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c00a730 tx=0x7f3c3c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:23.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.299+0000 7f3c337fe640 1 -- 172.21.15.105:0/4007295584 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3c4c0b9d60 con 0x7f3c4c0d1c90 2024-04-15T15:19:23.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.476+0000 7f3c50800640 1 -- 172.21.15.105:0/4007295584 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f3c4c068a40 con 0x7f3c4c0d1c90 2024-04-15T15:19:23.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.476+0000 7f3c337fe640 1 -- 172.21.15.105:0/4007295584 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7f3c40059de0 con 0x7f3c4c0d1c90 2024-04-15T15:19:23.478 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:19:23.478 INFO:teuthology.orchestra.run.smithi105.stdout:[ 2024-04-15T15:19:23.478 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-04-15T15:19:23.478 INFO:teuthology.orchestra.run.smithi105.stdout: "poolnum": 1, 2024-04-15T15:19:23.478 INFO:teuthology.orchestra.run.smithi105.stdout: "poolname": ".mgr" 2024-04-15T15:19:23.478 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-04-15T15:19:23.479 INFO:teuthology.orchestra.run.smithi105.stdout:] 2024-04-15T15:19:23.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.478+0000 7f3c50800640 1 -- 172.21.15.105:0/4007295584 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f3c3806c8f0 msgr2=0x7f3c3806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:23.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.478+0000 7f3c50800640 1 --2- 172.21.15.105:0/4007295584 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f3c3806c8f0 0x7f3c3806edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c00a730 tx=0x7f3c3c008040 comp rx=0 tx=0).stop 2024-04-15T15:19:23.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.478+0000 7f3c50800640 1 -- 172.21.15.105:0/4007295584 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3c4c0d1c90 msgr2=0x7f3c4c0cd020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:23.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.478+0000 7f3c50800640 1 --2- 172.21.15.105:0/4007295584 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3c4c0d1c90 0x7f3c4c0cd020 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3c4000c7f0 tx=0x7f3c4000ccb0 comp rx=0 tx=0).stop 2024-04-15T15:19:23.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.478+0000 7f3c50800640 1 -- 172.21.15.105:0/4007295584 shutdown_connections 2024-04-15T15:19:23.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.478+0000 7f3c50800640 1 --2- 172.21.15.105:0/4007295584 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3c4c15eae0 0x7f3c4c0cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:23.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.478+0000 7f3c50800640 1 --2- 172.21.15.105:0/4007295584 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f3c3806c8f0 0x7f3c3806edb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:23.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.478+0000 7f3c50800640 1 --2- 172.21.15.105:0/4007295584 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3c4c0d1c90 0x7f3c4c0cd020 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:23.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.478+0000 7f3c50800640 1 --2- 172.21.15.105:0/4007295584 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c4c0cf350 0x7f3c4c14c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:23.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.478+0000 7f3c50800640 1 -- 172.21.15.105:0/4007295584 >> 172.21.15.105:0/4007295584 conn(0x7f3c4c0b6560 msgr2=0x7f3c4c0b9650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:23.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.478+0000 7f3c50800640 1 -- 172.21.15.105:0/4007295584 shutdown_connections 2024-04-15T15:19:23.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.478+0000 7f3c50800640 1 -- 172.21.15.105:0/4007295584 wait complete. 2024-04-15T15:19:23.491 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-15T15:19:23.491 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-15T15:19:23.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.597+0000 7fbd30559640 1 Processor -- start 2024-04-15T15:19:23.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.597+0000 7fbd30559640 1 -- start start 2024-04-15T15:19:23.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.598+0000 7fbd30559640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbd2813e6f0 0x7fbd280cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:23.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.598+0000 7fbd30559640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbd280d0060 0x7fbd2806efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:23.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.598+0000 7fbd30559640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd2815f280 con 0x7fbd2813ecd0 2024-04-15T15:19:23.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.598+0000 7fbd30559640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fbd2815f560 con 0x7fbd280d0060 2024-04-15T15:19:23.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.598+0000 7fbd30559640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fbd2815f840 con 0x7fbd2813e6f0 2024-04-15T15:19:23.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.598+0000 7fbd2daf4640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbd280d0060 0x7fbd2806efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:23.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.598+0000 7fbd2daf4640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbd280d0060 0x7fbd2806efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57086/0 (socket says 172.21.15.105:57086) 2024-04-15T15:19:23.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.598+0000 7fbd2daf4640 1 -- 172.21.15.105:0/105428171 learned_addr learned my addr 172.21.15.105:0/105428171 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:23.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.598+0000 7fbd2e2f5640 1 --2- 172.21.15.105:0/105428171 >> v2:172.21.15.154:3300/0 conn(0x7fbd2813e6f0 0x7fbd280cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:23.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.598+0000 7fbd2daf4640 1 -- 172.21.15.105:0/105428171 >> v2:172.21.15.154:3300/0 conn(0x7fbd2813e6f0 msgr2=0x7fbd280cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:23.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.598+0000 7fbd2daf4640 1 --2- 172.21.15.105:0/105428171 >> v2:172.21.15.154:3300/0 conn(0x7fbd2813e6f0 0x7fbd280cfb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:23.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.598+0000 7fbd2daf4640 1 -- 172.21.15.105:0/105428171 >> v1:172.21.15.154:6789/0 conn(0x7fbd2813ecd0 legacy=0x7fbd28071eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:23.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.598+0000 7fbd2daf4640 1 -- 172.21.15.105:0/105428171 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd2815fb20 con 0x7fbd280d0060 2024-04-15T15:19:23.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.599+0000 7fbd2daf4640 1 --2- 172.21.15.105:0/105428171 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd280d0060 0x7fbd2806efc0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbd280703c0 tx=0x7fbd1802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=223307b9ffb51025 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:23.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.599+0000 7fbd2d2f3640 1 -- 172.21.15.105:0/105428171 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd18035020 con 0x7fbd280d0060 2024-04-15T15:19:23.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.599+0000 7fbd2d2f3640 1 -- 172.21.15.105:0/105428171 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd18031870 con 0x7fbd280d0060 2024-04-15T15:19:23.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.599+0000 7fbd2d2f3640 1 -- 172.21.15.105:0/105428171 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd18031c40 con 0x7fbd280d0060 2024-04-15T15:19:23.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.599+0000 7fbd30559640 1 -- 172.21.15.105:0/105428171 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd280d0060 msgr2=0x7fbd2806efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:23.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.599+0000 7fbd30559640 1 --2- 172.21.15.105:0/105428171 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd280d0060 0x7fbd2806efc0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbd280703c0 tx=0x7fbd1802b910 comp rx=0 tx=0).stop 2024-04-15T15:19:23.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.600+0000 7fbd30559640 1 -- 172.21.15.105:0/105428171 shutdown_connections 2024-04-15T15:19:23.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.600+0000 7fbd30559640 1 --2- 172.21.15.105:0/105428171 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd280d0060 0x7fbd2806efc0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:23.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.600+0000 7fbd30559640 1 --2- 172.21.15.105:0/105428171 >> v2:172.21.15.154:3300/0 conn(0x7fbd2813e6f0 0x7fbd280cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:23.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.600+0000 7fbd30559640 1 -- 172.21.15.105:0/105428171 >> 172.21.15.105:0/105428171 conn(0x7fbd28067e10 msgr2=0x7fbd280b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:23.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.600+0000 7fbd30559640 1 -- 172.21.15.105:0/105428171 shutdown_connections 2024-04-15T15:19:23.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.600+0000 7fbd30559640 1 -- 172.21.15.105:0/105428171 wait complete. 2024-04-15T15:19:23.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.601+0000 7fbd30559640 1 Processor -- start 2024-04-15T15:19:23.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.601+0000 7fbd30559640 1 -- start start 2024-04-15T15:19:23.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.601+0000 7fbd30559640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd280d0060 0x7fbd281367d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:23.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.601+0000 7fbd30559640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd2813e6f0 0x7fbd28134930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:23.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.601+0000 7fbd30559640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbd2813ecd0 0x7fbd28134e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:23.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.601+0000 7fbd30559640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbd280cf7f0 con 0x7fbd2813e6f0 2024-04-15T15:19:23.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.601+0000 7fbd30559640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbd280cf670 con 0x7fbd280d0060 2024-04-15T15:19:23.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.601+0000 7fbd30559640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fbd2806f500 con 0x7fbd2813ecd0 2024-04-15T15:19:23.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.601+0000 7fbd2daf4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd2813e6f0 0x7fbd28134930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:23.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.601+0000 7fbd2daf4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd2813e6f0 0x7fbd28134930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57102/0 (socket says 172.21.15.105:57102) 2024-04-15T15:19:23.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.601+0000 7fbd2daf4640 1 -- 172.21.15.105:0/3351867295 learned_addr learned my addr 172.21.15.105:0/3351867295 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:23.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.602+0000 7fbd2e2f5640 1 --2- 172.21.15.105:0/3351867295 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd280d0060 0x7fbd281367d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:23.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.602+0000 7fbd2eaf6640 1 --2- 172.21.15.105:0/3351867295 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbd2813ecd0 0x7fbd28134e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:23.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.602+0000 7fbd2daf4640 1 -- 172.21.15.105:0/3351867295 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbd2813ecd0 msgr2=0x7fbd28134e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:23.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.602+0000 7fbd2daf4640 1 --2- 172.21.15.105:0/3351867295 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbd2813ecd0 0x7fbd28134e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:23.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.602+0000 7fbd2daf4640 1 -- 172.21.15.105:0/3351867295 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd280d0060 msgr2=0x7fbd281367d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:23.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.602+0000 7fbd2daf4640 1 --2- 172.21.15.105:0/3351867295 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd280d0060 0x7fbd281367d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:23.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.602+0000 7fbd2daf4640 1 -- 172.21.15.105:0/3351867295 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd28155e60 con 0x7fbd2813e6f0 2024-04-15T15:19:23.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.602+0000 7fbd2eaf6640 1 --2- 172.21.15.105:0/3351867295 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbd2813ecd0 0x7fbd28134e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:19:23.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.602+0000 7fbd2daf4640 1 --2- 172.21.15.105:0/3351867295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd2813e6f0 0x7fbd28134930 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbd1802b950 tx=0x7fbd180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:23.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.602+0000 7fbd1effd640 1 -- 172.21.15.105:0/3351867295 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd18040070 con 0x7fbd2813e6f0 2024-04-15T15:19:23.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.602+0000 7fbd1effd640 1 -- 172.21.15.105:0/3351867295 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd18030d00 con 0x7fbd2813e6f0 2024-04-15T15:19:23.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.602+0000 7fbd30559640 1 -- 172.21.15.105:0/3351867295 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd281558a0 con 0x7fbd2813e6f0 2024-04-15T15:19:23.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.602+0000 7fbd1effd640 1 -- 172.21.15.105:0/3351867295 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd18035040 con 0x7fbd2813e6f0 2024-04-15T15:19:23.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.602+0000 7fbd30559640 1 -- 172.21.15.105:0/3351867295 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd281502f0 con 0x7fbd2813e6f0 2024-04-15T15:19:23.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.603+0000 7fbd30559640 1 -- 172.21.15.105:0/3351867295 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd28068a40 con 0x7fbd2813e6f0 2024-04-15T15:19:23.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.603+0000 7fbd1effd640 1 -- 172.21.15.105:0/3351867295 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbd18038020 con 0x7fbd2813e6f0 2024-04-15T15:19:23.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.604+0000 7fbd1effd640 1 --2- 172.21.15.105:0/3351867295 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fbd0c06c8f0 0x7fbd0c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:23.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.604+0000 7fbd1effd640 1 -- 172.21.15.105:0/3351867295 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fbd180ad4d0 con 0x7fbd2813e6f0 2024-04-15T15:19:23.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.604+0000 7fbd2e2f5640 1 --2- 172.21.15.105:0/3351867295 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fbd0c06c8f0 0x7fbd0c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:23.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.604+0000 7fbd2e2f5640 1 --2- 172.21.15.105:0/3351867295 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fbd0c06c8f0 0x7fbd0c06edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbd24006fd0 tx=0x7fbd24008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:23.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.608+0000 7fbd1effd640 1 -- 172.21.15.105:0/3351867295 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbd1807b690 con 0x7fbd2813e6f0 2024-04-15T15:19:23.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:23.785+0000 7fbd30559640 1 -- 172.21.15.105:0/3351867295 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fbd280b3800 con 0x7fbd2813e6f0 2024-04-15T15:19:24.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.399+0000 7fbd1effd640 1 -- 172.21.15.105:0/3351867295 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) ==== 138+0+0 (secure 0 0 0) 0x7fbd1807f990 con 0x7fbd2813e6f0 2024-04-15T15:19:24.401 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_metadata' created 2024-04-15T15:19:24.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.402+0000 7fbd30559640 1 -- 172.21.15.105:0/3351867295 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fbd0c06c8f0 msgr2=0x7fbd0c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:24.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.402+0000 7fbd30559640 1 --2- 172.21.15.105:0/3351867295 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fbd0c06c8f0 0x7fbd0c06edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbd24006fd0 tx=0x7fbd24008040 comp rx=0 tx=0).stop 2024-04-15T15:19:24.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.402+0000 7fbd30559640 1 -- 172.21.15.105:0/3351867295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd2813e6f0 msgr2=0x7fbd28134930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:24.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.402+0000 7fbd30559640 1 --2- 172.21.15.105:0/3351867295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd2813e6f0 0x7fbd28134930 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbd1802b950 tx=0x7fbd180046c0 comp rx=0 tx=0).stop 2024-04-15T15:19:24.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.402+0000 7fbd30559640 1 -- 172.21.15.105:0/3351867295 shutdown_connections 2024-04-15T15:19:24.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.402+0000 7fbd30559640 1 --2- 172.21.15.105:0/3351867295 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbd2813ecd0 0x7fbd28134e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:24.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.402+0000 7fbd30559640 1 --2- 172.21.15.105:0/3351867295 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fbd0c06c8f0 0x7fbd0c06edb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:24.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.402+0000 7fbd30559640 1 --2- 172.21.15.105:0/3351867295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd2813e6f0 0x7fbd28134930 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:24.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.402+0000 7fbd30559640 1 --2- 172.21.15.105:0/3351867295 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd280d0060 0x7fbd281367d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:24.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.402+0000 7fbd30559640 1 -- 172.21.15.105:0/3351867295 >> 172.21.15.105:0/3351867295 conn(0x7fbd28067e10 msgr2=0x7fbd2806b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:24.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.403+0000 7fbd30559640 1 -- 172.21.15.105:0/3351867295 shutdown_connections 2024-04-15T15:19:24.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.403+0000 7fbd30559640 1 -- 172.21.15.105:0/3351867295 wait complete. 2024-04-15T15:19:24.420 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-15T15:19:24.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.530+0000 7fa5df8e4640 1 Processor -- start 2024-04-15T15:19:24.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.530+0000 7fa5df8e4640 1 -- start start 2024-04-15T15:19:24.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.531+0000 7fa5df8e4640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa5d80d1c70 0x7fa5d80b6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:24.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.531+0000 7fa5df8e4640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa5d80d2ef0 0x7fa5d80d3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:24.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.531+0000 7fa5df8e4640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5d8162a10 con 0x7fa5d815eaf0 2024-04-15T15:19:24.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.531+0000 7fa5df8e4640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fa5d80b71b0 con 0x7fa5d80d1c70 2024-04-15T15:19:24.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.531+0000 7fa5df8e4640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fa5d8162cf0 con 0x7fa5d80d2ef0 2024-04-15T15:19:24.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.531+0000 7fa5dce7f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa5d80d1c70 0x7fa5d80b6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:24.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.531+0000 7fa5dce7f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa5d80d1c70 0x7fa5d80b6c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57118/0 (socket says 172.21.15.105:57118) 2024-04-15T15:19:24.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.531+0000 7fa5dce7f640 1 -- 172.21.15.105:0/520357421 learned_addr learned my addr 172.21.15.105:0/520357421 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:24.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.531+0000 7fa5dde81640 1 --2- 172.21.15.105:0/520357421 >> v2:172.21.15.154:3300/0 conn(0x7fa5d80d2ef0 0x7fa5d80d3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:24.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.531+0000 7fa5dce7f640 1 -- 172.21.15.105:0/520357421 >> v2:172.21.15.154:3300/0 conn(0x7fa5d80d2ef0 msgr2=0x7fa5d80d3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:24.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.531+0000 7fa5dce7f640 1 --2- 172.21.15.105:0/520357421 >> v2:172.21.15.154:3300/0 conn(0x7fa5d80d2ef0 0x7fa5d80d3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:24.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.531+0000 7fa5dce7f640 1 -- 172.21.15.105:0/520357421 >> v1:172.21.15.154:6789/0 conn(0x7fa5d815eaf0 legacy=0x7fa5d80d1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:24.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.532+0000 7fa5dce7f640 1 -- 172.21.15.105:0/520357421 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5d8162fd0 con 0x7fa5d80d1c70 2024-04-15T15:19:24.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.532+0000 7fa5dce7f640 1 --2- 172.21.15.105:0/520357421 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5d80d1c70 0x7fa5d80b6c70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa5d80ba8b0 tx=0x7fa5c802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=970c991c1bfffc93 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:24.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.532+0000 7fa5cffff640 1 -- 172.21.15.105:0/520357421 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c802e070 con 0x7fa5d80d1c70 2024-04-15T15:19:24.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.532+0000 7fa5cffff640 1 -- 172.21.15.105:0/520357421 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5c8037070 con 0x7fa5d80d1c70 2024-04-15T15:19:24.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.532+0000 7fa5cffff640 1 -- 172.21.15.105:0/520357421 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c8031a00 con 0x7fa5d80d1c70 2024-04-15T15:19:24.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.533+0000 7fa5df8e4640 1 -- 172.21.15.105:0/520357421 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5d80d1c70 msgr2=0x7fa5d80b6c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:24.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.533+0000 7fa5df8e4640 1 --2- 172.21.15.105:0/520357421 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5d80d1c70 0x7fa5d80b6c70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa5d80ba8b0 tx=0x7fa5c802b880 comp rx=0 tx=0).stop 2024-04-15T15:19:24.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.533+0000 7fa5df8e4640 1 -- 172.21.15.105:0/520357421 shutdown_connections 2024-04-15T15:19:24.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.533+0000 7fa5df8e4640 1 --2- 172.21.15.105:0/520357421 >> v2:172.21.15.154:3300/0 conn(0x7fa5d80d2ef0 0x7fa5d80d3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:24.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.533+0000 7fa5df8e4640 1 --2- 172.21.15.105:0/520357421 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5d80d1c70 0x7fa5d80b6c70 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:24.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.533+0000 7fa5df8e4640 1 -- 172.21.15.105:0/520357421 >> 172.21.15.105:0/520357421 conn(0x7fa5d806cf90 msgr2=0x7fa5d806d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:24.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.533+0000 7fa5df8e4640 1 -- 172.21.15.105:0/520357421 shutdown_connections 2024-04-15T15:19:24.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.533+0000 7fa5df8e4640 1 -- 172.21.15.105:0/520357421 wait complete. 2024-04-15T15:19:24.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.534+0000 7fa5df8e4640 1 Processor -- start 2024-04-15T15:19:24.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.534+0000 7fa5df8e4640 1 -- start start 2024-04-15T15:19:24.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.535+0000 7fa5df8e4640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa5d80d1c70 0x7fa5d81590e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:24.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.535+0000 7fa5df8e4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5d80d2ef0 0x7fa5d81617b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:24.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.535+0000 7fa5df8e4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa5d815eaf0 0x7fa5d80cd690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:24.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.535+0000 7fa5df8e4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d80b3d60 con 0x7fa5d80d2ef0 2024-04-15T15:19:24.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.535+0000 7fa5df8e4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d815eed0 con 0x7fa5d815eaf0 2024-04-15T15:19:24.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.535+0000 7fa5df8e4640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fa5d8162650 con 0x7fa5d80d1c70 2024-04-15T15:19:24.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.535+0000 7fa5dce7f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5d80d2ef0 0x7fa5d81617b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:24.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.535+0000 7fa5dce7f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5d80d2ef0 0x7fa5d81617b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57122/0 (socket says 172.21.15.105:57122) 2024-04-15T15:19:24.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.535+0000 7fa5dce7f640 1 -- 172.21.15.105:0/2663031996 learned_addr learned my addr 172.21.15.105:0/2663031996 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:24.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.535+0000 7fa5dde81640 1 --2- 172.21.15.105:0/2663031996 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa5d815eaf0 0x7fa5d80cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:24.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.535+0000 7fa5dd680640 1 --2- 172.21.15.105:0/2663031996 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa5d80d1c70 0x7fa5d81590e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:24.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.535+0000 7fa5dce7f640 1 -- 172.21.15.105:0/2663031996 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa5d80d1c70 msgr2=0x7fa5d81590e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:24.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.535+0000 7fa5dce7f640 1 --2- 172.21.15.105:0/2663031996 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa5d80d1c70 0x7fa5d81590e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:24.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.535+0000 7fa5dce7f640 1 -- 172.21.15.105:0/2663031996 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa5d815eaf0 msgr2=0x7fa5d80cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:24.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.535+0000 7fa5dce7f640 1 --2- 172.21.15.105:0/2663031996 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa5d815eaf0 0x7fa5d80cd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:24.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.535+0000 7fa5dce7f640 1 -- 172.21.15.105:0/2663031996 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5d815fa60 con 0x7fa5d80d2ef0 2024-04-15T15:19:24.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.535+0000 7fa5dde81640 1 --2- 172.21.15.105:0/2663031996 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa5d815eaf0 0x7fa5d80cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:19:24.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.535+0000 7fa5dce7f640 1 --2- 172.21.15.105:0/2663031996 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5d80d2ef0 0x7fa5d81617b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa5c802b8c0 tx=0x7fa5c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:24.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.536+0000 7fa5cdffb640 1 -- 172.21.15.105:0/2663031996 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c8032880 con 0x7fa5d80d2ef0 2024-04-15T15:19:24.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.536+0000 7fa5cdffb640 1 -- 172.21.15.105:0/2663031996 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5c8032a20 con 0x7fa5d80d2ef0 2024-04-15T15:19:24.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.536+0000 7fa5cdffb640 1 -- 172.21.15.105:0/2663031996 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c8032e10 con 0x7fa5d80d2ef0 2024-04-15T15:19:24.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.536+0000 7fa5df8e4640 1 -- 172.21.15.105:0/2663031996 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5d815fd40 con 0x7fa5d80d2ef0 2024-04-15T15:19:24.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.536+0000 7fa5df8e4640 1 -- 172.21.15.105:0/2663031996 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5d8160300 con 0x7fa5d80d2ef0 2024-04-15T15:19:24.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.537+0000 7fa5df8e4640 1 -- 172.21.15.105:0/2663031996 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5d80cfd90 con 0x7fa5d80d2ef0 2024-04-15T15:19:24.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.537+0000 7fa5cdffb640 1 -- 172.21.15.105:0/2663031996 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa5c80453f0 con 0x7fa5d80d2ef0 2024-04-15T15:19:24.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.538+0000 7fa5cdffb640 1 --2- 172.21.15.105:0/2663031996 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fa5bc06c8f0 0x7fa5bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:24.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.538+0000 7fa5cdffb640 1 -- 172.21.15.105:0/2663031996 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 5227+0+0 (secure 0 0 0) 0x7fa5c80adca0 con 0x7fa5d80d2ef0 2024-04-15T15:19:24.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.538+0000 7fa5dd680640 1 --2- 172.21.15.105:0/2663031996 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fa5bc06c8f0 0x7fa5bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:24.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.538+0000 7fa5dd680640 1 --2- 172.21.15.105:0/2663031996 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fa5bc06c8f0 0x7fa5bc06edb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa5d4006fd0 tx=0x7fa5d4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:24.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.542+0000 7fa5cdffb640 1 -- 172.21.15.105:0/2663031996 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa5d80cfd90 con 0x7fa5d80d2ef0 2024-04-15T15:19:24.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:24.718+0000 7fa5df8e4640 1 -- 172.21.15.105:0/2663031996 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fa5d80b6f30 con 0x7fa5d80d2ef0 2024-04-15T15:19:25.202 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-15T15:19:25.205 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-15T15:19:25.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.401+0000 7fa5cdffb640 1 -- 172.21.15.105:0/2663031996 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) ==== 170+0+0 (secure 0 0 0) 0x7fa5c8040070 con 0x7fa5d80d2ef0 2024-04-15T15:19:25.403 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_data' created 2024-04-15T15:19:25.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.404+0000 7fa5df8e4640 1 -- 172.21.15.105:0/2663031996 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fa5bc06c8f0 msgr2=0x7fa5bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:25.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.404+0000 7fa5df8e4640 1 --2- 172.21.15.105:0/2663031996 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fa5bc06c8f0 0x7fa5bc06edb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa5d4006fd0 tx=0x7fa5d4008040 comp rx=0 tx=0).stop 2024-04-15T15:19:25.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.404+0000 7fa5df8e4640 1 -- 172.21.15.105:0/2663031996 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5d80d2ef0 msgr2=0x7fa5d81617b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:25.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.404+0000 7fa5df8e4640 1 --2- 172.21.15.105:0/2663031996 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5d80d2ef0 0x7fa5d81617b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa5c802b8c0 tx=0x7fa5c80046c0 comp rx=0 tx=0).stop 2024-04-15T15:19:25.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.404+0000 7fa5df8e4640 1 -- 172.21.15.105:0/2663031996 shutdown_connections 2024-04-15T15:19:25.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.404+0000 7fa5df8e4640 1 --2- 172.21.15.105:0/2663031996 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fa5bc06c8f0 0x7fa5bc06edb0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:25.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.404+0000 7fa5df8e4640 1 --2- 172.21.15.105:0/2663031996 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa5d815eaf0 0x7fa5d80cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:25.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.404+0000 7fa5df8e4640 1 --2- 172.21.15.105:0/2663031996 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5d80d2ef0 0x7fa5d81617b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:25.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.404+0000 7fa5df8e4640 1 --2- 172.21.15.105:0/2663031996 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa5d80d1c70 0x7fa5d81590e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:25.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.404+0000 7fa5df8e4640 1 -- 172.21.15.105:0/2663031996 >> 172.21.15.105:0/2663031996 conn(0x7fa5d806cf90 msgr2=0x7fa5d80d4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:25.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.405+0000 7fa5df8e4640 1 -- 172.21.15.105:0/2663031996 shutdown_connections 2024-04-15T15:19:25.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.405+0000 7fa5df8e4640 1 -- 172.21.15.105:0/2663031996 wait complete. 2024-04-15T15:19:25.428 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-15T15:19:25.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.532+0000 7fdf28c37640 1 Processor -- start 2024-04-15T15:19:25.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.532+0000 7fdf28c37640 1 -- start start 2024-04-15T15:19:25.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.533+0000 7fdf28c37640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdf2415ec70 0x7fdf240b4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:25.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.533+0000 7fdf28c37640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf2406d180 con 0x7fdf240b1860 2024-04-15T15:19:25.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.533+0000 7fdf28c37640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf240d12e0 con 0x7fdf2415f050 2024-04-15T15:19:25.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.533+0000 7fdf28c37640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fdf2406d3f0 con 0x7fdf2415ec70 2024-04-15T15:19:25.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.533+0000 7fdf22d9d640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fdf240b1860 0x7fdf240cccd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:40294/0 (socket says 172.21.15.105:40294) 2024-04-15T15:19:25.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.533+0000 7fdf22d9d640 1 -- 172.21.15.105:0/3155230085 learned_addr learned my addr 172.21.15.105:0/3155230085 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:19:25.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.533+0000 7fdf2259c640 1 --2- 172.21.15.105:0/3155230085 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdf2415ec70 0x7fdf240b4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:25.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.533+0000 7fdf2159a640 1 -- 172.21.15.105:0/3155230085 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 219922334 0 0) 0x7fdf2406d180 con 0x7fdf240b1860 2024-04-15T15:19:25.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.533+0000 7fdf2159a640 1 -- 172.21.15.105:0/3155230085 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf0c003520 con 0x7fdf240b1860 2024-04-15T15:19:25.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.534+0000 7fdf2159a640 1 -- 172.21.15.105:0/3155230085 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1601814777 0 0) 0x7fdf240d12e0 con 0x7fdf2415f050 2024-04-15T15:19:25.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.534+0000 7fdf2159a640 1 -- 172.21.15.105:0/3155230085 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf2406d180 con 0x7fdf2415f050 2024-04-15T15:19:25.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.534+0000 7fdf2159a640 1 -- 172.21.15.105:0/3155230085 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3177936113 0 0) 0x7fdf0c003520 con 0x7fdf240b1860 2024-04-15T15:19:25.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.534+0000 7fdf2159a640 1 -- 172.21.15.105:0/3155230085 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdf2415ec70 msgr2=0x7fdf240b4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:25.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.534+0000 7fdf2159a640 1 --2- 172.21.15.105:0/3155230085 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdf2415ec70 0x7fdf240b4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:25.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.534+0000 7fdf2159a640 1 -- 172.21.15.105:0/3155230085 >> v1:172.21.15.154:6789/0 conn(0x7fdf2415f050 legacy=0x7fdf2406dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:25.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.534+0000 7fdf2159a640 1 -- 172.21.15.105:0/3155230085 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf2406d6d0 con 0x7fdf240b1860 2024-04-15T15:19:25.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.534+0000 7fdf2159a640 1 -- 172.21.15.105:0/3155230085 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7fdf1c0019f0 con 0x7fdf240b1860 2024-04-15T15:19:25.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.534+0000 7fdf2159a640 1 -- 172.21.15.105:0/3155230085 >> v1:172.21.15.105:6789/0 conn(0x7fdf240b1860 legacy=0x7fdf240cccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:25.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.534+0000 7fdf2159a640 1 --2- 172.21.15.105:0/3155230085 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdf2415ec70 0x7fdf0c003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:25.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.534+0000 7fdf2159a640 1 --2- 172.21.15.105:0/3155230085 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf0c004120 0x7fdf0c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:25.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.534+0000 7fdf2159a640 1 --2- 172.21.15.105:0/3155230085 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf0c004d40 0x7fdf0c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:25.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.534+0000 7fdf2159a640 1 -- 172.21.15.105:0/3155230085 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2406d3f0 con 0x7fdf0c004d40 2024-04-15T15:19:25.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.534+0000 7fdf2159a640 1 -- 172.21.15.105:0/3155230085 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0c00bb00 con 0x7fdf0c004120 2024-04-15T15:19:25.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.534+0000 7fdf2159a640 1 -- 172.21.15.105:0/3155230085 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fdf0c00bd00 con 0x7fdf2415ec70 2024-04-15T15:19:25.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.534+0000 7fdf22d9d640 1 --2- 172.21.15.105:0/3155230085 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf0c004d40 0x7fdf0c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:25.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.534+0000 7fdf2259c640 1 --2- 172.21.15.105:0/3155230085 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdf2415ec70 0x7fdf0c003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:25.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.534+0000 7fdf21d9b640 1 --2- 172.21.15.105:0/3155230085 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf0c004120 0x7fdf0c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:25.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.534+0000 7fdf22d9d640 1 -- 172.21.15.105:0/3155230085 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdf2415ec70 msgr2=0x7fdf0c003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:25.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.534+0000 7fdf22d9d640 1 --2- 172.21.15.105:0/3155230085 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdf2415ec70 0x7fdf0c003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:25.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.534+0000 7fdf22d9d640 1 -- 172.21.15.105:0/3155230085 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf0c004120 msgr2=0x7fdf0c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:25.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.534+0000 7fdf22d9d640 1 --2- 172.21.15.105:0/3155230085 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf0c004120 0x7fdf0c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:25.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.534+0000 7fdf22d9d640 1 -- 172.21.15.105:0/3155230085 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf0c003520 con 0x7fdf0c004d40 2024-04-15T15:19:25.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.535+0000 7fdf22d9d640 1 --2- 172.21.15.105:0/3155230085 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf0c004d40 0x7fdf0c0051c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdf24073210 tx=0x7fdf0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7efb46bfc9d87dff server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:25.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.535+0000 7fdf2159a640 1 -- 172.21.15.105:0/3155230085 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c00a370 con 0x7fdf0c004d40 2024-04-15T15:19:25.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.535+0000 7fdf2159a640 1 -- 172.21.15.105:0/3155230085 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf1c034650 con 0x7fdf0c004d40 2024-04-15T15:19:25.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.535+0000 7fdf2159a640 1 -- 172.21.15.105:0/3155230085 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c034a80 con 0x7fdf0c004d40 2024-04-15T15:19:25.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.535+0000 7fdf28c37640 1 -- 172.21.15.105:0/3155230085 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf0c004d40 msgr2=0x7fdf0c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:25.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.535+0000 7fdf28c37640 1 --2- 172.21.15.105:0/3155230085 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf0c004d40 0x7fdf0c0051c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdf24073210 tx=0x7fdf0c005830 comp rx=0 tx=0).stop 2024-04-15T15:19:25.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.535+0000 7fdf22d9d640 1 -- 172.21.15.105:0/3155230085 reap_dead start 2024-04-15T15:19:25.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.536+0000 7fdf28c37640 1 -- 172.21.15.105:0/3155230085 shutdown_connections 2024-04-15T15:19:25.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.536+0000 7fdf28c37640 1 -- 172.21.15.105:0/3155230085 >> 172.21.15.105:0/3155230085 conn(0x7fdf240b4170 msgr2=0x7fdf240b4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:25.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.536+0000 7fdf28c37640 1 -- 172.21.15.105:0/3155230085 shutdown_connections 2024-04-15T15:19:25.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.536+0000 7fdf28c37640 1 -- 172.21.15.105:0/3155230085 wait complete. 2024-04-15T15:19:25.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.537+0000 7fdf28c37640 1 Processor -- start 2024-04-15T15:19:25.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.537+0000 7fdf28c37640 1 -- start start 2024-04-15T15:19:25.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.538+0000 7fdf28c37640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf0c004d40 0x7fdf241f3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:25.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.538+0000 7fdf28c37640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdf2415dd60 0x7fdf2415e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:25.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.538+0000 7fdf28c37640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf241f7e70 0x7fdf241f8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:25.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.538+0000 7fdf28c37640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2406e480 con 0x7fdf0c004d40 2024-04-15T15:19:25.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.538+0000 7fdf28c37640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2406e300 con 0x7fdf241f7e70 2024-04-15T15:19:25.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.538+0000 7fdf28c37640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fdf240d15c0 con 0x7fdf2415dd60 2024-04-15T15:19:25.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.538+0000 7fdf2259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf0c004d40 0x7fdf241f3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:25.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.538+0000 7fdf2259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf0c004d40 0x7fdf241f3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57140/0 (socket says 172.21.15.105:57140) 2024-04-15T15:19:25.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.538+0000 7fdf2259c640 1 -- 172.21.15.105:0/1985937600 learned_addr learned my addr 172.21.15.105:0/1985937600 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:25.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.538+0000 7fdf21d9b640 1 --2- 172.21.15.105:0/1985937600 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdf2415dd60 0x7fdf2415e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:25.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.538+0000 7fdf22d9d640 1 --2- 172.21.15.105:0/1985937600 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf241f7e70 0x7fdf241f8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:25.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.538+0000 7fdf2259c640 1 -- 172.21.15.105:0/1985937600 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdf2415dd60 msgr2=0x7fdf2415e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:25.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.538+0000 7fdf2259c640 1 --2- 172.21.15.105:0/1985937600 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdf2415dd60 0x7fdf2415e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:25.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.538+0000 7fdf2259c640 1 -- 172.21.15.105:0/1985937600 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf241f7e70 msgr2=0x7fdf241f8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:25.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.538+0000 7fdf2259c640 1 --2- 172.21.15.105:0/1985937600 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf241f7e70 0x7fdf241f8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:25.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.538+0000 7fdf2259c640 1 -- 172.21.15.105:0/1985937600 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf24147520 con 0x7fdf0c004d40 2024-04-15T15:19:25.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.538+0000 7fdf2259c640 1 --2- 172.21.15.105:0/1985937600 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf0c004d40 0x7fdf241f3f30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdf140119a0 tx=0x7fdf14011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:25.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.538+0000 7fdf12ffd640 1 -- 172.21.15.105:0/1985937600 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1400faf0 con 0x7fdf0c004d40 2024-04-15T15:19:25.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.538+0000 7fdf12ffd640 1 -- 172.21.15.105:0/1985937600 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf1400fc90 con 0x7fdf0c004d40 2024-04-15T15:19:25.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.538+0000 7fdf12ffd640 1 -- 172.21.15.105:0/1985937600 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf14014440 con 0x7fdf0c004d40 2024-04-15T15:19:25.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.538+0000 7fdf28c37640 1 -- 172.21.15.105:0/1985937600 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf24146760 con 0x7fdf0c004d40 2024-04-15T15:19:25.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.538+0000 7fdf28c37640 1 -- 172.21.15.105:0/1985937600 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf24146d20 con 0x7fdf0c004d40 2024-04-15T15:19:25.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.539+0000 7fdf28c37640 1 -- 172.21.15.105:0/1985937600 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf2415e720 con 0x7fdf0c004d40 2024-04-15T15:19:25.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.540+0000 7fdf12ffd640 1 -- 172.21.15.105:0/1985937600 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdf14012040 con 0x7fdf0c004d40 2024-04-15T15:19:25.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.540+0000 7fdf12ffd640 1 --2- 172.21.15.105:0/1985937600 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fdf0406c8f0 0x7fdf0406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:25.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.540+0000 7fdf12ffd640 1 -- 172.21.15.105:0/1985937600 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5609+0+0 (secure 0 0 0) 0x7fdf1408bf70 con 0x7fdf0c004d40 2024-04-15T15:19:25.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.540+0000 7fdf21d9b640 1 --2- 172.21.15.105:0/1985937600 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fdf0406c8f0 0x7fdf0406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:25.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.541+0000 7fdf21d9b640 1 --2- 172.21.15.105:0/1985937600 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fdf0406c8f0 0x7fdf0406edb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdf2415e920 tx=0x7fdf18009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:25.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.545+0000 7fdf12ffd640 1 -- 172.21.15.105:0/1985937600 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdf14019030 con 0x7fdf0c004d40 2024-04-15T15:19:25.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:25.759+0000 7fdf28c37640 1 -- 172.21.15.105:0/1985937600 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fdf24068a40 con 0x7fdf0c004d40 2024-04-15T15:19:26.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.429+0000 7fdf12ffd640 1 -- 172.21.15.105:0/1985937600 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7fdf14059e40 con 0x7fdf0c004d40 2024-04-15T15:19:26.434 INFO:teuthology.orchestra.run.smithi105.stderr:new fs with metadata pool 2 and data pool 3 2024-04-15T15:19:26.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.435+0000 7fdf28c37640 1 -- 172.21.15.105:0/1985937600 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fdf0406c8f0 msgr2=0x7fdf0406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:26.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.435+0000 7fdf28c37640 1 --2- 172.21.15.105:0/1985937600 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fdf0406c8f0 0x7fdf0406edb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdf2415e920 tx=0x7fdf18009040 comp rx=0 tx=0).stop 2024-04-15T15:19:26.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.435+0000 7fdf28c37640 1 -- 172.21.15.105:0/1985937600 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf0c004d40 msgr2=0x7fdf241f3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:26.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.435+0000 7fdf28c37640 1 --2- 172.21.15.105:0/1985937600 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf0c004d40 0x7fdf241f3f30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdf140119a0 tx=0x7fdf14011e60 comp rx=0 tx=0).stop 2024-04-15T15:19:26.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.436+0000 7fdf28c37640 1 -- 172.21.15.105:0/1985937600 shutdown_connections 2024-04-15T15:19:26.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.436+0000 7fdf28c37640 1 --2- 172.21.15.105:0/1985937600 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdf2415dd60 0x7fdf2415e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:26.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.436+0000 7fdf28c37640 1 --2- 172.21.15.105:0/1985937600 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fdf0406c8f0 0x7fdf0406edb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:26.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.436+0000 7fdf28c37640 1 --2- 172.21.15.105:0/1985937600 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf241f7e70 0x7fdf241f8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:26.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.436+0000 7fdf28c37640 1 --2- 172.21.15.105:0/1985937600 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf0c004d40 0x7fdf241f3f30 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:26.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.436+0000 7fdf28c37640 1 -- 172.21.15.105:0/1985937600 >> 172.21.15.105:0/1985937600 conn(0x7fdf240b4170 msgr2=0x7fdf24070050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:26.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.436+0000 7fdf28c37640 1 -- 172.21.15.105:0/1985937600 shutdown_connections 2024-04-15T15:19:26.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.436+0000 7fdf28c37640 1 -- 172.21.15.105:0/1985937600 wait complete. 2024-04-15T15:19:26.455 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:19:26.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.557+0000 7fdb02d8d640 1 Processor -- start 2024-04-15T15:19:26.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.557+0000 7fdb02d8d640 1 -- start start 2024-04-15T15:19:26.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.558+0000 7fdb02d8d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdafc146380 0x7fdafc0afbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:26.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.558+0000 7fdb02d8d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fdafc0ce8e0 0x7fdafc0ced20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:26.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.558+0000 7fdb02d8d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdafc06b9c0 0x7fdafc06be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:26.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.558+0000 7fdb02d8d640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fdafc06c350 con 0x7fdafc06b9c0 2024-04-15T15:19:26.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.558+0000 7fdb02d8d640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fdafc0cffb0 con 0x7fdafc0ce8e0 2024-04-15T15:19:26.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.558+0000 7fdb02d8d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fdafc0cf3a0 con 0x7fdafc146380 2024-04-15T15:19:26.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.558+0000 7fdb0132a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdafc06b9c0 0x7fdafc06be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:26.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.558+0000 7fdb0132a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdafc06b9c0 0x7fdafc06be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57156/0 (socket says 172.21.15.105:57156) 2024-04-15T15:19:26.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.559+0000 7fdb0132a640 1 -- 172.21.15.105:0/4136641945 learned_addr learned my addr 172.21.15.105:0/4136641945 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:26.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.559+0000 7fdafbfff640 1 --2- 172.21.15.105:0/4136641945 >> v2:172.21.15.154:3300/0 conn(0x7fdafc0ce8e0 0x7fdafc0ced20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:26.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.559+0000 7fdb00b29640 1 --2- 172.21.15.105:0/4136641945 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdafc146380 0x7fdafc0afbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:26.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.559+0000 7fdb0132a640 1 -- 172.21.15.105:0/4136641945 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdafc146380 msgr2=0x7fdafc0afbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:26.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.559+0000 7fdb0132a640 1 --2- 172.21.15.105:0/4136641945 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdafc146380 0x7fdafc0afbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:26.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.559+0000 7fdb0132a640 1 -- 172.21.15.105:0/4136641945 >> v2:172.21.15.154:3300/0 conn(0x7fdafc0ce8e0 msgr2=0x7fdafc0ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:26.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.559+0000 7fdb0132a640 1 --2- 172.21.15.105:0/4136641945 >> v2:172.21.15.154:3300/0 conn(0x7fdafc0ce8e0 0x7fdafc0ced20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:26.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.559+0000 7fdb0132a640 1 -- 172.21.15.105:0/4136641945 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdafc0cf680 con 0x7fdafc06b9c0 2024-04-15T15:19:26.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.559+0000 7fdb0132a640 1 --2- 172.21.15.105:0/4136641945 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdafc06b9c0 0x7fdafc06be10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdafc0733f0 tx=0x7fdaf402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d39c05c1020ed656 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:26.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.560+0000 7fdafb7fe640 1 -- 172.21.15.105:0/4136641945 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf402c9e0 con 0x7fdafc06b9c0 2024-04-15T15:19:26.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.560+0000 7fdafb7fe640 1 -- 172.21.15.105:0/4136641945 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaf402cb80 con 0x7fdafc06b9c0 2024-04-15T15:19:26.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.560+0000 7fdafb7fe640 1 -- 172.21.15.105:0/4136641945 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf40343f0 con 0x7fdafc06b9c0 2024-04-15T15:19:26.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.560+0000 7fdb02d8d640 1 -- 172.21.15.105:0/4136641945 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdafc06b9c0 msgr2=0x7fdafc06be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:26.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.560+0000 7fdb02d8d640 1 --2- 172.21.15.105:0/4136641945 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdafc06b9c0 0x7fdafc06be10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdafc0733f0 tx=0x7fdaf402be40 comp rx=0 tx=0).stop 2024-04-15T15:19:26.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.560+0000 7fdb02d8d640 1 -- 172.21.15.105:0/4136641945 shutdown_connections 2024-04-15T15:19:26.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.560+0000 7fdb02d8d640 1 --2- 172.21.15.105:0/4136641945 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdafc06b9c0 0x7fdafc06be10 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:26.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.560+0000 7fdb02d8d640 1 --2- 172.21.15.105:0/4136641945 >> v2:172.21.15.154:3300/0 conn(0x7fdafc0ce8e0 0x7fdafc0ced20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:26.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.560+0000 7fdb02d8d640 1 --2- 172.21.15.105:0/4136641945 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdafc146380 0x7fdafc0afbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:26.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.560+0000 7fdb02d8d640 1 -- 172.21.15.105:0/4136641945 >> 172.21.15.105:0/4136641945 conn(0x7fdafc09aa40 msgr2=0x7fdafc0b6950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:26.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.560+0000 7fdb02d8d640 1 -- 172.21.15.105:0/4136641945 shutdown_connections 2024-04-15T15:19:26.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.560+0000 7fdb02d8d640 1 -- 172.21.15.105:0/4136641945 wait complete. 2024-04-15T15:19:26.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.561+0000 7fdb02d8d640 1 Processor -- start 2024-04-15T15:19:26.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.561+0000 7fdb02d8d640 1 -- start start 2024-04-15T15:19:26.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.562+0000 7fdb02d8d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdafc06b9c0 0x7fdafc13c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:26.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.562+0000 7fdb02d8d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdafc0ce8e0 0x7fdafc074a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:26.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.562+0000 7fdb02d8d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdafc146380 0x7fdafc074f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:26.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.562+0000 7fdb02d8d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdafc0b0130 con 0x7fdafc06b9c0 2024-04-15T15:19:26.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.562+0000 7fdb02d8d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdafc0cfc40 con 0x7fdafc0ce8e0 2024-04-15T15:19:26.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.562+0000 7fdb02d8d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fdafc0b7290 con 0x7fdafc146380 2024-04-15T15:19:26.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.562+0000 7fdb00b29640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdafc06b9c0 0x7fdafc13c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:26.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.562+0000 7fdb00b29640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdafc06b9c0 0x7fdafc13c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57162/0 (socket says 172.21.15.105:57162) 2024-04-15T15:19:26.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.562+0000 7fdb00b29640 1 -- 172.21.15.105:0/1315057446 learned_addr learned my addr 172.21.15.105:0/1315057446 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:26.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.563+0000 7fdafbfff640 1 --2- 172.21.15.105:0/1315057446 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdafc0ce8e0 0x7fdafc074a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:26.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.563+0000 7fdb0132a640 1 --2- 172.21.15.105:0/1315057446 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdafc146380 0x7fdafc074f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:26.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.563+0000 7fdb00b29640 1 -- 172.21.15.105:0/1315057446 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdafc146380 msgr2=0x7fdafc074f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:26.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.563+0000 7fdb00b29640 1 --2- 172.21.15.105:0/1315057446 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdafc146380 0x7fdafc074f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:26.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.563+0000 7fdb00b29640 1 -- 172.21.15.105:0/1315057446 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdafc0ce8e0 msgr2=0x7fdafc074a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:26.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.563+0000 7fdb00b29640 1 --2- 172.21.15.105:0/1315057446 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdafc0ce8e0 0x7fdafc074a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:26.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.563+0000 7fdb00b29640 1 -- 172.21.15.105:0/1315057446 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdafc13b970 con 0x7fdafc06b9c0 2024-04-15T15:19:26.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.563+0000 7fdafbfff640 1 --2- 172.21.15.105:0/1315057446 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdafc0ce8e0 0x7fdafc074a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:26.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.563+0000 7fdb0132a640 1 --2- 172.21.15.105:0/1315057446 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdafc146380 0x7fdafc074f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:26.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.563+0000 7fdb00b29640 1 --2- 172.21.15.105:0/1315057446 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdafc06b9c0 0x7fdafc13c3f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdaec00d800 tx=0x7fdaec00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:26.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.563+0000 7fdaf97fa640 1 -- 172.21.15.105:0/1315057446 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec00b6c0 con 0x7fdafc06b9c0 2024-04-15T15:19:26.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.563+0000 7fdaf97fa640 1 -- 172.21.15.105:0/1315057446 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaec019070 con 0x7fdafc06b9c0 2024-04-15T15:19:26.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.563+0000 7fdb02d8d640 1 -- 172.21.15.105:0/1315057446 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdafc139c90 con 0x7fdafc06b9c0 2024-04-15T15:19:26.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.563+0000 7fdaf97fa640 1 -- 172.21.15.105:0/1315057446 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec00ba80 con 0x7fdafc06b9c0 2024-04-15T15:19:26.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.563+0000 7fdb02d8d640 1 -- 172.21.15.105:0/1315057446 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdafc139f30 con 0x7fdafc06b9c0 2024-04-15T15:19:26.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.564+0000 7fdb02d8d640 1 -- 172.21.15.105:0/1315057446 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdafc069150 con 0x7fdafc06b9c0 2024-04-15T15:19:26.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.564+0000 7fdaf97fa640 1 -- 172.21.15.105:0/1315057446 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdaec00bc20 con 0x7fdafc06b9c0 2024-04-15T15:19:26.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.565+0000 7fdaf97fa640 1 --2- 172.21.15.105:0/1315057446 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fdae806c8f0 0x7fdae806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:26.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.565+0000 7fdaf97fa640 1 -- 172.21.15.105:0/1315057446 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7fdaec08c010 con 0x7fdafc06b9c0 2024-04-15T15:19:26.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.565+0000 7fdafbfff640 1 --2- 172.21.15.105:0/1315057446 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fdae806c8f0 0x7fdae806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:26.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.565+0000 7fdafbfff640 1 --2- 172.21.15.105:0/1315057446 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fdae806c8f0 0x7fdae806edb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdafc0b2cb0 tx=0x7fdaf0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:26.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.569+0000 7fdaf97fa640 1 -- 172.21.15.105:0/1315057446 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdafc13b970 con 0x7fdafc06b9c0 2024-04-15T15:19:26.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.744+0000 7fdb02d8d640 1 -- 172.21.15.105:0/1315057446 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdafc140930 con 0x7fdafc06b9c0 2024-04-15T15:19:26.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.745+0000 7fdaf97fa640 1 -- 172.21.15.105:0/1315057446 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16666 (secure 0 0 0) 0x7fdaec05a450 con 0x7fdafc06b9c0 2024-04-15T15:19:26.748 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:19:26.748 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":19,"fsid":"09e4d2a4-a324-4793-af77-644b25e520e1","created":"2024-04-15T15:19:01.524543+0000","modified":"2024-04-15T15:19:25.760397+0000","last_up_change":"2024-04-15T15:19:12.825815+0000","last_in_change":"2024-04-15T15:19:03.788126+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:19:13.748585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:19:23.789287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:19:24.722250+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6f6292b-2d7e-4aa0-9d1c-6ddd0f97287c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6801","nonce":2873040054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6803","nonce":2873040054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6807","nonce":2873040054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6805","nonce":2873040054}]},"public_addr":"172.21.15.105:6801/2873040054","cluster_addr":"172.21.15.105:6803/2873040054","heartbeat_back_addr":"172.21.15.105:6807/2873040054","heartbeat_front_addr":"172.21.15.105:6805/2873040054","state":["exists","up"]},{"osd":1,"uuid":"8900966a-b393-41e3-8f93-e5a43bd23622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6818","nonce":2494994479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6821","nonce":2494994479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6828","nonce":2494994479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6823","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6825","nonce":2494994479}]},"public_addr":"172.21.15.105:6818/2494994479","cluster_addr":"172.21.15.105:6821/2494994479","heartbeat_back_addr":"172.21.15.105:6828/2494994479","heartbeat_front_addr":"172.21.15.105:6825/2494994479","state":["exists","up"]},{"osd":2,"uuid":"81a5dc7a-9325-4ff5-941d-a2d5fba21f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6809","nonce":3798408699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6811","nonce":3798408699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6815","nonce":3798408699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6813","nonce":3798408699}]},"public_addr":"172.21.15.105:6809/3798408699","cluster_addr":"172.21.15.105:6811/3798408699","heartbeat_back_addr":"172.21.15.105:6815/3798408699","heartbeat_front_addr":"172.21.15.105:6813/3798408699","state":["exists","up"]},{"osd":3,"uuid":"df766ed2-5c71-4805-96f4-3d4970edc960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6819","nonce":1920720186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6824","nonce":1920720186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6831","nonce":1920720186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6827","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6829","nonce":1920720186}]},"public_addr":"172.21.15.105:6819/1920720186","cluster_addr":"172.21.15.105:6824/1920720186","heartbeat_back_addr":"172.21.15.105:6831/1920720186","heartbeat_front_addr":"172.21.15.105:6829/1920720186","state":["exists","up"]},{"osd":4,"uuid":"e1039e03-28b8-49df-b951-c91930b93c9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6810","nonce":3516494456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6813","nonce":3516494456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6820","nonce":3516494456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6817","nonce":3516494456}]},"public_addr":"172.21.15.154:6810/3516494456","cluster_addr":"172.21.15.154:6813/3516494456","heartbeat_back_addr":"172.21.15.154:6820/3516494456","heartbeat_front_addr":"172.21.15.154:6817/3516494456","state":["exists","up"]},{"osd":5,"uuid":"a9f5eaf1-66ff-405a-a9e4-ec3340812a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6801","nonce":540367751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6803","nonce":540367751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6807","nonce":540367751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6805","nonce":540367751}]},"public_addr":"172.21.15.154:6801/540367751","cluster_addr":"172.21.15.154:6803/540367751","heartbeat_back_addr":"172.21.15.154:6807/540367751","heartbeat_front_addr":"172.21.15.154:6805/540367751","state":["exists","up"]},{"osd":6,"uuid":"267d02ea-675f-428f-aae8-e0375e4d4feb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6825","nonce":1550829005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6827","nonce":1550829005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6831","nonce":1550829005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6829","nonce":1550829005}]},"public_addr":"172.21.15.154:6825/1550829005","cluster_addr":"172.21.15.154:6827/1550829005","heartbeat_back_addr":"172.21.15.154:6831/1550829005","heartbeat_front_addr":"172.21.15.154:6829/1550829005","state":["exists","up"]},{"osd":7,"uuid":"759ac6cc-896a-423c-885f-9fe90d8dcb14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6809","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6811","nonce":3920089903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6816","nonce":3920089903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6823","nonce":3920089903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6819","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6821","nonce":3920089903}]},"public_addr":"172.21.15.154:6811/3920089903","cluster_addr":"172.21.15.154:6816/3920089903","heartbeat_back_addr":"172.21.15.154:6823/3920089903","heartbeat_front_addr":"172.21.15.154:6821/3920089903","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.949509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.984072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.907278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.947391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.975447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.998896+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:09.028739+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.894119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:19:26.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.747+0000 7fdb02d8d640 1 -- 172.21.15.105:0/1315057446 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fdae806c8f0 msgr2=0x7fdae806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:26.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.747+0000 7fdb02d8d640 1 --2- 172.21.15.105:0/1315057446 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fdae806c8f0 0x7fdae806edb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdafc0b2cb0 tx=0x7fdaf0008040 comp rx=0 tx=0).stop 2024-04-15T15:19:26.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.747+0000 7fdb02d8d640 1 -- 172.21.15.105:0/1315057446 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdafc06b9c0 msgr2=0x7fdafc13c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:26.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.747+0000 7fdb02d8d640 1 --2- 172.21.15.105:0/1315057446 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdafc06b9c0 0x7fdafc13c3f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdaec00d800 tx=0x7fdaec00dcc0 comp rx=0 tx=0).stop 2024-04-15T15:19:26.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.747+0000 7fdb02d8d640 1 -- 172.21.15.105:0/1315057446 shutdown_connections 2024-04-15T15:19:26.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.747+0000 7fdb02d8d640 1 --2- 172.21.15.105:0/1315057446 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdafc146380 0x7fdafc074f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:26.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.747+0000 7fdb02d8d640 1 --2- 172.21.15.105:0/1315057446 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fdae806c8f0 0x7fdae806edb0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:26.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.747+0000 7fdb02d8d640 1 --2- 172.21.15.105:0/1315057446 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdafc0ce8e0 0x7fdafc074a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:26.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.747+0000 7fdb02d8d640 1 --2- 172.21.15.105:0/1315057446 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdafc06b9c0 0x7fdafc13c3f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:26.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.747+0000 7fdb02d8d640 1 -- 172.21.15.105:0/1315057446 >> 172.21.15.105:0/1315057446 conn(0x7fdafc09aa40 msgr2=0x7fdafc0d0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:26.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.748+0000 7fdb02d8d640 1 -- 172.21.15.105:0/1315057446 shutdown_connections 2024-04-15T15:19:26.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.748+0000 7fdb02d8d640 1 -- 172.21.15.105:0/1315057446 wait complete. 2024-04-15T15:19:26.761 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:19:26.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.861+0000 7fc00099d640 1 Processor -- start 2024-04-15T15:19:26.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.861+0000 7fc00099d640 1 -- start start 2024-04-15T15:19:26.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.862+0000 7fc00099d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbffc15d6a0 0x7fbffc0d10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:26.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.862+0000 7fc00099d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbffc09b790 0x7fbffc0ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:26.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.862+0000 7fc00099d640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbffc0d02e0 con 0x7fbffc09aa40 2024-04-15T15:19:26.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.862+0000 7fc00099d640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fbffc15a640 con 0x7fbffc15d6a0 2024-04-15T15:19:26.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.862+0000 7fc00099d640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fbffc0d05c0 con 0x7fbffc09b790 2024-04-15T15:19:26.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.862+0000 7fbff9d9b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbffc15d6a0 0x7fbffc0d10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:26.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.862+0000 7fbff9d9b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbffc15d6a0 0x7fbffc0d10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57168/0 (socket says 172.21.15.105:57168) 2024-04-15T15:19:26.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.862+0000 7fbff9d9b640 1 -- 172.21.15.105:0/756590627 learned_addr learned my addr 172.21.15.105:0/756590627 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:26.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.862+0000 7fbffad9d640 1 --2- 172.21.15.105:0/756590627 >> v2:172.21.15.154:3300/0 conn(0x7fbffc09b790 0x7fbffc0ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:26.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.862+0000 7fbff9d9b640 1 -- 172.21.15.105:0/756590627 >> v2:172.21.15.154:3300/0 conn(0x7fbffc09b790 msgr2=0x7fbffc0ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:26.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.862+0000 7fbff9d9b640 1 --2- 172.21.15.105:0/756590627 >> v2:172.21.15.154:3300/0 conn(0x7fbffc09b790 0x7fbffc0ccfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:26.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.862+0000 7fbff9d9b640 1 -- 172.21.15.105:0/756590627 >> v1:172.21.15.154:6789/0 conn(0x7fbffc09aa40 legacy=0x7fbffc09ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:26.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.862+0000 7fbff9d9b640 1 -- 172.21.15.105:0/756590627 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbffc0d08a0 con 0x7fbffc15d6a0 2024-04-15T15:19:26.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.863+0000 7fbff9d9b640 1 --2- 172.21.15.105:0/756590627 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbffc15d6a0 0x7fbffc0d10a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbffc09bf80 tx=0x7fbff002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3e12889d6d76da8 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:26.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.863+0000 7fbff959a640 1 -- 172.21.15.105:0/756590627 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff0034070 con 0x7fbffc15d6a0 2024-04-15T15:19:26.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.863+0000 7fbff959a640 1 -- 172.21.15.105:0/756590627 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbff0030860 con 0x7fbffc15d6a0 2024-04-15T15:19:26.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.863+0000 7fbff959a640 1 -- 172.21.15.105:0/756590627 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff0030c30 con 0x7fbffc15d6a0 2024-04-15T15:19:26.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.864+0000 7fc00099d640 1 -- 172.21.15.105:0/756590627 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbffc15d6a0 msgr2=0x7fbffc0d10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:26.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.864+0000 7fc00099d640 1 --2- 172.21.15.105:0/756590627 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbffc15d6a0 0x7fbffc0d10a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbffc09bf80 tx=0x7fbff002b9a0 comp rx=0 tx=0).stop 2024-04-15T15:19:26.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.864+0000 7fc00099d640 1 -- 172.21.15.105:0/756590627 shutdown_connections 2024-04-15T15:19:26.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.864+0000 7fc00099d640 1 --2- 172.21.15.105:0/756590627 >> v2:172.21.15.154:3300/0 conn(0x7fbffc09b790 0x7fbffc0ccfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:26.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.864+0000 7fc00099d640 1 --2- 172.21.15.105:0/756590627 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbffc15d6a0 0x7fbffc0d10a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:26.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.864+0000 7fc00099d640 1 -- 172.21.15.105:0/756590627 >> 172.21.15.105:0/756590627 conn(0x7fbffc0d2570 msgr2=0x7fbffc0d2980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:26.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.864+0000 7fc00099d640 1 -- 172.21.15.105:0/756590627 shutdown_connections 2024-04-15T15:19:26.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.864+0000 7fc00099d640 1 -- 172.21.15.105:0/756590627 wait complete. 2024-04-15T15:19:26.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.865+0000 7fc00099d640 1 Processor -- start 2024-04-15T15:19:26.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.865+0000 7fc00099d640 1 -- start start 2024-04-15T15:19:26.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.866+0000 7fc00099d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbffc09aa40 0x7fbffc0b6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:26.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.866+0000 7fc00099d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbffc09b790 0x7fbffc15dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:26.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.866+0000 7fc00099d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbffc15d6a0 0x7fbffc15e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:26.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.866+0000 7fc00099d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc0cd520 con 0x7fbffc09aa40 2024-04-15T15:19:26.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.866+0000 7fc00099d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc0ceca0 con 0x7fbffc15d6a0 2024-04-15T15:19:26.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.866+0000 7fc00099d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fbffc0cd6a0 con 0x7fbffc09b790 2024-04-15T15:19:26.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.866+0000 7fbffa59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbffc09aa40 0x7fbffc0b6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:26.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.866+0000 7fbffa59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbffc09aa40 0x7fbffc0b6410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57170/0 (socket says 172.21.15.105:57170) 2024-04-15T15:19:26.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.866+0000 7fbffa59c640 1 -- 172.21.15.105:0/466564972 learned_addr learned my addr 172.21.15.105:0/466564972 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:26.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.866+0000 7fbff9d9b640 1 --2- 172.21.15.105:0/466564972 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbffc09b790 0x7fbffc15dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:26.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.866+0000 7fbffad9d640 1 --2- 172.21.15.105:0/466564972 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbffc15d6a0 0x7fbffc15e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:26.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.866+0000 7fbffa59c640 1 -- 172.21.15.105:0/466564972 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbffc09b790 msgr2=0x7fbffc15dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:26.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.866+0000 7fbffa59c640 1 --2- 172.21.15.105:0/466564972 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbffc09b790 0x7fbffc15dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:26.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.866+0000 7fbffa59c640 1 -- 172.21.15.105:0/466564972 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbffc15d6a0 msgr2=0x7fbffc15e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:26.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.866+0000 7fbffa59c640 1 --2- 172.21.15.105:0/466564972 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbffc15d6a0 0x7fbffc15e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:26.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.866+0000 7fbffa59c640 1 -- 172.21.15.105:0/466564972 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbffc0b6d10 con 0x7fbffc09aa40 2024-04-15T15:19:26.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.866+0000 7fbffad9d640 1 --2- 172.21.15.105:0/466564972 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbffc15d6a0 0x7fbffc15e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:26.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.866+0000 7fbffa59c640 1 --2- 172.21.15.105:0/466564972 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbffc09aa40 0x7fbffc0b6410 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbfec00c7c0 tx=0x7fbfec00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:26.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.866+0000 7fbfeaffd640 1 -- 172.21.15.105:0/466564972 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfec016020 con 0x7fbffc09aa40 2024-04-15T15:19:26.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.866+0000 7fbfeaffd640 1 -- 172.21.15.105:0/466564972 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfec006d50 con 0x7fbffc09aa40 2024-04-15T15:19:26.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.866+0000 7fc00099d640 1 -- 172.21.15.105:0/466564972 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbffc0b6ff0 con 0x7fbffc09aa40 2024-04-15T15:19:26.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.866+0000 7fbfeaffd640 1 -- 172.21.15.105:0/466564972 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfec003bb0 con 0x7fbffc09aa40 2024-04-15T15:19:26.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.867+0000 7fc00099d640 1 -- 172.21.15.105:0/466564972 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbffc0b75b0 con 0x7fbffc09aa40 2024-04-15T15:19:26.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.868+0000 7fc00099d640 1 -- 172.21.15.105:0/466564972 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbffc068a40 con 0x7fbffc09aa40 2024-04-15T15:19:26.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.868+0000 7fbfeaffd640 1 -- 172.21.15.105:0/466564972 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbfec00a850 con 0x7fbffc09aa40 2024-04-15T15:19:26.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.868+0000 7fbfeaffd640 1 --2- 172.21.15.105:0/466564972 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fbfdc06c8f0 0x7fbfdc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:26.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.869+0000 7fbfeaffd640 1 -- 172.21.15.105:0/466564972 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7fbfec08b7e0 con 0x7fbffc09aa40 2024-04-15T15:19:26.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.869+0000 7fbff9d9b640 1 --2- 172.21.15.105:0/466564972 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fbfdc06c8f0 0x7fbfdc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:26.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.869+0000 7fbff9d9b640 1 --2- 172.21.15.105:0/466564972 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fbfdc06c8f0 0x7fbfdc06edb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbff0036000 tx=0x7fbff003b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:26.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:26.873+0000 7fbfeaffd640 1 -- 172.21.15.105:0/466564972 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbfec020020 con 0x7fbffc09aa40 2024-04-15T15:19:27.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.051+0000 7fc00099d640 1 -- 172.21.15.105:0/466564972 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbffc09b570 con 0x7fbffc09aa40 2024-04-15T15:19:27.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.052+0000 7fbfeaffd640 1 -- 172.21.15.105:0/466564972 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16666 (secure 0 0 0) 0x7fbfec019310 con 0x7fbffc09aa40 2024-04-15T15:19:27.054 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:19:27.054 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":19,"fsid":"09e4d2a4-a324-4793-af77-644b25e520e1","created":"2024-04-15T15:19:01.524543+0000","modified":"2024-04-15T15:19:25.760397+0000","last_up_change":"2024-04-15T15:19:12.825815+0000","last_in_change":"2024-04-15T15:19:03.788126+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:19:13.748585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:19:23.789287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:19:24.722250+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6f6292b-2d7e-4aa0-9d1c-6ddd0f97287c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6801","nonce":2873040054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6803","nonce":2873040054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6807","nonce":2873040054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6805","nonce":2873040054}]},"public_addr":"172.21.15.105:6801/2873040054","cluster_addr":"172.21.15.105:6803/2873040054","heartbeat_back_addr":"172.21.15.105:6807/2873040054","heartbeat_front_addr":"172.21.15.105:6805/2873040054","state":["exists","up"]},{"osd":1,"uuid":"8900966a-b393-41e3-8f93-e5a43bd23622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6818","nonce":2494994479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6821","nonce":2494994479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6828","nonce":2494994479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6823","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6825","nonce":2494994479}]},"public_addr":"172.21.15.105:6818/2494994479","cluster_addr":"172.21.15.105:6821/2494994479","heartbeat_back_addr":"172.21.15.105:6828/2494994479","heartbeat_front_addr":"172.21.15.105:6825/2494994479","state":["exists","up"]},{"osd":2,"uuid":"81a5dc7a-9325-4ff5-941d-a2d5fba21f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6809","nonce":3798408699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6811","nonce":3798408699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6815","nonce":3798408699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6813","nonce":3798408699}]},"public_addr":"172.21.15.105:6809/3798408699","cluster_addr":"172.21.15.105:6811/3798408699","heartbeat_back_addr":"172.21.15.105:6815/3798408699","heartbeat_front_addr":"172.21.15.105:6813/3798408699","state":["exists","up"]},{"osd":3,"uuid":"df766ed2-5c71-4805-96f4-3d4970edc960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6819","nonce":1920720186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6824","nonce":1920720186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6831","nonce":1920720186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6827","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6829","nonce":1920720186}]},"public_addr":"172.21.15.105:6819/1920720186","cluster_addr":"172.21.15.105:6824/1920720186","heartbeat_back_addr":"172.21.15.105:6831/1920720186","heartbeat_front_addr":"172.21.15.105:6829/1920720186","state":["exists","up"]},{"osd":4,"uuid":"e1039e03-28b8-49df-b951-c91930b93c9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6810","nonce":3516494456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6813","nonce":3516494456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6820","nonce":3516494456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6817","nonce":3516494456}]},"public_addr":"172.21.15.154:6810/3516494456","cluster_addr":"172.21.15.154:6813/3516494456","heartbeat_back_addr":"172.21.15.154:6820/3516494456","heartbeat_front_addr":"172.21.15.154:6817/3516494456","state":["exists","up"]},{"osd":5,"uuid":"a9f5eaf1-66ff-405a-a9e4-ec3340812a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6801","nonce":540367751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6803","nonce":540367751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6807","nonce":540367751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6805","nonce":540367751}]},"public_addr":"172.21.15.154:6801/540367751","cluster_addr":"172.21.15.154:6803/540367751","heartbeat_back_addr":"172.21.15.154:6807/540367751","heartbeat_front_addr":"172.21.15.154:6805/540367751","state":["exists","up"]},{"osd":6,"uuid":"267d02ea-675f-428f-aae8-e0375e4d4feb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6825","nonce":1550829005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6827","nonce":1550829005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6831","nonce":1550829005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6829","nonce":1550829005}]},"public_addr":"172.21.15.154:6825/1550829005","cluster_addr":"172.21.15.154:6827/1550829005","heartbeat_back_addr":"172.21.15.154:6831/1550829005","heartbeat_front_addr":"172.21.15.154:6829/1550829005","state":["exists","up"]},{"osd":7,"uuid":"759ac6cc-896a-423c-885f-9fe90d8dcb14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6809","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6811","nonce":3920089903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6816","nonce":3920089903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6823","nonce":3920089903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6819","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6821","nonce":3920089903}]},"public_addr":"172.21.15.154:6811/3920089903","cluster_addr":"172.21.15.154:6816/3920089903","heartbeat_back_addr":"172.21.15.154:6823/3920089903","heartbeat_front_addr":"172.21.15.154:6821/3920089903","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.949509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.984072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.907278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.947391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.975447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.998896+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:09.028739+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.894119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:19:27.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.054+0000 7fc00099d640 1 -- 172.21.15.105:0/466564972 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fbfdc06c8f0 msgr2=0x7fbfdc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:27.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.054+0000 7fc00099d640 1 --2- 172.21.15.105:0/466564972 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fbfdc06c8f0 0x7fbfdc06edb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbff0036000 tx=0x7fbff003b040 comp rx=0 tx=0).stop 2024-04-15T15:19:27.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.054+0000 7fc00099d640 1 -- 172.21.15.105:0/466564972 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbffc09aa40 msgr2=0x7fbffc0b6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:27.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.054+0000 7fc00099d640 1 --2- 172.21.15.105:0/466564972 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbffc09aa40 0x7fbffc0b6410 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbfec00c7c0 tx=0x7fbfec00cc80 comp rx=0 tx=0).stop 2024-04-15T15:19:27.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.054+0000 7fc00099d640 1 -- 172.21.15.105:0/466564972 shutdown_connections 2024-04-15T15:19:27.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.054+0000 7fc00099d640 1 --2- 172.21.15.105:0/466564972 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbffc09b790 0x7fbffc15dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:27.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.054+0000 7fc00099d640 1 --2- 172.21.15.105:0/466564972 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fbfdc06c8f0 0x7fbfdc06edb0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:27.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.054+0000 7fc00099d640 1 --2- 172.21.15.105:0/466564972 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbffc15d6a0 0x7fbffc15e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:27.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.054+0000 7fc00099d640 1 --2- 172.21.15.105:0/466564972 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbffc09aa40 0x7fbffc0b6410 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:27.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.054+0000 7fc00099d640 1 -- 172.21.15.105:0/466564972 >> 172.21.15.105:0/466564972 conn(0x7fbffc0d2570 msgr2=0x7fbffc09ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:27.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.054+0000 7fc00099d640 1 -- 172.21.15.105:0/466564972 shutdown_connections 2024-04-15T15:19:27.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.054+0000 7fc00099d640 1 -- 172.21.15.105:0/466564972 wait complete. 2024-04-15T15:19:27.067 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-15T15:19:27.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.171+0000 7fb542b0e640 1 Processor -- start 2024-04-15T15:19:27.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.171+0000 7fb542b0e640 1 -- start start 2024-04-15T15:19:27.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.172+0000 7fb542b0e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb53c06b9c0 0x7fb53c0cba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:27.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.172+0000 7fb542b0e640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb53c0cf150 con 0x7fb53c0d07c0 2024-04-15T15:19:27.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.172+0000 7fb542b0e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb53c06c030 con 0x7fb53c0cd5a0 2024-04-15T15:19:27.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.172+0000 7fb542b0e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fb53c06c310 con 0x7fb53c06b9c0 2024-04-15T15:19:27.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.172+0000 7fb5408aa640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fb53c0d07c0 0x7fb53c0d0ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:40304/0 (socket says 172.21.15.105:40304) 2024-04-15T15:19:27.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.172+0000 7fb5408aa640 1 -- 172.21.15.105:0/1762054332 learned_addr learned my addr 172.21.15.105:0/1762054332 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:19:27.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.172+0000 7fb53b7fe640 1 -- 172.21.15.105:0/1762054332 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 398054752 0 0) 0x7fb53c0cf150 con 0x7fb53c0d07c0 2024-04-15T15:19:27.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.172+0000 7fb5410ab640 1 --2- 172.21.15.105:0/1762054332 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb53c06b9c0 0x7fb53c0cba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:27.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.172+0000 7fb53b7fe640 1 -- 172.21.15.105:0/1762054332 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb524003520 con 0x7fb53c0d07c0 2024-04-15T15:19:27.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.172+0000 7fb53b7fe640 1 -- 172.21.15.105:0/1762054332 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3119491493 0 0) 0x7fb524003520 con 0x7fb53c0d07c0 2024-04-15T15:19:27.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.172+0000 7fb53b7fe640 1 -- 172.21.15.105:0/1762054332 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb53c06b9c0 msgr2=0x7fb53c0cba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:27.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.172+0000 7fb53b7fe640 1 --2- 172.21.15.105:0/1762054332 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb53c06b9c0 0x7fb53c0cba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:27.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.172+0000 7fb53b7fe640 1 -- 172.21.15.105:0/1762054332 >> v1:172.21.15.154:6789/0 conn(0x7fb53c0cd5a0 legacy=0x7fb53c0ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:27.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.172+0000 7fb53b7fe640 1 -- 172.21.15.105:0/1762054332 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb53c06c5f0 con 0x7fb53c0d07c0 2024-04-15T15:19:27.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.172+0000 7fb53b7fe640 1 -- 172.21.15.105:0/1762054332 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7fb5340019f0 con 0x7fb53c0d07c0 2024-04-15T15:19:27.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.172+0000 7fb53b7fe640 1 -- 172.21.15.105:0/1762054332 >> v1:172.21.15.105:6789/0 conn(0x7fb53c0d07c0 legacy=0x7fb53c0d0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:27.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.173+0000 7fb53b7fe640 1 --2- 172.21.15.105:0/1762054332 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5240039e0 0x7fb524003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:27.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.173+0000 7fb53b7fe640 1 --2- 172.21.15.105:0/1762054332 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb53c06b9c0 0x7fb524004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:27.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.173+0000 7fb53b7fe640 1 --2- 172.21.15.105:0/1762054332 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb524004d40 0x7fb5240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:27.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.173+0000 7fb53b7fe640 1 -- 172.21.15.105:0/1762054332 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb53c06c310 con 0x7fb5240039e0 2024-04-15T15:19:27.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.173+0000 7fb53b7fe640 1 -- 172.21.15.105:0/1762054332 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb52400bb30 con 0x7fb524004d40 2024-04-15T15:19:27.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.173+0000 7fb53b7fe640 1 -- 172.21.15.105:0/1762054332 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fb52400bd30 con 0x7fb53c06b9c0 2024-04-15T15:19:27.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.173+0000 7fb5408aa640 1 --2- 172.21.15.105:0/1762054332 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5240039e0 0x7fb524003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:27.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.173+0000 7fb53bfff640 1 --2- 172.21.15.105:0/1762054332 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb53c06b9c0 0x7fb524004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:27.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.173+0000 7fb5410ab640 1 --2- 172.21.15.105:0/1762054332 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb524004d40 0x7fb5240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:27.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.173+0000 7fb5408aa640 1 -- 172.21.15.105:0/1762054332 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb53c06b9c0 msgr2=0x7fb524004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:27.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.173+0000 7fb5408aa640 1 --2- 172.21.15.105:0/1762054332 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb53c06b9c0 0x7fb524004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:27.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.173+0000 7fb5408aa640 1 -- 172.21.15.105:0/1762054332 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb524004d40 msgr2=0x7fb5240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:27.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.173+0000 7fb5408aa640 1 --2- 172.21.15.105:0/1762054332 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb524004d40 0x7fb5240051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:27.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.173+0000 7fb5408aa640 1 -- 172.21.15.105:0/1762054332 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb524003520 con 0x7fb5240039e0 2024-04-15T15:19:27.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.174+0000 7fb5408aa640 1 --2- 172.21.15.105:0/1762054332 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5240039e0 0x7fb524003e00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb5340013d0 tx=0x7fb524005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef3b4b977ed627d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:27.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.174+0000 7fb53b7fe640 1 -- 172.21.15.105:0/1762054332 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb53400a4a0 con 0x7fb5240039e0 2024-04-15T15:19:27.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.174+0000 7fb53b7fe640 1 -- 172.21.15.105:0/1762054332 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb534034630 con 0x7fb5240039e0 2024-04-15T15:19:27.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.174+0000 7fb53b7fe640 1 -- 172.21.15.105:0/1762054332 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb534034a00 con 0x7fb5240039e0 2024-04-15T15:19:27.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.174+0000 7fb542b0e640 1 -- 172.21.15.105:0/1762054332 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5240039e0 msgr2=0x7fb524003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:27.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.174+0000 7fb542b0e640 1 --2- 172.21.15.105:0/1762054332 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5240039e0 0x7fb524003e00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb5340013d0 tx=0x7fb524005830 comp rx=0 tx=0).stop 2024-04-15T15:19:27.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.174+0000 7fb5410ab640 1 -- 172.21.15.105:0/1762054332 reap_dead start 2024-04-15T15:19:27.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.174+0000 7fb542b0e640 1 -- 172.21.15.105:0/1762054332 shutdown_connections 2024-04-15T15:19:27.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.174+0000 7fb542b0e640 1 -- 172.21.15.105:0/1762054332 >> 172.21.15.105:0/1762054332 conn(0x7fb53c0b2fc0 msgr2=0x7fb53c0b33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:27.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.174+0000 7fb542b0e640 1 -- 172.21.15.105:0/1762054332 shutdown_connections 2024-04-15T15:19:27.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.174+0000 7fb542b0e640 1 -- 172.21.15.105:0/1762054332 wait complete. 2024-04-15T15:19:27.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.175+0000 7fb542b0e640 1 Processor -- start 2024-04-15T15:19:27.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.176+0000 7fb542b0e640 1 -- start start 2024-04-15T15:19:27.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.176+0000 7fb542b0e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb5240039e0 0x7fb53c1e9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:27.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.176+0000 7fb542b0e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb53c0b6a10 0x7fb53c14e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:27.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.176+0000 7fb542b0e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb53c0b6e90 0x7fb53c14e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:27.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.176+0000 7fb542b0e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb53c0cec30 con 0x7fb53c0b6a10 2024-04-15T15:19:27.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.176+0000 7fb542b0e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb53c0ce000 con 0x7fb5240039e0 2024-04-15T15:19:27.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.176+0000 7fb542b0e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fb53c0cba80 con 0x7fb53c0b6e90 2024-04-15T15:19:27.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.176+0000 7fb53bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb53c0b6a10 0x7fb53c14e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:27.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.176+0000 7fb53bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb53c0b6a10 0x7fb53c14e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57186/0 (socket says 172.21.15.105:57186) 2024-04-15T15:19:27.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.176+0000 7fb53bfff640 1 -- 172.21.15.105:0/1455005084 learned_addr learned my addr 172.21.15.105:0/1455005084 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:27.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.177+0000 7fb5410ab640 1 --2- 172.21.15.105:0/1455005084 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb53c0b6e90 0x7fb53c14e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:27.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.177+0000 7fb5408aa640 1 --2- 172.21.15.105:0/1455005084 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb5240039e0 0x7fb53c1e9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:27.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.177+0000 7fb53bfff640 1 -- 172.21.15.105:0/1455005084 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb53c0b6e90 msgr2=0x7fb53c14e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:27.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.177+0000 7fb53bfff640 1 --2- 172.21.15.105:0/1455005084 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb53c0b6e90 0x7fb53c14e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:27.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.177+0000 7fb53bfff640 1 -- 172.21.15.105:0/1455005084 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb5240039e0 msgr2=0x7fb53c1e9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:27.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.177+0000 7fb53bfff640 1 --2- 172.21.15.105:0/1455005084 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb5240039e0 0x7fb53c1e9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:27.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.177+0000 7fb53bfff640 1 -- 172.21.15.105:0/1455005084 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb53c0b5970 con 0x7fb53c0b6a10 2024-04-15T15:19:27.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.177+0000 7fb5410ab640 1 --2- 172.21.15.105:0/1455005084 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb53c0b6e90 0x7fb53c14e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:27.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.177+0000 7fb53bfff640 1 --2- 172.21.15.105:0/1455005084 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb53c0b6a10 0x7fb53c14e2e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb53000f820 tx=0x7fb53000fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:27.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.177+0000 7fb5397fa640 1 -- 172.21.15.105:0/1455005084 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5300050d0 con 0x7fb53c0b6a10 2024-04-15T15:19:27.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.177+0000 7fb5397fa640 1 -- 172.21.15.105:0/1455005084 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb530018070 con 0x7fb53c0b6a10 2024-04-15T15:19:27.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.177+0000 7fb5397fa640 1 -- 172.21.15.105:0/1455005084 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb530005490 con 0x7fb53c0b6a10 2024-04-15T15:19:27.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.177+0000 7fb542b0e640 1 -- 172.21.15.105:0/1455005084 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb53c0b5c50 con 0x7fb53c0b6a10 2024-04-15T15:19:27.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.177+0000 7fb542b0e640 1 -- 172.21.15.105:0/1455005084 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb53c0b6210 con 0x7fb53c0b6a10 2024-04-15T15:19:27.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.178+0000 7fb542b0e640 1 -- 172.21.15.105:0/1455005084 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb53c06ff10 con 0x7fb53c0b6a10 2024-04-15T15:19:27.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.178+0000 7fb5397fa640 1 -- 172.21.15.105:0/1455005084 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb53000d730 con 0x7fb53c0b6a10 2024-04-15T15:19:27.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.179+0000 7fb5397fa640 1 --2- 172.21.15.105:0/1455005084 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fb52806c920 0x7fb52806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:27.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.179+0000 7fb5397fa640 1 -- 172.21.15.105:0/1455005084 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7fb53008b970 con 0x7fb53c0b6a10 2024-04-15T15:19:27.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.179+0000 7fb5408aa640 1 --2- 172.21.15.105:0/1455005084 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fb52806c920 0x7fb52806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:27.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.179+0000 7fb5408aa640 1 --2- 172.21.15.105:0/1455005084 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fb52806c920 0x7fb52806ede0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb534034000 tx=0x7fb534009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:27.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.183+0000 7fb5397fa640 1 -- 172.21.15.105:0/1455005084 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb53c06ff10 con 0x7fb53c0b6a10 2024-04-15T15:19:27.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.394+0000 7fb542b0e640 1 -- 172.21.15.105:0/1455005084 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fb53c068a40 con 0x7fb53c0b6a10 2024-04-15T15:19:27.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.429+0000 7fb5397fa640 1 -- 172.21.15.105:0/1455005084 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) ==== 118+0+0 (secure 0 0 0) 0x7fb530059da0 con 0x7fb53c0b6a10 2024-04-15T15:19:27.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.431+0000 7fb542b0e640 1 -- 172.21.15.105:0/1455005084 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fb52806c920 msgr2=0x7fb52806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:27.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.431+0000 7fb542b0e640 1 --2- 172.21.15.105:0/1455005084 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fb52806c920 0x7fb52806ede0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb534034000 tx=0x7fb534009030 comp rx=0 tx=0).stop 2024-04-15T15:19:27.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.431+0000 7fb542b0e640 1 -- 172.21.15.105:0/1455005084 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb53c0b6a10 msgr2=0x7fb53c14e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:27.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.431+0000 7fb542b0e640 1 --2- 172.21.15.105:0/1455005084 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb53c0b6a10 0x7fb53c14e2e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb53000f820 tx=0x7fb53000fce0 comp rx=0 tx=0).stop 2024-04-15T15:19:27.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.431+0000 7fb542b0e640 1 -- 172.21.15.105:0/1455005084 shutdown_connections 2024-04-15T15:19:27.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.431+0000 7fb542b0e640 1 --2- 172.21.15.105:0/1455005084 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb53c0b6e90 0x7fb53c14e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:27.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.431+0000 7fb542b0e640 1 --2- 172.21.15.105:0/1455005084 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fb52806c920 0x7fb52806ede0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:27.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.431+0000 7fb542b0e640 1 --2- 172.21.15.105:0/1455005084 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb53c0b6a10 0x7fb53c14e2e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:27.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.431+0000 7fb542b0e640 1 --2- 172.21.15.105:0/1455005084 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb5240039e0 0x7fb53c1e9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:27.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.431+0000 7fb542b0e640 1 -- 172.21.15.105:0/1455005084 >> 172.21.15.105:0/1455005084 conn(0x7fb53c0b2fc0 msgr2=0x7fb53c074a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:27.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.431+0000 7fb542b0e640 1 -- 172.21.15.105:0/1455005084 shutdown_connections 2024-04-15T15:19:27.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.431+0000 7fb542b0e640 1 -- 172.21.15.105:0/1455005084 wait complete. 2024-04-15T15:19:27.525 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-15T15:19:27.525 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-15T15:19:27.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.630+0000 7ff5c3e14640 1 Processor -- start 2024-04-15T15:19:27.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.630+0000 7ff5c3e14640 1 -- start start 2024-04-15T15:19:27.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.631+0000 7ff5c3e14640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff5bc09b150 0x7ff5bc070070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:27.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.631+0000 7ff5c3e14640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5bc0d0870 con 0x7ff5bc15e8b0 2024-04-15T15:19:27.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.631+0000 7ff5c3e14640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5bc0d0af0 con 0x7ff5bc0705b0 2024-04-15T15:19:27.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.631+0000 7ff5c3e14640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7ff5bc15f280 con 0x7ff5bc09b150 2024-04-15T15:19:27.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.631+0000 7ff5c1bb0640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7ff5bc15e8b0 0x7ff5bc09aa40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:40308/0 (socket says 172.21.15.105:40308) 2024-04-15T15:19:27.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.631+0000 7ff5c1bb0640 1 -- 172.21.15.105:0/4269229923 learned_addr learned my addr 172.21.15.105:0/4269229923 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:19:27.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.631+0000 7ff5c13af640 1 --2- 172.21.15.105:0/4269229923 >> v2:172.21.15.154:3300/0 conn(0x7ff5bc09b150 0x7ff5bc070070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:27.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.631+0000 7ff5c0bae640 1 -- 172.21.15.105:0/4269229923 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1125087173 0 0) 0x7ff5b4001110 con 0x7ff5bc15e8b0 2024-04-15T15:19:27.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.631+0000 7ff5c0bae640 1 -- 172.21.15.105:0/4269229923 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff5a4003520 con 0x7ff5bc15e8b0 2024-04-15T15:19:27.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.631+0000 7ff5c0bae640 1 -- 172.21.15.105:0/4269229923 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 487163986 0 0) 0x7ff5a4003520 con 0x7ff5bc15e8b0 2024-04-15T15:19:27.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.631+0000 7ff5c0bae640 1 -- 172.21.15.105:0/4269229923 >> v2:172.21.15.154:3300/0 conn(0x7ff5bc09b150 msgr2=0x7ff5bc070070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:27.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.631+0000 7ff5c0bae640 1 --2- 172.21.15.105:0/4269229923 >> v2:172.21.15.154:3300/0 conn(0x7ff5bc09b150 0x7ff5bc070070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:27.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.631+0000 7ff5c0bae640 1 -- 172.21.15.105:0/4269229923 >> v1:172.21.15.154:6789/0 conn(0x7ff5bc0705b0 legacy=0x7ff5bc06de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:27.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.631+0000 7ff5c0bae640 1 -- 172.21.15.105:0/4269229923 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5bc0714f0 con 0x7ff5bc15e8b0 2024-04-15T15:19:27.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.631+0000 7ff5c0bae640 1 -- 172.21.15.105:0/4269229923 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7ff5b4001840 con 0x7ff5bc15e8b0 2024-04-15T15:19:27.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.631+0000 7ff5c0bae640 1 -- 172.21.15.105:0/4269229923 >> v1:172.21.15.105:6789/0 conn(0x7ff5bc15e8b0 legacy=0x7ff5bc09aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:27.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.631+0000 7ff5c0bae640 1 --2- 172.21.15.105:0/4269229923 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5a40039b0 0x7ff5a4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:27.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.631+0000 7ff5c0bae640 1 --2- 172.21.15.105:0/4269229923 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5a4004800 0x7ff5a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:27.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.631+0000 7ff5c0bae640 1 --2- 172.21.15.105:0/4269229923 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5a40051c0 0x7ff5a400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:27.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.631+0000 7ff5c0bae640 1 -- 172.21.15.105:0/4269229923 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a400c030 con 0x7ff5a40051c0 2024-04-15T15:19:27.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.631+0000 7ff5c0bae640 1 -- 172.21.15.105:0/4269229923 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a400c1e0 con 0x7ff5a4004800 2024-04-15T15:19:27.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.631+0000 7ff5c0bae640 1 -- 172.21.15.105:0/4269229923 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7ff5a400c3e0 con 0x7ff5a40039b0 2024-04-15T15:19:27.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.631+0000 7ff5c23b1640 1 --2- 172.21.15.105:0/4269229923 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5a40051c0 0x7ff5a400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:27.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.631+0000 7ff5c13af640 1 --2- 172.21.15.105:0/4269229923 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5a4004800 0x7ff5a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:27.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.631+0000 7ff5c1bb0640 1 --2- 172.21.15.105:0/4269229923 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5a40039b0 0x7ff5a4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:27.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.632+0000 7ff5c23b1640 1 -- 172.21.15.105:0/4269229923 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5a40039b0 msgr2=0x7ff5a4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:27.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.632+0000 7ff5c23b1640 1 --2- 172.21.15.105:0/4269229923 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5a40039b0 0x7ff5a4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:27.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.632+0000 7ff5c23b1640 1 -- 172.21.15.105:0/4269229923 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5a4004800 msgr2=0x7ff5a4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:27.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.632+0000 7ff5c23b1640 1 --2- 172.21.15.105:0/4269229923 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5a4004800 0x7ff5a4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:27.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.632+0000 7ff5c23b1640 1 -- 172.21.15.105:0/4269229923 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5a4003520 con 0x7ff5a40051c0 2024-04-15T15:19:27.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.632+0000 7ff5c23b1640 1 -- 172.21.15.105:0/4269229923 reap_dead start 2024-04-15T15:19:27.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.632+0000 7ff5c23b1640 1 --2- 172.21.15.105:0/4269229923 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5a40051c0 0x7ff5a400b9c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac001e50 tx=0x7ff5a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6e04d034442285a server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:27.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.632+0000 7ff5c0bae640 1 -- 172.21.15.105:0/4269229923 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac001a70 con 0x7ff5a40051c0 2024-04-15T15:19:27.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.633+0000 7ff5c0bae640 1 -- 172.21.15.105:0/4269229923 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5ac003bb0 con 0x7ff5a40051c0 2024-04-15T15:19:27.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.633+0000 7ff5c0bae640 1 -- 172.21.15.105:0/4269229923 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac0050d0 con 0x7ff5a40051c0 2024-04-15T15:19:27.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.633+0000 7ff5c3e14640 1 -- 172.21.15.105:0/4269229923 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5a40051c0 msgr2=0x7ff5a400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:27.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.633+0000 7ff5c3e14640 1 --2- 172.21.15.105:0/4269229923 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5a40051c0 0x7ff5a400b9c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac001e50 tx=0x7ff5a4005830 comp rx=0 tx=0).stop 2024-04-15T15:19:27.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.633+0000 7ff5c3e14640 1 -- 172.21.15.105:0/4269229923 shutdown_connections 2024-04-15T15:19:27.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.633+0000 7ff5c3e14640 1 --2- 172.21.15.105:0/4269229923 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5a40051c0 0x7ff5a400b9c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:27.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.633+0000 7ff5c3e14640 1 -- 172.21.15.105:0/4269229923 >> 172.21.15.105:0/4269229923 conn(0x7ff5bc0b4090 msgr2=0x7ff5bc0b44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:27.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.633+0000 7ff5c3e14640 1 -- 172.21.15.105:0/4269229923 shutdown_connections 2024-04-15T15:19:27.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.633+0000 7ff5c3e14640 1 -- 172.21.15.105:0/4269229923 wait complete. 2024-04-15T15:19:27.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.634+0000 7ff5c3e14640 1 Processor -- start 2024-04-15T15:19:27.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.634+0000 7ff5c3e14640 1 -- start start 2024-04-15T15:19:27.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.635+0000 7ff5c3e14640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5a40051c0 0x7ff5bc160b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:27.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.635+0000 7ff5c3e14640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5bc06d5c0 0x7ff5bc152680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:27.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.635+0000 7ff5c3e14640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5bc06da40 0x7ff5bc152bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:27.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.635+0000 7ff5c3e14640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff5bc071210 con 0x7ff5bc06d5c0 2024-04-15T15:19:27.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.635+0000 7ff5c3e14640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff5bc070f30 con 0x7ff5bc06da40 2024-04-15T15:19:27.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.635+0000 7ff5c3e14640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7ff5bc15ec90 con 0x7ff5a40051c0 2024-04-15T15:19:27.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.635+0000 7ff5c13af640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5bc06d5c0 0x7ff5bc152680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:27.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.635+0000 7ff5c13af640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5bc06d5c0 0x7ff5bc152680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57196/0 (socket says 172.21.15.105:57196) 2024-04-15T15:19:27.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.635+0000 7ff5c13af640 1 -- 172.21.15.105:0/962608087 learned_addr learned my addr 172.21.15.105:0/962608087 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:27.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.635+0000 7ff5c13af640 1 -- 172.21.15.105:0/962608087 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5a40051c0 msgr2=0x7ff5bc160b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:27.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.635+0000 7ff5c13af640 1 --2- 172.21.15.105:0/962608087 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5a40051c0 0x7ff5bc160b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:27.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.635+0000 7ff5c13af640 1 -- 172.21.15.105:0/962608087 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5bc06da40 msgr2=0x7ff5bc152bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:27.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.635+0000 7ff5c13af640 1 --2- 172.21.15.105:0/962608087 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5bc06da40 0x7ff5bc152bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:27.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.635+0000 7ff5c13af640 1 -- 172.21.15.105:0/962608087 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5bc0b5640 con 0x7ff5bc06d5c0 2024-04-15T15:19:27.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.636+0000 7ff5c13af640 1 --2- 172.21.15.105:0/962608087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5bc06d5c0 0x7ff5bc152680 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0011910 tx=0x7ff5b0011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:27.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.636+0000 7ff5ba7fc640 1 -- 172.21.15.105:0/962608087 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b000faf0 con 0x7ff5bc06d5c0 2024-04-15T15:19:27.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.636+0000 7ff5ba7fc640 1 -- 172.21.15.105:0/962608087 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5b000fc90 con 0x7ff5bc06d5c0 2024-04-15T15:19:27.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.636+0000 7ff5c3e14640 1 -- 172.21.15.105:0/962608087 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5bc0b5920 con 0x7ff5bc06d5c0 2024-04-15T15:19:27.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.636+0000 7ff5ba7fc640 1 -- 172.21.15.105:0/962608087 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b0014440 con 0x7ff5bc06d5c0 2024-04-15T15:19:27.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.636+0000 7ff5c3e14640 1 -- 172.21.15.105:0/962608087 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5bc0b5ee0 con 0x7ff5bc06d5c0 2024-04-15T15:19:27.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.637+0000 7ff5c3e14640 1 -- 172.21.15.105:0/962608087 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5bc068a40 con 0x7ff5bc06d5c0 2024-04-15T15:19:27.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.637+0000 7ff5ba7fc640 1 -- 172.21.15.105:0/962608087 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff5b0012040 con 0x7ff5bc06d5c0 2024-04-15T15:19:27.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.638+0000 7ff5ba7fc640 1 --2- 172.21.15.105:0/962608087 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff5a806c8f0 0x7ff5a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:27.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.638+0000 7ff5ba7fc640 1 -- 172.21.15.105:0/962608087 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff5b008c1a0 con 0x7ff5bc06d5c0 2024-04-15T15:19:27.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.638+0000 7ff5c1bb0640 1 --2- 172.21.15.105:0/962608087 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff5a806c8f0 0x7ff5a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:27.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.638+0000 7ff5c1bb0640 1 --2- 172.21.15.105:0/962608087 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff5a806c8f0 0x7ff5a806edb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff5b40012b0 tx=0x7ff5b4000ed0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:27.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.642+0000 7ff5ba7fc640 1 -- 172.21.15.105:0/962608087 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff5b0019030 con 0x7ff5bc06d5c0 2024-04-15T15:19:27.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:27.856+0000 7ff5c3e14640 1 -- 172.21.15.105:0/962608087 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7ff5bc0b4f10 con 0x7ff5bc06d5c0 2024-04-15T15:19:28.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.439+0000 7ff5ba7fc640 1 -- 172.21.15.105:0/962608087 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v6) ==== 122+0+0 (secure 0 0 0) 0x7ff5b005a5d0 con 0x7ff5bc06d5c0 2024-04-15T15:19:28.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.441+0000 7ff5c3e14640 1 -- 172.21.15.105:0/962608087 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff5a806c8f0 msgr2=0x7ff5a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:28.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.441+0000 7ff5c3e14640 1 --2- 172.21.15.105:0/962608087 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff5a806c8f0 0x7ff5a806edb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff5b40012b0 tx=0x7ff5b4000ed0 comp rx=0 tx=0).stop 2024-04-15T15:19:28.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.441+0000 7ff5c3e14640 1 -- 172.21.15.105:0/962608087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5bc06d5c0 msgr2=0x7ff5bc152680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:28.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.441+0000 7ff5c3e14640 1 --2- 172.21.15.105:0/962608087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5bc06d5c0 0x7ff5bc152680 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0011910 tx=0x7ff5b0011dd0 comp rx=0 tx=0).stop 2024-04-15T15:19:28.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.441+0000 7ff5c3e14640 1 -- 172.21.15.105:0/962608087 shutdown_connections 2024-04-15T15:19:28.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.441+0000 7ff5c3e14640 1 --2- 172.21.15.105:0/962608087 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff5a806c8f0 0x7ff5a806edb0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:28.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.441+0000 7ff5c3e14640 1 --2- 172.21.15.105:0/962608087 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5bc06da40 0x7ff5bc152bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:28.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.441+0000 7ff5c3e14640 1 --2- 172.21.15.105:0/962608087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5bc06d5c0 0x7ff5bc152680 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:28.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.441+0000 7ff5c3e14640 1 --2- 172.21.15.105:0/962608087 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5a40051c0 0x7ff5bc160b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:28.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.441+0000 7ff5c3e14640 1 -- 172.21.15.105:0/962608087 >> 172.21.15.105:0/962608087 conn(0x7ff5bc0b4090 msgr2=0x7ff5bc09aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:28.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.441+0000 7ff5c3e14640 1 -- 172.21.15.105:0/962608087 shutdown_connections 2024-04-15T15:19:28.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.441+0000 7ff5c3e14640 1 -- 172.21.15.105:0/962608087 wait complete. 2024-04-15T15:19:28.455 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-15T15:19:28.455 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-15T15:19:28.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.558+0000 7f07bb893640 1 Processor -- start 2024-04-15T15:19:28.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.558+0000 7f07bb893640 1 -- start start 2024-04-15T15:19:28.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.559+0000 7f07bb893640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f07b415ec60 0x7f07b40d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:28.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.559+0000 7f07bb893640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07b40714f0 con 0x7f07b409b780 2024-04-15T15:19:28.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.559+0000 7f07bb893640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07b40d1940 con 0x7f07b409b150 2024-04-15T15:19:28.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.559+0000 7f07bb893640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f07b406cad0 con 0x7f07b415ec60 2024-04-15T15:19:28.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.559+0000 7f07b8e2e640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f07b409b780 0x7f07b409aa40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:40310/0 (socket says 172.21.15.105:40310) 2024-04-15T15:19:28.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.559+0000 7f07b8e2e640 1 -- 172.21.15.105:0/3191405418 learned_addr learned my addr 172.21.15.105:0/3191405418 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:19:28.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.559+0000 7f07abfff640 1 -- 172.21.15.105:0/3191405418 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 433477678 0 0) 0x7f07b40714f0 con 0x7f07b409b780 2024-04-15T15:19:28.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.559+0000 7f07abfff640 1 -- 172.21.15.105:0/3191405418 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f07a0003520 con 0x7f07b409b780 2024-04-15T15:19:28.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.560+0000 7f07b962f640 1 --2- 172.21.15.105:0/3191405418 >> v2:172.21.15.154:3300/0 conn(0x7f07b415ec60 0x7f07b40d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:28.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.560+0000 7f07abfff640 1 -- 172.21.15.105:0/3191405418 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1591907820 0 0) 0x7f07a0003520 con 0x7f07b409b780 2024-04-15T15:19:28.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.560+0000 7f07abfff640 1 -- 172.21.15.105:0/3191405418 >> v2:172.21.15.154:3300/0 conn(0x7f07b415ec60 msgr2=0x7f07b40d1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:28.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.560+0000 7f07abfff640 1 --2- 172.21.15.105:0/3191405418 >> v2:172.21.15.154:3300/0 conn(0x7f07b415ec60 0x7f07b40d1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:28.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.560+0000 7f07abfff640 1 -- 172.21.15.105:0/3191405418 >> v1:172.21.15.154:6789/0 conn(0x7f07b409b150 legacy=0x7f07b40cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:28.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.560+0000 7f07abfff640 1 -- 172.21.15.105:0/3191405418 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07b40ccd60 con 0x7f07b409b780 2024-04-15T15:19:28.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.560+0000 7f07abfff640 1 -- 172.21.15.105:0/3191405418 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7f07a4001670 con 0x7f07b409b780 2024-04-15T15:19:28.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.560+0000 7f07abfff640 1 -- 172.21.15.105:0/3191405418 >> v1:172.21.15.105:6789/0 conn(0x7f07b409b780 legacy=0x7f07b409aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:28.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.560+0000 7f07abfff640 1 --2- 172.21.15.105:0/3191405418 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07a00039b0 0x7f07a0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:28.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.560+0000 7f07abfff640 1 --2- 172.21.15.105:0/3191405418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07a0004800 0x7f07a0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:28.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.560+0000 7f07abfff640 1 --2- 172.21.15.105:0/3191405418 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07a00051c0 0x7f07a000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:28.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.560+0000 7f07abfff640 1 -- 172.21.15.105:0/3191405418 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f07a000c0e0 con 0x7f07a0004800 2024-04-15T15:19:28.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.560+0000 7f07abfff640 1 -- 172.21.15.105:0/3191405418 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f07a000c290 con 0x7f07a00051c0 2024-04-15T15:19:28.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.560+0000 7f07abfff640 1 -- 172.21.15.105:0/3191405418 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f07a000c490 con 0x7f07a00039b0 2024-04-15T15:19:28.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.560+0000 7f07b8e2e640 1 --2- 172.21.15.105:0/3191405418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07a0004800 0x7f07a0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:28.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.560+0000 7f07b962f640 1 --2- 172.21.15.105:0/3191405418 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07a00039b0 0x7f07a0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:28.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.560+0000 7f07b9e30640 1 --2- 172.21.15.105:0/3191405418 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07a00051c0 0x7f07a000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:28.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.560+0000 7f07b8e2e640 1 -- 172.21.15.105:0/3191405418 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07a00039b0 msgr2=0x7f07a0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:28.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.560+0000 7f07b8e2e640 1 --2- 172.21.15.105:0/3191405418 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07a00039b0 0x7f07a0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:28.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.560+0000 7f07b8e2e640 1 -- 172.21.15.105:0/3191405418 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07a00051c0 msgr2=0x7f07a000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:28.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.560+0000 7f07b8e2e640 1 --2- 172.21.15.105:0/3191405418 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07a00051c0 0x7f07a000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:28.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.560+0000 7f07b9e30640 1 -- 172.21.15.105:0/3191405418 reap_dead start 2024-04-15T15:19:28.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.560+0000 7f07b8e2e640 1 -- 172.21.15.105:0/3191405418 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07a0003520 con 0x7f07a0004800 2024-04-15T15:19:28.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.561+0000 7f07b8e2e640 1 --2- 172.21.15.105:0/3191405418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07a0004800 0x7f07a0004c80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f07a400c210 tx=0x7f07a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2328a02173b0158c server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:28.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.561+0000 7f07abfff640 1 -- 172.21.15.105:0/3191405418 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a4007d30 con 0x7f07a0004800 2024-04-15T15:19:28.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.561+0000 7f07abfff640 1 -- 172.21.15.105:0/3191405418 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07a4030070 con 0x7f07a0004800 2024-04-15T15:19:28.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.561+0000 7f07abfff640 1 -- 172.21.15.105:0/3191405418 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a402cda0 con 0x7f07a0004800 2024-04-15T15:19:28.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.562+0000 7f07bb893640 1 -- 172.21.15.105:0/3191405418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07a0004800 msgr2=0x7f07a0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:28.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.562+0000 7f07bb893640 1 --2- 172.21.15.105:0/3191405418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07a0004800 0x7f07a0004c80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f07a400c210 tx=0x7f07a0005830 comp rx=0 tx=0).stop 2024-04-15T15:19:28.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.562+0000 7f07bb893640 1 -- 172.21.15.105:0/3191405418 shutdown_connections 2024-04-15T15:19:28.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.562+0000 7f07bb893640 1 --2- 172.21.15.105:0/3191405418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07a0004800 0x7f07a0004c80 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:28.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.562+0000 7f07bb893640 1 -- 172.21.15.105:0/3191405418 >> 172.21.15.105:0/3191405418 conn(0x7f07b40d2570 msgr2=0x7f07b40d2980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:28.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.562+0000 7f07bb893640 1 -- 172.21.15.105:0/3191405418 shutdown_connections 2024-04-15T15:19:28.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.562+0000 7f07bb893640 1 -- 172.21.15.105:0/3191405418 wait complete. 2024-04-15T15:19:28.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.563+0000 7f07bb893640 1 Processor -- start 2024-04-15T15:19:28.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.563+0000 7f07bb893640 1 -- start start 2024-04-15T15:19:28.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.564+0000 7f07bb893640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07a0004800 0x7f07b4150bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:28.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.564+0000 7f07bb893640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07b40c6660 0x7f07b409f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:28.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.564+0000 7f07bb893640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07b409fcb0 0x7f07b4156920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:28.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.564+0000 7f07bb893640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f07b40d1210 con 0x7f07b40c6660 2024-04-15T15:19:28.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.564+0000 7f07bb893640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f07b40cd3b0 con 0x7f07a0004800 2024-04-15T15:19:28.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.564+0000 7f07bb893640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f07b40d1390 con 0x7f07b409fcb0 2024-04-15T15:19:28.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.564+0000 7f07b8e2e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07b40c6660 0x7f07b409f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:28.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.564+0000 7f07b8e2e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07b40c6660 0x7f07b409f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57216/0 (socket says 172.21.15.105:57216) 2024-04-15T15:19:28.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.564+0000 7f07b8e2e640 1 -- 172.21.15.105:0/1838350658 learned_addr learned my addr 172.21.15.105:0/1838350658 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:28.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.564+0000 7f07b9e30640 1 --2- 172.21.15.105:0/1838350658 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07b409fcb0 0x7f07b4156920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:28.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.564+0000 7f07b962f640 1 --2- 172.21.15.105:0/1838350658 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07a0004800 0x7f07b4150bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:28.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.564+0000 7f07b8e2e640 1 -- 172.21.15.105:0/1838350658 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07b409fcb0 msgr2=0x7f07b4156920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:28.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.564+0000 7f07b8e2e640 1 --2- 172.21.15.105:0/1838350658 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07b409fcb0 0x7f07b4156920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:28.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.564+0000 7f07b8e2e640 1 -- 172.21.15.105:0/1838350658 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07a0004800 msgr2=0x7f07b4150bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:28.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.564+0000 7f07b8e2e640 1 --2- 172.21.15.105:0/1838350658 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07a0004800 0x7f07b4150bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:28.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.564+0000 7f07b8e2e640 1 -- 172.21.15.105:0/1838350658 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07b4151130 con 0x7f07b40c6660 2024-04-15T15:19:28.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.564+0000 7f07b9e30640 1 --2- 172.21.15.105:0/1838350658 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07b409fcb0 0x7f07b4156920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:28.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.564+0000 7f07b962f640 1 --2- 172.21.15.105:0/1838350658 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07a0004800 0x7f07b4150bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:28.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.564+0000 7f07b8e2e640 1 --2- 172.21.15.105:0/1838350658 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07b40c6660 0x7f07b409f770 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f07a400c210 tx=0x7f07a402e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:28.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.564+0000 7f07a9ffb640 1 -- 172.21.15.105:0/1838350658 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a4046070 con 0x7f07b40c6660 2024-04-15T15:19:28.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.564+0000 7f07a9ffb640 1 -- 172.21.15.105:0/1838350658 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07a4004bf0 con 0x7f07b40c6660 2024-04-15T15:19:28.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.564+0000 7f07bb893640 1 -- 172.21.15.105:0/1838350658 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07b40c8430 con 0x7f07b40c6660 2024-04-15T15:19:28.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.564+0000 7f07a9ffb640 1 -- 172.21.15.105:0/1838350658 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a4007d10 con 0x7f07b40c6660 2024-04-15T15:19:28.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.565+0000 7f07bb893640 1 -- 172.21.15.105:0/1838350658 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07b40c89f0 con 0x7f07b40c6660 2024-04-15T15:19:28.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.565+0000 7f07bb893640 1 -- 172.21.15.105:0/1838350658 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07b4068a40 con 0x7f07b40c6660 2024-04-15T15:19:28.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.566+0000 7f07a9ffb640 1 -- 172.21.15.105:0/1838350658 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f07a4007070 con 0x7f07b40c6660 2024-04-15T15:19:28.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.566+0000 7f07a9ffb640 1 --2- 172.21.15.105:0/1838350658 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f079806c920 0x7f079806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:28.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.566+0000 7f07a9ffb640 1 -- 172.21.15.105:0/1838350658 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f07a40acf50 con 0x7f07b40c6660 2024-04-15T15:19:28.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.566+0000 7f07b962f640 1 --2- 172.21.15.105:0/1838350658 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f079806c920 0x7f079806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:28.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.567+0000 7f07b962f640 1 --2- 172.21.15.105:0/1838350658 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f079806c920 0x7f079806ede0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f07b40a04a0 tx=0x7f07b0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:28.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.571+0000 7f07a9ffb640 1 -- 172.21.15.105:0/1838350658 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f07a407ad40 con 0x7f07b40c6660 2024-04-15T15:19:28.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.783+0000 7f07bb893640 1 -- 172.21.15.105:0/1838350658 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f07b40a0090 con 0x7f07b40c6660 2024-04-15T15:19:28.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.784+0000 7f07a9ffb640 1 -- 172.21.15.105:0/1838350658 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5187 (secure 0 0 0) 0x7f07a407f040 con 0x7f07b40c6660 2024-04-15T15:19:28.786 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:19:28.786 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4259,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/3319657830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":3319657830},{"type":"v1","addr":"172.21.15.154:6835","nonce":3319657830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2677199890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2677199890},{"type":"v1","addr":"172.21.15.105:6835","nonce":2677199890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2867739743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2867739743},{"type":"v1","addr":"172.21.15.105:6837","nonce":2867739743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-15T15:19:25.760362+0000","modified":"2024-04-15T15:19:27.857599+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.154:6834/1529675504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1529675504},{"type":"v1","addr":"172.21.15.154:6834","nonce":1529675504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4291,"qdb_cluster":[4291]},"id":1}]} 2024-04-15T15:19:28.787 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 6 2024-04-15T15:19:28.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.786+0000 7f07bb893640 1 -- 172.21.15.105:0/1838350658 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f079806c920 msgr2=0x7f079806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:28.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.786+0000 7f07bb893640 1 --2- 172.21.15.105:0/1838350658 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f079806c920 0x7f079806ede0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f07b40a04a0 tx=0x7f07b0008040 comp rx=0 tx=0).stop 2024-04-15T15:19:28.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.786+0000 7f07bb893640 1 -- 172.21.15.105:0/1838350658 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07b40c6660 msgr2=0x7f07b409f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:28.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.786+0000 7f07bb893640 1 --2- 172.21.15.105:0/1838350658 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07b40c6660 0x7f07b409f770 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f07a400c210 tx=0x7f07a402e070 comp rx=0 tx=0).stop 2024-04-15T15:19:28.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.786+0000 7f07bb893640 1 -- 172.21.15.105:0/1838350658 shutdown_connections 2024-04-15T15:19:28.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.786+0000 7f07bb893640 1 --2- 172.21.15.105:0/1838350658 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07b409fcb0 0x7f07b4156920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:28.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.786+0000 7f07bb893640 1 --2- 172.21.15.105:0/1838350658 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f079806c920 0x7f079806ede0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:28.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.786+0000 7f07bb893640 1 --2- 172.21.15.105:0/1838350658 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07b40c6660 0x7f07b409f770 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:28.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.786+0000 7f07bb893640 1 --2- 172.21.15.105:0/1838350658 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07a0004800 0x7f07b4150bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:28.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.786+0000 7f07bb893640 1 -- 172.21.15.105:0/1838350658 >> 172.21.15.105:0/1838350658 conn(0x7f07b40d2570 msgr2=0x7f07b40d06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:28.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.786+0000 7f07bb893640 1 -- 172.21.15.105:0/1838350658 shutdown_connections 2024-04-15T15:19:28.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.786+0000 7f07bb893640 1 -- 172.21.15.105:0/1838350658 wait complete. 2024-04-15T15:19:28.799 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:19:28.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.901+0000 7f3d6f02c640 1 Processor -- start 2024-04-15T15:19:28.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.901+0000 7f3d6f02c640 1 -- start start 2024-04-15T15:19:28.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.901+0000 7f3d6f02c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3d68163130 0x7f3d680cf820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:28.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.901+0000 7f3d6f02c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3d680d0ef0 0x7f3d680d1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:28.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.901+0000 7f3d6f02c640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d6815edb0 con 0x7f3d6815e9d0 2024-04-15T15:19:28.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.901+0000 7f3d6f02c640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f3d680d18c0 con 0x7f3d680d0ef0 2024-04-15T15:19:28.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.901+0000 7f3d6f02c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f3d680cef90 con 0x7f3d68163130 2024-04-15T15:19:28.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.901+0000 7f3d6d5c9640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3d680d0ef0 0x7f3d680d1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:28.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.901+0000 7f3d6cdc8640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f3d6815e9d0 0x7f3d680d3f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:40326/0 (socket says 172.21.15.105:40326) 2024-04-15T15:19:28.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.902+0000 7f3d6cdc8640 1 -- 172.21.15.105:0/2324097348 learned_addr learned my addr 172.21.15.105:0/2324097348 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:19:28.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.902+0000 7f3d67fff640 1 --2- 172.21.15.105:0/2324097348 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3d68163130 0x7f3d680cf820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:28.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.902+0000 7f3d6d5c9640 1 -- 172.21.15.105:0/2324097348 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3d68163130 msgr2=0x7f3d680cf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:28.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.902+0000 7f3d6d5c9640 1 --2- 172.21.15.105:0/2324097348 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3d68163130 0x7f3d680cf820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:28.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.902+0000 7f3d6d5c9640 1 -- 172.21.15.105:0/2324097348 >> v1:172.21.15.105:6789/0 conn(0x7f3d6815e9d0 legacy=0x7f3d680d3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:28.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.902+0000 7f3d677fe640 1 -- 172.21.15.105:0/2324097348 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2302828373 0 0) 0x7f3d58001110 con 0x7f3d6815e9d0 2024-04-15T15:19:28.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.902+0000 7f3d6d5c9640 1 -- 172.21.15.105:0/2324097348 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d680ba280 con 0x7f3d680d0ef0 2024-04-15T15:19:28.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.902+0000 7f3d67fff640 1 --2- 172.21.15.105:0/2324097348 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3d68163130 0x7f3d680cf820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:28.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.903+0000 7f3d6d5c9640 1 --2- 172.21.15.105:0/2324097348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d680d0ef0 0x7f3d680d1380 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3d60009d90 tx=0x7f3d6002be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bee4574206c267c server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:28.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.903+0000 7f3d677fe640 1 -- 172.21.15.105:0/2324097348 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d60039070 con 0x7f3d680d0ef0 2024-04-15T15:19:28.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.903+0000 7f3d677fe640 1 -- 172.21.15.105:0/2324097348 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d6002cbc0 con 0x7f3d680d0ef0 2024-04-15T15:19:28.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.903+0000 7f3d677fe640 1 -- 172.21.15.105:0/2324097348 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d60034440 con 0x7f3d680d0ef0 2024-04-15T15:19:28.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.903+0000 7f3d6f02c640 1 -- 172.21.15.105:0/2324097348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d680d0ef0 msgr2=0x7f3d680d1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:28.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.903+0000 7f3d6f02c640 1 --2- 172.21.15.105:0/2324097348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d680d0ef0 0x7f3d680d1380 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3d60009d90 tx=0x7f3d6002be10 comp rx=0 tx=0).stop 2024-04-15T15:19:28.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.903+0000 7f3d6f02c640 1 -- 172.21.15.105:0/2324097348 shutdown_connections 2024-04-15T15:19:28.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.903+0000 7f3d6f02c640 1 --2- 172.21.15.105:0/2324097348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d680d0ef0 0x7f3d680d1380 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:28.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.903+0000 7f3d6f02c640 1 --2- 172.21.15.105:0/2324097348 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3d68163130 0x7f3d680cf820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:28.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.903+0000 7f3d6f02c640 1 -- 172.21.15.105:0/2324097348 >> 172.21.15.105:0/2324097348 conn(0x7f3d68067720 msgr2=0x7f3d68067b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:28.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.903+0000 7f3d6f02c640 1 -- 172.21.15.105:0/2324097348 shutdown_connections 2024-04-15T15:19:28.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.903+0000 7f3d6f02c640 1 -- 172.21.15.105:0/2324097348 wait complete. 2024-04-15T15:19:28.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.904+0000 7f3d6f02c640 1 Processor -- start 2024-04-15T15:19:28.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.904+0000 7f3d6f02c640 1 -- start start 2024-04-15T15:19:28.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.905+0000 7f3d6f02c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d680d0ef0 0x7f3d680cadc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:28.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.905+0000 7f3d6f02c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3d6815e9d0 0x7f3d680c9540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:28.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.905+0000 7f3d6f02c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d68163130 0x7f3d680b6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:28.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.905+0000 7f3d6f02c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3d680cfd60 con 0x7f3d68163130 2024-04-15T15:19:28.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.905+0000 7f3d6f02c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3d680cf250 con 0x7f3d680d0ef0 2024-04-15T15:19:28.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.905+0000 7f3d6f02c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f3d680cfee0 con 0x7f3d6815e9d0 2024-04-15T15:19:28.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.905+0000 7f3d6d5c9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d68163130 0x7f3d680b6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:28.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.905+0000 7f3d6d5c9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d68163130 0x7f3d680b6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57234/0 (socket says 172.21.15.105:57234) 2024-04-15T15:19:28.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.905+0000 7f3d6d5c9640 1 -- 172.21.15.105:0/522346409 learned_addr learned my addr 172.21.15.105:0/522346409 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:28.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.906+0000 7f3d67fff640 1 --2- 172.21.15.105:0/522346409 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3d6815e9d0 0x7f3d680c9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:28.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.906+0000 7f3d6cdc8640 1 --2- 172.21.15.105:0/522346409 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d680d0ef0 0x7f3d680cadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:28.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.906+0000 7f3d6d5c9640 1 -- 172.21.15.105:0/522346409 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3d6815e9d0 msgr2=0x7f3d680c9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:28.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.906+0000 7f3d6d5c9640 1 --2- 172.21.15.105:0/522346409 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3d6815e9d0 0x7f3d680c9540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:28.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.906+0000 7f3d6d5c9640 1 -- 172.21.15.105:0/522346409 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d680d0ef0 msgr2=0x7f3d680cadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:28.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.906+0000 7f3d6d5c9640 1 --2- 172.21.15.105:0/522346409 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d680d0ef0 0x7f3d680cadc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:28.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.906+0000 7f3d6d5c9640 1 -- 172.21.15.105:0/522346409 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d680c9a80 con 0x7f3d68163130 2024-04-15T15:19:28.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.906+0000 7f3d6d5c9640 1 --2- 172.21.15.105:0/522346409 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d68163130 0x7f3d680b6660 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3d60037040 tx=0x7f3d60004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:28.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.906+0000 7f3d657fa640 1 -- 172.21.15.105:0/522346409 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d60044020 con 0x7f3d68163130 2024-04-15T15:19:28.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.906+0000 7f3d657fa640 1 -- 172.21.15.105:0/522346409 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d6003a040 con 0x7f3d68163130 2024-04-15T15:19:28.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.906+0000 7f3d6f02c640 1 -- 172.21.15.105:0/522346409 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d680c9d20 con 0x7f3d68163130 2024-04-15T15:19:28.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.906+0000 7f3d657fa640 1 -- 172.21.15.105:0/522346409 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d60039050 con 0x7f3d68163130 2024-04-15T15:19:28.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.906+0000 7f3d6f02c640 1 -- 172.21.15.105:0/522346409 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d680ca2e0 con 0x7f3d68163130 2024-04-15T15:19:28.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.907+0000 7f3d6f02c640 1 -- 172.21.15.105:0/522346409 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d68069150 con 0x7f3d68163130 2024-04-15T15:19:28.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.908+0000 7f3d657fa640 1 -- 172.21.15.105:0/522346409 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3d60003bd0 con 0x7f3d68163130 2024-04-15T15:19:28.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.908+0000 7f3d657fa640 1 --2- 172.21.15.105:0/522346409 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f3d5406c8f0 0x7f3d5406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:28.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.908+0000 7f3d657fa640 1 -- 172.21.15.105:0/522346409 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3d600ae2b0 con 0x7f3d68163130 2024-04-15T15:19:28.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.908+0000 7f3d6cdc8640 1 --2- 172.21.15.105:0/522346409 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f3d5406c8f0 0x7f3d5406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:28.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.908+0000 7f3d6cdc8640 1 --2- 172.21.15.105:0/522346409 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f3d5406c8f0 0x7f3d5406edb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3d58000f60 tx=0x7f3d58009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:28.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:28.912+0000 7f3d657fa640 1 -- 172.21.15.105:0/522346409 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3d6004e060 con 0x7f3d68163130 2024-04-15T15:19:29.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.088+0000 7f3d6f02c640 1 -- 172.21.15.105:0/522346409 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3d680d2540 con 0x7f3d68163130 2024-04-15T15:19:29.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.089+0000 7f3d657fa640 1 -- 172.21.15.105:0/522346409 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16653 (secure 0 0 0) 0x7f3d6007c770 con 0x7f3d68163130 2024-04-15T15:19:29.091 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:19:29.092 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":20,"fsid":"09e4d2a4-a324-4793-af77-644b25e520e1","created":"2024-04-15T15:19:01.524543+0000","modified":"2024-04-15T15:19:27.414890+0000","last_up_change":"2024-04-15T15:19:12.825815+0000","last_in_change":"2024-04-15T15:19:03.788126+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:19:13.748585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:19:23.789287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:19:24.722250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6f6292b-2d7e-4aa0-9d1c-6ddd0f97287c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6801","nonce":2873040054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6803","nonce":2873040054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6807","nonce":2873040054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6805","nonce":2873040054}]},"public_addr":"172.21.15.105:6801/2873040054","cluster_addr":"172.21.15.105:6803/2873040054","heartbeat_back_addr":"172.21.15.105:6807/2873040054","heartbeat_front_addr":"172.21.15.105:6805/2873040054","state":["exists","up"]},{"osd":1,"uuid":"8900966a-b393-41e3-8f93-e5a43bd23622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6818","nonce":2494994479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6821","nonce":2494994479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6828","nonce":2494994479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6823","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6825","nonce":2494994479}]},"public_addr":"172.21.15.105:6818/2494994479","cluster_addr":"172.21.15.105:6821/2494994479","heartbeat_back_addr":"172.21.15.105:6828/2494994479","heartbeat_front_addr":"172.21.15.105:6825/2494994479","state":["exists","up"]},{"osd":2,"uuid":"81a5dc7a-9325-4ff5-941d-a2d5fba21f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6809","nonce":3798408699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6811","nonce":3798408699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6815","nonce":3798408699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6813","nonce":3798408699}]},"public_addr":"172.21.15.105:6809/3798408699","cluster_addr":"172.21.15.105:6811/3798408699","heartbeat_back_addr":"172.21.15.105:6815/3798408699","heartbeat_front_addr":"172.21.15.105:6813/3798408699","state":["exists","up"]},{"osd":3,"uuid":"df766ed2-5c71-4805-96f4-3d4970edc960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6819","nonce":1920720186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6824","nonce":1920720186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6831","nonce":1920720186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6827","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6829","nonce":1920720186}]},"public_addr":"172.21.15.105:6819/1920720186","cluster_addr":"172.21.15.105:6824/1920720186","heartbeat_back_addr":"172.21.15.105:6831/1920720186","heartbeat_front_addr":"172.21.15.105:6829/1920720186","state":["exists","up"]},{"osd":4,"uuid":"e1039e03-28b8-49df-b951-c91930b93c9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6810","nonce":3516494456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6813","nonce":3516494456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6820","nonce":3516494456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6817","nonce":3516494456}]},"public_addr":"172.21.15.154:6810/3516494456","cluster_addr":"172.21.15.154:6813/3516494456","heartbeat_back_addr":"172.21.15.154:6820/3516494456","heartbeat_front_addr":"172.21.15.154:6817/3516494456","state":["exists","up"]},{"osd":5,"uuid":"a9f5eaf1-66ff-405a-a9e4-ec3340812a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6801","nonce":540367751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6803","nonce":540367751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6807","nonce":540367751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6805","nonce":540367751}]},"public_addr":"172.21.15.154:6801/540367751","cluster_addr":"172.21.15.154:6803/540367751","heartbeat_back_addr":"172.21.15.154:6807/540367751","heartbeat_front_addr":"172.21.15.154:6805/540367751","state":["exists","up"]},{"osd":6,"uuid":"267d02ea-675f-428f-aae8-e0375e4d4feb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6825","nonce":1550829005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6827","nonce":1550829005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6831","nonce":1550829005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6829","nonce":1550829005}]},"public_addr":"172.21.15.154:6825/1550829005","cluster_addr":"172.21.15.154:6827/1550829005","heartbeat_back_addr":"172.21.15.154:6831/1550829005","heartbeat_front_addr":"172.21.15.154:6829/1550829005","state":["exists","up"]},{"osd":7,"uuid":"759ac6cc-896a-423c-885f-9fe90d8dcb14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6809","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6811","nonce":3920089903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6816","nonce":3920089903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6823","nonce":3920089903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6819","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6821","nonce":3920089903}]},"public_addr":"172.21.15.154:6811/3920089903","cluster_addr":"172.21.15.154:6816/3920089903","heartbeat_back_addr":"172.21.15.154:6823/3920089903","heartbeat_front_addr":"172.21.15.154:6821/3920089903","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.949509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.984072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.907278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.947391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.975447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.998896+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:09.028739+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.894119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:19:29.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.091+0000 7f3d6f02c640 1 -- 172.21.15.105:0/522346409 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f3d5406c8f0 msgr2=0x7f3d5406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:29.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.091+0000 7f3d6f02c640 1 --2- 172.21.15.105:0/522346409 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f3d5406c8f0 0x7f3d5406edb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3d58000f60 tx=0x7f3d58009040 comp rx=0 tx=0).stop 2024-04-15T15:19:29.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.091+0000 7f3d6f02c640 1 -- 172.21.15.105:0/522346409 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d68163130 msgr2=0x7f3d680b6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:29.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.091+0000 7f3d6f02c640 1 --2- 172.21.15.105:0/522346409 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d68163130 0x7f3d680b6660 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3d60037040 tx=0x7f3d60004a40 comp rx=0 tx=0).stop 2024-04-15T15:19:29.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.091+0000 7f3d6f02c640 1 -- 172.21.15.105:0/522346409 shutdown_connections 2024-04-15T15:19:29.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.091+0000 7f3d6f02c640 1 --2- 172.21.15.105:0/522346409 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3d6815e9d0 0x7f3d680c9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:29.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.091+0000 7f3d6f02c640 1 --2- 172.21.15.105:0/522346409 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f3d5406c8f0 0x7f3d5406edb0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:29.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.091+0000 7f3d6f02c640 1 --2- 172.21.15.105:0/522346409 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d68163130 0x7f3d680b6660 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:29.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.091+0000 7f3d6f02c640 1 --2- 172.21.15.105:0/522346409 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d680d0ef0 0x7f3d680cadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:29.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.091+0000 7f3d6f02c640 1 -- 172.21.15.105:0/522346409 >> 172.21.15.105:0/522346409 conn(0x7f3d68067720 msgr2=0x7f3d6815c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:29.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.092+0000 7f3d6f02c640 1 -- 172.21.15.105:0/522346409 shutdown_connections 2024-04-15T15:19:29.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.092+0000 7f3d6f02c640 1 -- 172.21.15.105:0/522346409 wait complete. 2024-04-15T15:19:29.104 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-15T15:19:29.105 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-15T15:19:29.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.206+0000 7f2458fe6640 1 Processor -- start 2024-04-15T15:19:29.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.206+0000 7f2458fe6640 1 -- start start 2024-04-15T15:19:29.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.206+0000 7f2458fe6640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f24540b1280 0x7f2454167470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:29.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.206+0000 7f2458fe6640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24540af4b0 con 0x7f245415cb00 2024-04-15T15:19:29.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.206+0000 7f2458fe6640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24540af790 con 0x7f24540d0120 2024-04-15T15:19:29.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.206+0000 7f2458fe6640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f24540afa70 con 0x7f24540b1280 2024-04-15T15:19:29.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.206+0000 7f244bfff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f245415cb00 0x7f24540cdae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:40330/0 (socket says 172.21.15.105:40330) 2024-04-15T15:19:29.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.207+0000 7f244bfff640 1 -- 172.21.15.105:0/3953497572 learned_addr learned my addr 172.21.15.105:0/3953497572 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:19:29.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.207+0000 7f2452d9d640 1 --2- 172.21.15.105:0/3953497572 >> v2:172.21.15.154:3300/0 conn(0x7f24540b1280 0x7f2454167470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:29.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.207+0000 7f2451d9b640 1 -- 172.21.15.105:0/3953497572 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4217848112 0 0) 0x7f24540af4b0 con 0x7f245415cb00 2024-04-15T15:19:29.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.207+0000 7f2451d9b640 1 -- 172.21.15.105:0/3953497572 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2440003520 con 0x7f245415cb00 2024-04-15T15:19:29.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.207+0000 7f2451d9b640 1 -- 172.21.15.105:0/3953497572 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3405769392 0 0) 0x7f2440003520 con 0x7f245415cb00 2024-04-15T15:19:29.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.207+0000 7f2451d9b640 1 -- 172.21.15.105:0/3953497572 >> v2:172.21.15.154:3300/0 conn(0x7f24540b1280 msgr2=0x7f2454167470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:29.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.207+0000 7f2451d9b640 1 --2- 172.21.15.105:0/3953497572 >> v2:172.21.15.154:3300/0 conn(0x7f24540b1280 0x7f2454167470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:29.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.207+0000 7f2451d9b640 1 -- 172.21.15.105:0/3953497572 >> v1:172.21.15.154:6789/0 conn(0x7f24540d0120 legacy=0x7f24540d0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:29.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.207+0000 7f2451d9b640 1 -- 172.21.15.105:0/3953497572 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24540afd50 con 0x7f245415cb00 2024-04-15T15:19:29.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.207+0000 7f2451d9b640 1 -- 172.21.15.105:0/3953497572 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7f243c001670 con 0x7f245415cb00 2024-04-15T15:19:29.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.207+0000 7f2452d9d640 1 --2- 172.21.15.105:0/3953497572 >> v2:172.21.15.154:3300/0 conn(0x7f24540b1280 0x7f2454167470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:19:29.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.207+0000 7f2451d9b640 1 -- 172.21.15.105:0/3953497572 >> v1:172.21.15.105:6789/0 conn(0x7f245415cb00 legacy=0x7f24540cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:29.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.207+0000 7f2451d9b640 1 --2- 172.21.15.105:0/3953497572 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24400039e0 0x7f2440003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:29.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.207+0000 7f2451d9b640 1 --2- 172.21.15.105:0/3953497572 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2440004800 0x7f2440004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:29.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.207+0000 7f2451d9b640 1 --2- 172.21.15.105:0/3953497572 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f24400051c0 0x7f244000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:29.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.207+0000 7f2451d9b640 1 -- 172.21.15.105:0/3953497572 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f24540afa70 con 0x7f2440004800 2024-04-15T15:19:29.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.207+0000 7f2451d9b640 1 -- 172.21.15.105:0/3953497572 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f244000c090 con 0x7f24400039e0 2024-04-15T15:19:29.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.207+0000 7f2451d9b640 1 -- 172.21.15.105:0/3953497572 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f244000c290 con 0x7f24400051c0 2024-04-15T15:19:29.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.207+0000 7f244bfff640 1 --2- 172.21.15.105:0/3953497572 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2440004800 0x7f2440004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:29.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.207+0000 7f245259c640 1 --2- 172.21.15.105:0/3953497572 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24400039e0 0x7f2440003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:29.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.207+0000 7f2452d9d640 1 --2- 172.21.15.105:0/3953497572 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f24400051c0 0x7f244000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:29.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.207+0000 7f244bfff640 1 -- 172.21.15.105:0/3953497572 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f24400051c0 msgr2=0x7f244000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:29.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.207+0000 7f244bfff640 1 --2- 172.21.15.105:0/3953497572 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f24400051c0 0x7f244000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:29.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.207+0000 7f244bfff640 1 -- 172.21.15.105:0/3953497572 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24400039e0 msgr2=0x7f2440003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:29.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.207+0000 7f244bfff640 1 --2- 172.21.15.105:0/3953497572 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24400039e0 0x7f2440003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:29.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.207+0000 7f244bfff640 1 -- 172.21.15.105:0/3953497572 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2440003520 con 0x7f2440004800 2024-04-15T15:19:29.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.207+0000 7f2452d9d640 1 -- 172.21.15.105:0/3953497572 reap_dead start 2024-04-15T15:19:29.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.208+0000 7f244bfff640 1 --2- 172.21.15.105:0/3953497572 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2440004800 0x7f2440004c80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f245415cf00 tx=0x7f2440005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=996d57214d33bd1b server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:29.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.208+0000 7f2451d9b640 1 -- 172.21.15.105:0/3953497572 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f243c007d30 con 0x7f2440004800 2024-04-15T15:19:29.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.208+0000 7f2451d9b640 1 -- 172.21.15.105:0/3953497572 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f243c030070 con 0x7f2440004800 2024-04-15T15:19:29.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.208+0000 7f2451d9b640 1 -- 172.21.15.105:0/3953497572 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f243c005d70 con 0x7f2440004800 2024-04-15T15:19:29.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.209+0000 7f2458fe6640 1 -- 172.21.15.105:0/3953497572 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2440004800 msgr2=0x7f2440004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:29.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.209+0000 7f2458fe6640 1 --2- 172.21.15.105:0/3953497572 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2440004800 0x7f2440004c80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f245415cf00 tx=0x7f2440005830 comp rx=0 tx=0).stop 2024-04-15T15:19:29.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.209+0000 7f2458fe6640 1 -- 172.21.15.105:0/3953497572 shutdown_connections 2024-04-15T15:19:29.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.209+0000 7f2458fe6640 1 --2- 172.21.15.105:0/3953497572 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2440004800 0x7f2440004c80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:29.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.209+0000 7f2458fe6640 1 -- 172.21.15.105:0/3953497572 >> 172.21.15.105:0/3953497572 conn(0x7f24540b4040 msgr2=0x7f24540b4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:29.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.209+0000 7f2458fe6640 1 -- 172.21.15.105:0/3953497572 shutdown_connections 2024-04-15T15:19:29.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.209+0000 7f2458fe6640 1 -- 172.21.15.105:0/3953497572 wait complete. 2024-04-15T15:19:29.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.210+0000 7f2458fe6640 1 Processor -- start 2024-04-15T15:19:29.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.210+0000 7f2458fe6640 1 -- start start 2024-04-15T15:19:29.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.211+0000 7f2458fe6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2440004800 0x7f24540c9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:29.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.211+0000 7f2458fe6640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f24540cb180 0x7f24540cb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:29.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.211+0000 7f2458fe6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24540cbb40 0x7f245415d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:29.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.211+0000 7f2458fe6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f24540d0680 con 0x7f24540cbb40 2024-04-15T15:19:29.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.211+0000 7f2458fe6640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f24540d0500 con 0x7f2440004800 2024-04-15T15:19:29.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.211+0000 7f2458fe6640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f24540d0800 con 0x7f24540cb180 2024-04-15T15:19:29.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.211+0000 7f2452d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24540cbb40 0x7f245415d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:29.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.211+0000 7f2452d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24540cbb40 0x7f245415d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57242/0 (socket says 172.21.15.105:57242) 2024-04-15T15:19:29.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.211+0000 7f2452d9d640 1 -- 172.21.15.105:0/1381903579 learned_addr learned my addr 172.21.15.105:0/1381903579 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:29.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.211+0000 7f244bfff640 1 --2- 172.21.15.105:0/1381903579 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f24540cb180 0x7f24540cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:29.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.211+0000 7f245259c640 1 --2- 172.21.15.105:0/1381903579 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2440004800 0x7f24540c9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:29.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.211+0000 7f2452d9d640 1 -- 172.21.15.105:0/1381903579 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f24540cb180 msgr2=0x7f24540cb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:29.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.211+0000 7f2452d9d640 1 --2- 172.21.15.105:0/1381903579 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f24540cb180 0x7f24540cb600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:29.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.211+0000 7f2452d9d640 1 -- 172.21.15.105:0/1381903579 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2440004800 msgr2=0x7f24540c9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:29.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.211+0000 7f2452d9d640 1 --2- 172.21.15.105:0/1381903579 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2440004800 0x7f24540c9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:29.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.211+0000 7f2452d9d640 1 -- 172.21.15.105:0/1381903579 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24540b6ec0 con 0x7f24540cbb40 2024-04-15T15:19:29.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.212+0000 7f2452d9d640 1 --2- 172.21.15.105:0/1381903579 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24540cbb40 0x7f245415d440 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2454072fa0 tx=0x7f244c00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:29.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.212+0000 7f244affd640 1 -- 172.21.15.105:0/1381903579 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244c015430 con 0x7f24540cbb40 2024-04-15T15:19:29.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.212+0000 7f244affd640 1 -- 172.21.15.105:0/1381903579 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f244c01d070 con 0x7f24540cbb40 2024-04-15T15:19:29.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.212+0000 7f2458fe6640 1 -- 172.21.15.105:0/1381903579 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24540b71a0 con 0x7f24540cbb40 2024-04-15T15:19:29.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.212+0000 7f244affd640 1 -- 172.21.15.105:0/1381903579 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244c015830 con 0x7f24540cbb40 2024-04-15T15:19:29.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.212+0000 7f2458fe6640 1 -- 172.21.15.105:0/1381903579 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24540b7a40 con 0x7f24540cbb40 2024-04-15T15:19:29.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.213+0000 7f2458fe6640 1 -- 172.21.15.105:0/1381903579 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2454068a40 con 0x7f24540cbb40 2024-04-15T15:19:29.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.213+0000 7f244affd640 1 -- 172.21.15.105:0/1381903579 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f244c0159d0 con 0x7f24540cbb40 2024-04-15T15:19:29.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.214+0000 7f244affd640 1 --2- 172.21.15.105:0/1381903579 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f243406c920 0x7f243406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:29.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.214+0000 7f245259c640 1 --2- 172.21.15.105:0/1381903579 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f243406c920 0x7f243406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:29.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.214+0000 7f244affd640 1 -- 172.21.15.105:0/1381903579 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f244c08c6d0 con 0x7f24540cbb40 2024-04-15T15:19:29.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.214+0000 7f245259c640 1 --2- 172.21.15.105:0/1381903579 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f243406c920 0x7f243406ede0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2444000c00 tx=0x7f2444009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:29.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.218+0000 7f244affd640 1 -- 172.21.15.105:0/1381903579 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f244c027020 con 0x7f24540cbb40 2024-04-15T15:19:29.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.388+0000 7f2458fe6640 1 -- 172.21.15.105:0/1381903579 --> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f24540b1060 con 0x7f243406c920 2024-04-15T15:19:29.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.394+0000 7f244affd640 1 -- 172.21.15.105:0/1381903579 <== mgr.4107 v2:172.21.15.105:6832/3506549322 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+237079 (secure 0 0 0) 0x7f24540b1060 con 0x7f243406c920 2024-04-15T15:19:29.399 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:19:29.400 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-04-15T15:19:29.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.397+0000 7f2458fe6640 1 -- 172.21.15.105:0/1381903579 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f243406c920 msgr2=0x7f243406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:29.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.397+0000 7f2458fe6640 1 --2- 172.21.15.105:0/1381903579 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f243406c920 0x7f243406ede0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2444000c00 tx=0x7f2444009040 comp rx=0 tx=0).stop 2024-04-15T15:19:29.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.397+0000 7f2458fe6640 1 -- 172.21.15.105:0/1381903579 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24540cbb40 msgr2=0x7f245415d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:29.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.397+0000 7f2458fe6640 1 --2- 172.21.15.105:0/1381903579 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24540cbb40 0x7f245415d440 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2454072fa0 tx=0x7f244c00f580 comp rx=0 tx=0).stop 2024-04-15T15:19:29.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.397+0000 7f2458fe6640 1 -- 172.21.15.105:0/1381903579 shutdown_connections 2024-04-15T15:19:29.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.397+0000 7f2458fe6640 1 --2- 172.21.15.105:0/1381903579 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f24540cb180 0x7f24540cb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:29.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.397+0000 7f2458fe6640 1 --2- 172.21.15.105:0/1381903579 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f243406c920 0x7f243406ede0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:29.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.397+0000 7f2458fe6640 1 --2- 172.21.15.105:0/1381903579 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24540cbb40 0x7f245415d440 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:29.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.397+0000 7f2458fe6640 1 --2- 172.21.15.105:0/1381903579 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2440004800 0x7f24540c9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:29.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.397+0000 7f2458fe6640 1 -- 172.21.15.105:0/1381903579 >> 172.21.15.105:0/1381903579 conn(0x7f24540b4040 msgr2=0x7f2454071200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:29.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.397+0000 7f2458fe6640 1 -- 172.21.15.105:0/1381903579 shutdown_connections 2024-04-15T15:19:29.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.397+0000 7f2458fe6640 1 -- 172.21.15.105:0/1381903579 wait complete. 2024-04-15T15:19:29.410 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-04-15T15:19:27.734114+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":218572,"kb_used_data":2376,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756148,"statfs":{"total":773094113280,"available":772870295552,"internally_reserved":0,"allocated":2433024,"data_stored":1546872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000646"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:24.396509+0000","last_change":"2024-04-15T15:19:24.396509+0000","last_active":"2024-04-15T15:19:24.396509+0000","last_peered":"2024-04-15T15:19:24.396509+0000","last_clean":"2024-04-15T15:19:24.396509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:24.396509+0000","last_undegraded":"2024-04-15T15:19:24.396509+0000","last_fullsized":"2024-04-15T15:19:24.396509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:24.396509+0000","last_change":"2024-04-15T15:19:24.396509+0000","last_active":"2024-04-15T15:19:24.396509+0000","last_peered":"2024-04-15T15:19:24.396509+0000","last_clean":"2024-04-15T15:19:24.396509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:24.396509+0000","last_undegraded":"2024-04-15T15:19:24.396509+0000","last_fullsized":"2024-04-15T15:19:24.396509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:24.396509+0000","last_change":"2024-04-15T15:19:24.396509+0000","last_active":"2024-04-15T15:19:24.396509+0000","last_peered":"2024-04-15T15:19:24.396509+0000","last_clean":"2024-04-15T15:19:24.396509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:24.396509+0000","last_undegraded":"2024-04-15T15:19:24.396509+0000","last_fullsized":"2024-04-15T15:19:24.396509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:24.396509+0000","last_change":"2024-04-15T15:19:24.396509+0000","last_active":"2024-04-15T15:19:24.396509+0000","last_peered":"2024-04-15T15:19:24.396509+0000","last_clean":"2024-04-15T15:19:24.396509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:24.396509+0000","last_undegraded":"2024-04-15T15:19:24.396509+0000","last_fullsized":"2024-04-15T15:19:24.396509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:24.396509+0000","last_change":"2024-04-15T15:19:24.396509+0000","last_active":"2024-04-15T15:19:24.396509+0000","last_peered":"2024-04-15T15:19:24.396509+0000","last_clean":"2024-04-15T15:19:24.396509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:24.396509+0000","last_undegraded":"2024-04-15T15:19:24.396509+0000","last_fullsized":"2024-04-15T15:19:24.396509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:24.396509+0000","last_change":"2024-04-15T15:19:24.396509+0000","last_active":"2024-04-15T15:19:24.396509+0000","last_peered":"2024-04-15T15:19:24.396509+0000","last_clean":"2024-04-15T15:19:24.396509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:24.396509+0000","last_undegraded":"2024-04-15T15:19:24.396509+0000","last_fullsized":"2024-04-15T15:19:24.396509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:24.396509+0000","last_change":"2024-04-15T15:19:24.396509+0000","last_active":"2024-04-15T15:19:24.396509+0000","last_peered":"2024-04-15T15:19:24.396509+0000","last_clean":"2024-04-15T15:19:24.396509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:24.396509+0000","last_undegraded":"2024-04-15T15:19:24.396509+0000","last_fullsized":"2024-04-15T15:19:24.396509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:24.396509+0000","last_change":"2024-04-15T15:19:24.396509+0000","last_active":"2024-04-15T15:19:24.396509+0000","last_peered":"2024-04-15T15:19:24.396509+0000","last_clean":"2024-04-15T15:19:24.396509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:24.396509+0000","last_undegraded":"2024-04-15T15:19:24.396509+0000","last_fullsized":"2024-04-15T15:19:24.396509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:24.396509+0000","last_change":"2024-04-15T15:19:24.396509+0000","last_active":"2024-04-15T15:19:24.396509+0000","last_peered":"2024-04-15T15:19:24.396509+0000","last_clean":"2024-04-15T15:19:24.396509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:24.396509+0000","last_undegraded":"2024-04-15T15:19:24.396509+0000","last_fullsized":"2024-04-15T15:19:24.396509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:24.396509+0000","last_change":"2024-04-15T15:19:24.396509+0000","last_active":"2024-04-15T15:19:24.396509+0000","last_peered":"2024-04-15T15:19:24.396509+0000","last_clean":"2024-04-15T15:19:24.396509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:24.396509+0000","last_undegraded":"2024-04-15T15:19:24.396509+0000","last_fullsized":"2024-04-15T15:19:24.396509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:24.396509+0000","last_change":"2024-04-15T15:19:24.396509+0000","last_active":"2024-04-15T15:19:24.396509+0000","last_peered":"2024-04-15T15:19:24.396509+0000","last_clean":"2024-04-15T15:19:24.396509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:24.396509+0000","last_undegraded":"2024-04-15T15:19:24.396509+0000","last_fullsized":"2024-04-15T15:19:24.396509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:24.396509+0000","last_change":"2024-04-15T15:19:24.396509+0000","last_active":"2024-04-15T15:19:24.396509+0000","last_peered":"2024-04-15T15:19:24.396509+0000","last_clean":"2024-04-15T15:19:24.396509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:24.396509+0000","last_undegraded":"2024-04-15T15:19:24.396509+0000","last_fullsized":"2024-04-15T15:19:24.396509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:24.396509+0000","last_change":"2024-04-15T15:19:24.396509+0000","last_active":"2024-04-15T15:19:24.396509+0000","last_peered":"2024-04-15T15:19:24.396509+0000","last_clean":"2024-04-15T15:19:24.396509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:24.396509+0000","last_undegraded":"2024-04-15T15:19:24.396509+0000","last_fullsized":"2024-04-15T15:19:24.396509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:24.396509+0000","last_change":"2024-04-15T15:19:24.396509+0000","last_active":"2024-04-15T15:19:24.396509+0000","last_peered":"2024-04-15T15:19:24.396509+0000","last_clean":"2024-04-15T15:19:24.396509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:24.396509+0000","last_undegraded":"2024-04-15T15:19:24.396509+0000","last_fullsized":"2024-04-15T15:19:24.396509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:24.396509+0000","last_change":"2024-04-15T15:19:24.396509+0000","last_active":"2024-04-15T15:19:24.396509+0000","last_peered":"2024-04-15T15:19:24.396509+0000","last_clean":"2024-04-15T15:19:24.396509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:24.396509+0000","last_undegraded":"2024-04-15T15:19:24.396509+0000","last_fullsized":"2024-04-15T15:19:24.396509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:24.396509+0000","last_change":"2024-04-15T15:19:24.396509+0000","last_active":"2024-04-15T15:19:24.396509+0000","last_peered":"2024-04-15T15:19:24.396509+0000","last_clean":"2024-04-15T15:19:24.396509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:24.396509+0000","last_undegraded":"2024-04-15T15:19:24.396509+0000","last_fullsized":"2024-04-15T15:19:24.396509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":162,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-15T15:19:15.884437+0000","last_change":"2024-04-15T15:19:14.878269+0000","last_active":"2024-04-15T15:19:15.884437+0000","last_peered":"2024-04-15T15:19:15.884437+0000","last_clean":"2024-04-15T15:19:15.884437+0000","last_became_active":"2024-04-15T15:19:14.877367+0000","last_became_peered":"2024-04-15T15:19:14.877367+0000","last_unstale":"2024-04-15T15:19:15.884437+0000","last_undegraded":"2024-04-15T15:19:15.884437+0000","last_fullsized":"2024-04-15T15:19:15.884437+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:13.827511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:13.827511+0000","last_clean_scrub_stamp":"2024-04-15T15:19:13.827511+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:32:55.902561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:24.396509+0000","last_change":"2024-04-15T15:19:24.396509+0000","last_active":"2024-04-15T15:19:24.396509+0000","last_peered":"2024-04-15T15:19:24.396509+0000","last_clean":"2024-04-15T15:19:24.396509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:24.396509+0000","last_undegraded":"2024-04-15T15:19:24.396509+0000","last_fullsized":"2024-04-15T15:19:24.396509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:24.396509+0000","last_change":"2024-04-15T15:19:24.396509+0000","last_active":"2024-04-15T15:19:24.396509+0000","last_peered":"2024-04-15T15:19:24.396509+0000","last_clean":"2024-04-15T15:19:24.396509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:24.396509+0000","last_undegraded":"2024-04-15T15:19:24.396509+0000","last_fullsized":"2024-04-15T15:19:24.396509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:24.396509+0000","last_change":"2024-04-15T15:19:24.396509+0000","last_active":"2024-04-15T15:19:24.396509+0000","last_peered":"2024-04-15T15:19:24.396509+0000","last_clean":"2024-04-15T15:19:24.396509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:24.396509+0000","last_undegraded":"2024-04-15T15:19:24.396509+0000","last_fullsized":"2024-04-15T15:19:24.396509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:24.396509+0000","last_change":"2024-04-15T15:19:24.396509+0000","last_active":"2024-04-15T15:19:24.396509+0000","last_peered":"2024-04-15T15:19:24.396509+0000","last_clean":"2024-04-15T15:19:24.396509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:24.396509+0000","last_undegraded":"2024-04-15T15:19:24.396509+0000","last_fullsized":"2024-04-15T15:19:24.396509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:24.396509+0000","last_change":"2024-04-15T15:19:24.396509+0000","last_active":"2024-04-15T15:19:24.396509+0000","last_peered":"2024-04-15T15:19:24.396509+0000","last_clean":"2024-04-15T15:19:24.396509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:24.396509+0000","last_undegraded":"2024-04-15T15:19:24.396509+0000","last_fullsized":"2024-04-15T15:19:24.396509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:24.396509+0000","last_change":"2024-04-15T15:19:24.396509+0000","last_active":"2024-04-15T15:19:24.396509+0000","last_peered":"2024-04-15T15:19:24.396509+0000","last_clean":"2024-04-15T15:19:24.396509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:24.396509+0000","last_undegraded":"2024-04-15T15:19:24.396509+0000","last_fullsized":"2024-04-15T15:19:24.396509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:24.396509+0000","last_change":"2024-04-15T15:19:24.396509+0000","last_active":"2024-04-15T15:19:24.396509+0000","last_peered":"2024-04-15T15:19:24.396509+0000","last_clean":"2024-04-15T15:19:24.396509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:24.396509+0000","last_undegraded":"2024-04-15T15:19:24.396509+0000","last_fullsized":"2024-04-15T15:19:24.396509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:24.396509+0000","last_change":"2024-04-15T15:19:24.396509+0000","last_active":"2024-04-15T15:19:24.396509+0000","last_peered":"2024-04-15T15:19:24.396509+0000","last_clean":"2024-04-15T15:19:24.396509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:24.396509+0000","last_undegraded":"2024-04-15T15:19:24.396509+0000","last_fullsized":"2024-04-15T15:19:24.396509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:24.396509+0000","last_change":"2024-04-15T15:19:24.396509+0000","last_active":"2024-04-15T15:19:24.396509+0000","last_peered":"2024-04-15T15:19:24.396509+0000","last_clean":"2024-04-15T15:19:24.396509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:24.396509+0000","last_undegraded":"2024-04-15T15:19:24.396509+0000","last_fullsized":"2024-04-15T15:19:24.396509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:24.396509+0000","last_change":"2024-04-15T15:19:24.396509+0000","last_active":"2024-04-15T15:19:24.396509+0000","last_peered":"2024-04-15T15:19:24.396509+0000","last_clean":"2024-04-15T15:19:24.396509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:24.396509+0000","last_undegraded":"2024-04-15T15:19:24.396509+0000","last_fullsized":"2024-04-15T15:19:24.396509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:24.396509+0000","last_change":"2024-04-15T15:19:24.396509+0000","last_active":"2024-04-15T15:19:24.396509+0000","last_peered":"2024-04-15T15:19:24.396509+0000","last_clean":"2024-04-15T15:19:24.396509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:24.396509+0000","last_undegraded":"2024-04-15T15:19:24.396509+0000","last_fullsized":"2024-04-15T15:19:24.396509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:24.396509+0000","last_change":"2024-04-15T15:19:24.396509+0000","last_active":"2024-04-15T15:19:24.396509+0000","last_peered":"2024-04-15T15:19:24.396509+0000","last_clean":"2024-04-15T15:19:24.396509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:24.396509+0000","last_undegraded":"2024-04-15T15:19:24.396509+0000","last_fullsized":"2024-04-15T15:19:24.396509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:24.396509+0000","last_change":"2024-04-15T15:19:24.396509+0000","last_active":"2024-04-15T15:19:24.396509+0000","last_peered":"2024-04-15T15:19:24.396509+0000","last_clean":"2024-04-15T15:19:24.396509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:24.396509+0000","last_undegraded":"2024-04-15T15:19:24.396509+0000","last_fullsized":"2024-04-15T15:19:24.396509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:24.396509+0000","last_change":"2024-04-15T15:19:24.396509+0000","last_active":"2024-04-15T15:19:24.396509+0000","last_peered":"2024-04-15T15:19:24.396509+0000","last_clean":"2024-04-15T15:19:24.396509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:24.396509+0000","last_undegraded":"2024-04-15T15:19:24.396509+0000","last_fullsized":"2024-04-15T15:19:24.396509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:24.396509+0000","last_change":"2024-04-15T15:19:24.396509+0000","last_active":"2024-04-15T15:19:24.396509+0000","last_peered":"2024-04-15T15:19:24.396509+0000","last_clean":"2024-04-15T15:19:24.396509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:24.396509+0000","last_undegraded":"2024-04-15T15:19:24.396509+0000","last_fullsized":"2024-04-15T15:19:24.396509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:25.400451+0000","last_change":"2024-04-15T15:19:25.400451+0000","last_active":"2024-04-15T15:19:25.400451+0000","last_peered":"2024-04-15T15:19:25.400451+0000","last_clean":"2024-04-15T15:19:25.400451+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:25.400451+0000","last_undegraded":"2024-04-15T15:19:25.400451+0000","last_fullsized":"2024-04-15T15:19:25.400451+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-15T15:19:24.396509+0000","last_change":"2024-04-15T15:19:24.396509+0000","last_active":"2024-04-15T15:19:24.396509+0000","last_peered":"2024-04-15T15:19:24.396509+0000","last_clean":"2024-04-15T15:19:24.396509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:19:24.396509+0000","last_undegraded":"2024-04-15T15:19:24.396509+0000","last_fullsized":"2024-04-15T15:19:24.396509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27752,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344088,"statfs":{"total":96636764160,"available":96608346112,"internally_reserved":0,"allocated":749568,"data_stored":636135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27180,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27180,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27180,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27752,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344088,"statfs":{"total":96636764160,"available":96608346112,"internally_reserved":0,"allocated":749568,"data_stored":636135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-15T15:19:29.414 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-15T15:19:29.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.552+0000 7f6d12b15640 1 Processor -- start 2024-04-15T15:19:29.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.552+0000 7f6d12b15640 1 -- start start 2024-04-15T15:19:29.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.553+0000 7f6d12b15640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6d0c15ec60 0x7f6d0c0d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:29.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.553+0000 7f6d12b15640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d0c0714f0 con 0x7f6d0c09b780 2024-04-15T15:19:29.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.553+0000 7f6d12b15640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d0c0d1940 con 0x7f6d0c09b150 2024-04-15T15:19:29.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.553+0000 7f6d12b15640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f6d0c06cad0 con 0x7f6d0c15ec60 2024-04-15T15:19:29.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.553+0000 7f6d108b1640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6d0c15ec60 0x7f6d0c0d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:29.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.553+0000 7f6d03fff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f6d0c09b780 0x7f6d0c09aa40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:40338/0 (socket says 172.21.15.105:40338) 2024-04-15T15:19:29.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.553+0000 7f6d108b1640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6d0c15ec60 0x7f6d0c0d1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57244/0 (socket says 172.21.15.105:57244) 2024-04-15T15:19:29.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.553+0000 7f6d03fff640 1 -- 172.21.15.105:0/314694659 learned_addr learned my addr 172.21.15.105:0/314694659 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:19:29.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.553+0000 7f6d108b1640 1 -- 172.21.15.105:0/314694659 >> v1:172.21.15.154:6789/0 conn(0x7f6d0c09b150 legacy=0x7f6d0c0cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:29.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.553+0000 7f6d108b1640 1 -- 172.21.15.105:0/314694659 >> v1:172.21.15.105:6789/0 conn(0x7f6d0c09b780 legacy=0x7f6d0c09aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:29.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.553+0000 7f6d037fe640 1 -- 172.21.15.105:0/314694659 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1652123915 0 0) 0x7f6d0c0714f0 con 0x7f6d0c09b780 2024-04-15T15:19:29.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.553+0000 7f6d108b1640 1 -- 172.21.15.105:0/314694659 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d0c0ccd60 con 0x7f6d0c15ec60 2024-04-15T15:19:29.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.554+0000 7f6d108b1640 1 --2- 172.21.15.105:0/314694659 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d0c15ec60 0x7f6d0c0d1210 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6d080097a0 tx=0x7f6d0802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3151ee5f2ee19e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:29.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.554+0000 7f6d037fe640 1 -- 172.21.15.105:0/314694659 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d08035020 con 0x7f6d0c15ec60 2024-04-15T15:19:29.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.554+0000 7f6d037fe640 1 -- 172.21.15.105:0/314694659 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d08031830 con 0x7f6d0c15ec60 2024-04-15T15:19:29.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.554+0000 7f6d037fe640 1 -- 172.21.15.105:0/314694659 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d08031c00 con 0x7f6d0c15ec60 2024-04-15T15:19:29.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.555+0000 7f6d12b15640 1 -- 172.21.15.105:0/314694659 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d0c15ec60 msgr2=0x7f6d0c0d1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:29.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.555+0000 7f6d12b15640 1 --2- 172.21.15.105:0/314694659 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d0c15ec60 0x7f6d0c0d1210 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6d080097a0 tx=0x7f6d0802b8d0 comp rx=0 tx=0).stop 2024-04-15T15:19:29.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.555+0000 7f6d12b15640 1 -- 172.21.15.105:0/314694659 shutdown_connections 2024-04-15T15:19:29.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.555+0000 7f6d12b15640 1 --2- 172.21.15.105:0/314694659 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d0c15ec60 0x7f6d0c0d1210 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:29.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.555+0000 7f6d12b15640 1 -- 172.21.15.105:0/314694659 >> 172.21.15.105:0/314694659 conn(0x7f6d0c0d2570 msgr2=0x7f6d0c0d2980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:29.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.555+0000 7f6d12b15640 1 -- 172.21.15.105:0/314694659 shutdown_connections 2024-04-15T15:19:29.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.555+0000 7f6d12b15640 1 -- 172.21.15.105:0/314694659 wait complete. 2024-04-15T15:19:29.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.556+0000 7f6d12b15640 1 Processor -- start 2024-04-15T15:19:29.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.556+0000 7f6d12b15640 1 -- start start 2024-04-15T15:19:29.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.557+0000 7f6d12b15640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d0c09b150 0x7f6d0c0b27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:29.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.557+0000 7f6d12b15640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d0c09b780 0x7f6d0c0ca760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:29.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.557+0000 7f6d12b15640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6d0c0caca0 0x7f6d0c158920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:29.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.557+0000 7f6d12b15640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0c0cdf50 con 0x7f6d0c09b780 2024-04-15T15:19:29.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.557+0000 7f6d12b15640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0c0cd3b0 con 0x7f6d0c09b150 2024-04-15T15:19:29.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.557+0000 7f6d12b15640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f6d0c0ce0d0 con 0x7f6d0c0caca0 2024-04-15T15:19:29.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.557+0000 7f6d03fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d0c09b780 0x7f6d0c0ca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:29.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.557+0000 7f6d03fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d0c09b780 0x7f6d0c0ca760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57250/0 (socket says 172.21.15.105:57250) 2024-04-15T15:19:29.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.557+0000 7f6d03fff640 1 -- 172.21.15.105:0/2971863808 learned_addr learned my addr 172.21.15.105:0/2971863808 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:29.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.557+0000 7f6d108b1640 1 --2- 172.21.15.105:0/2971863808 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d0c09b150 0x7f6d0c0b27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:29.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.557+0000 7f6d110b2640 1 --2- 172.21.15.105:0/2971863808 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6d0c0caca0 0x7f6d0c158920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:29.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.557+0000 7f6d03fff640 1 -- 172.21.15.105:0/2971863808 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6d0c0caca0 msgr2=0x7f6d0c158920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:29.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.557+0000 7f6d03fff640 1 --2- 172.21.15.105:0/2971863808 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6d0c0caca0 0x7f6d0c158920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:29.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.557+0000 7f6d03fff640 1 -- 172.21.15.105:0/2971863808 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d0c09b150 msgr2=0x7f6d0c0b27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:29.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.557+0000 7f6d03fff640 1 --2- 172.21.15.105:0/2971863808 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d0c09b150 0x7f6d0c0b27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:29.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.557+0000 7f6d03fff640 1 -- 172.21.15.105:0/2971863808 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d0c1612d0 con 0x7f6d0c09b780 2024-04-15T15:19:29.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.557+0000 7f6d03fff640 1 --2- 172.21.15.105:0/2971863808 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d0c09b780 0x7f6d0c0ca760 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc00c4d0 tx=0x7f6cfc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:29.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.557+0000 7f6d017fa640 1 -- 172.21.15.105:0/2971863808 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc018070 con 0x7f6d0c09b780 2024-04-15T15:19:29.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.557+0000 7f6d017fa640 1 -- 172.21.15.105:0/2971863808 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cfc007500 con 0x7f6d0c09b780 2024-04-15T15:19:29.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.557+0000 7f6d12b15640 1 -- 172.21.15.105:0/2971863808 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d0c1615b0 con 0x7f6d0c09b780 2024-04-15T15:19:29.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.557+0000 7f6d017fa640 1 -- 172.21.15.105:0/2971863808 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc004a20 con 0x7f6d0c09b780 2024-04-15T15:19:29.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.558+0000 7f6d12b15640 1 -- 172.21.15.105:0/2971863808 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d0c161b70 con 0x7f6d0c09b780 2024-04-15T15:19:29.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.558+0000 7f6d12b15640 1 -- 172.21.15.105:0/2971863808 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d0c15e4c0 con 0x7f6d0c09b780 2024-04-15T15:19:29.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.559+0000 7f6d017fa640 1 -- 172.21.15.105:0/2971863808 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6cfc01b070 con 0x7f6d0c09b780 2024-04-15T15:19:29.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.559+0000 7f6d017fa640 1 --2- 172.21.15.105:0/2971863808 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f6cf806c8f0 0x7f6cf806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:29.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.559+0000 7f6d017fa640 1 -- 172.21.15.105:0/2971863808 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6cfc08aeb0 con 0x7f6d0c09b780 2024-04-15T15:19:29.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.559+0000 7f6d108b1640 1 --2- 172.21.15.105:0/2971863808 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f6cf806c8f0 0x7f6cf806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:29.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.560+0000 7f6d108b1640 1 --2- 172.21.15.105:0/2971863808 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f6cf806c8f0 0x7f6cf806edb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6d08036000 tx=0x7f6d0803c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:29.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.564+0000 7f6d017fa640 1 -- 172.21.15.105:0/2971863808 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6d0c15e4c0 con 0x7f6d0c09b780 2024-04-15T15:19:29.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.735+0000 7f6d12b15640 1 -- 172.21.15.105:0/2971863808 --> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6d0c068a40 con 0x7f6cf806c8f0 2024-04-15T15:19:29.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.741+0000 7f6d017fa640 1 -- 172.21.15.105:0/2971863808 <== mgr.4107 v2:172.21.15.105:6832/3506549322 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251948 (secure 0 0 0) 0x7f6d08085840 con 0x7f6cf806c8f0 2024-04-15T15:19:29.746 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:19:29.747 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-04-15T15:19:29.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.744+0000 7f6d12b15640 1 -- 172.21.15.105:0/2971863808 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f6cf806c8f0 msgr2=0x7f6cf806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:29.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.744+0000 7f6d12b15640 1 --2- 172.21.15.105:0/2971863808 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f6cf806c8f0 0x7f6cf806edb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6d08036000 tx=0x7f6d0803c040 comp rx=0 tx=0).stop 2024-04-15T15:19:29.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.744+0000 7f6d12b15640 1 -- 172.21.15.105:0/2971863808 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d0c09b780 msgr2=0x7f6d0c0ca760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:29.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.744+0000 7f6d12b15640 1 --2- 172.21.15.105:0/2971863808 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d0c09b780 0x7f6d0c0ca760 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc00c4d0 tx=0x7f6cfc00c990 comp rx=0 tx=0).stop 2024-04-15T15:19:29.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.744+0000 7f6d12b15640 1 -- 172.21.15.105:0/2971863808 shutdown_connections 2024-04-15T15:19:29.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.744+0000 7f6d12b15640 1 --2- 172.21.15.105:0/2971863808 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6d0c0caca0 0x7f6d0c158920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:29.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.744+0000 7f6d12b15640 1 --2- 172.21.15.105:0/2971863808 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f6cf806c8f0 0x7f6cf806edb0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:29.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.744+0000 7f6d12b15640 1 --2- 172.21.15.105:0/2971863808 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d0c09b780 0x7f6d0c0ca760 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:29.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.744+0000 7f6d12b15640 1 --2- 172.21.15.105:0/2971863808 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d0c09b150 0x7f6d0c0b27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:29.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.744+0000 7f6d12b15640 1 -- 172.21.15.105:0/2971863808 >> 172.21.15.105:0/2971863808 conn(0x7f6d0c0d2570 msgr2=0x7f6d0c09aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:29.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.744+0000 7f6d12b15640 1 -- 172.21.15.105:0/2971863808 shutdown_connections 2024-04-15T15:19:29.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.744+0000 7f6d12b15640 1 -- 172.21.15.105:0/2971863808 wait complete. 2024-04-15T15:19:29.758 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-04-15T15:19:29.734400+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219024,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755696,"statfs":{"total":773094113280,"available":772869832704,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.333055"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.724996+0000","last_change":"2024-04-15T15:19:26.440158+0000","last_active":"2024-04-15T15:19:27.724996+0000","last_peered":"2024-04-15T15:19:27.724996+0000","last_clean":"2024-04-15T15:19:27.724996+0000","last_became_active":"2024-04-15T15:19:26.439860+0000","last_became_peered":"2024-04-15T15:19:26.439860+0000","last_unstale":"2024-04-15T15:19:27.724996+0000","last_undegraded":"2024-04-15T15:19:27.724996+0000","last_fullsized":"2024-04-15T15:19:27.724996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:39:29.689447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.440059+0000","last_change":"2024-04-15T15:19:26.426289+0000","last_active":"2024-04-15T15:19:27.440059+0000","last_peered":"2024-04-15T15:19:27.440059+0000","last_clean":"2024-04-15T15:19:27.440059+0000","last_became_active":"2024-04-15T15:19:26.425937+0000","last_became_peered":"2024-04-15T15:19:26.425937+0000","last_unstale":"2024-04-15T15:19:27.440059+0000","last_undegraded":"2024-04-15T15:19:27.440059+0000","last_fullsized":"2024-04-15T15:19:27.440059+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:06:20.973059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.773961+0000","last_change":"2024-04-15T15:19:26.425640+0000","last_active":"2024-04-15T15:19:27.773961+0000","last_peered":"2024-04-15T15:19:27.773961+0000","last_clean":"2024-04-15T15:19:27.773961+0000","last_became_active":"2024-04-15T15:19:26.425228+0000","last_became_peered":"2024-04-15T15:19:26.425228+0000","last_unstale":"2024-04-15T15:19:27.773961+0000","last_undegraded":"2024-04-15T15:19:27.773961+0000","last_fullsized":"2024-04-15T15:19:27.773961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:17:45.061394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774465+0000","last_change":"2024-04-15T15:19:26.439598+0000","last_active":"2024-04-15T15:19:27.774465+0000","last_peered":"2024-04-15T15:19:27.774465+0000","last_clean":"2024-04-15T15:19:27.774465+0000","last_became_active":"2024-04-15T15:19:26.439290+0000","last_became_peered":"2024-04-15T15:19:26.439290+0000","last_unstale":"2024-04-15T15:19:27.774465+0000","last_undegraded":"2024-04-15T15:19:27.774465+0000","last_fullsized":"2024-04-15T15:19:27.774465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:35:32.148665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.724710+0000","last_change":"2024-04-15T15:19:26.433863+0000","last_active":"2024-04-15T15:19:27.724710+0000","last_peered":"2024-04-15T15:19:27.724710+0000","last_clean":"2024-04-15T15:19:27.724710+0000","last_became_active":"2024-04-15T15:19:26.433559+0000","last_became_peered":"2024-04-15T15:19:26.433559+0000","last_unstale":"2024-04-15T15:19:27.724710+0000","last_undegraded":"2024-04-15T15:19:27.724710+0000","last_fullsized":"2024-04-15T15:19:27.724710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T03:07:46.810235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774361+0000","last_change":"2024-04-15T15:19:26.442278+0000","last_active":"2024-04-15T15:19:27.774361+0000","last_peered":"2024-04-15T15:19:27.774361+0000","last_clean":"2024-04-15T15:19:27.774361+0000","last_became_active":"2024-04-15T15:19:26.441910+0000","last_became_peered":"2024-04-15T15:19:26.441910+0000","last_unstale":"2024-04-15T15:19:27.774361+0000","last_undegraded":"2024-04-15T15:19:27.774361+0000","last_fullsized":"2024-04-15T15:19:27.774361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:05:03.041116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.724443+0000","last_change":"2024-04-15T15:19:26.430368+0000","last_active":"2024-04-15T15:19:27.724443+0000","last_peered":"2024-04-15T15:19:27.724443+0000","last_clean":"2024-04-15T15:19:27.724443+0000","last_became_active":"2024-04-15T15:19:26.430083+0000","last_became_peered":"2024-04-15T15:19:26.430083+0000","last_unstale":"2024-04-15T15:19:27.724443+0000","last_undegraded":"2024-04-15T15:19:27.724443+0000","last_fullsized":"2024-04-15T15:19:27.724443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:52:38.985490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.417632+0000","last_change":"2024-04-15T15:19:26.444027+0000","last_active":"2024-04-15T15:19:27.417632+0000","last_peered":"2024-04-15T15:19:27.417632+0000","last_clean":"2024-04-15T15:19:27.417632+0000","last_became_active":"2024-04-15T15:19:26.443699+0000","last_became_peered":"2024-04-15T15:19:26.443699+0000","last_unstale":"2024-04-15T15:19:27.417632+0000","last_undegraded":"2024-04-15T15:19:27.417632+0000","last_fullsized":"2024-04-15T15:19:27.417632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:19:31.027967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.725018+0000","last_change":"2024-04-15T15:19:26.443898+0000","last_active":"2024-04-15T15:19:27.725018+0000","last_peered":"2024-04-15T15:19:27.725018+0000","last_clean":"2024-04-15T15:19:27.725018+0000","last_became_active":"2024-04-15T15:19:26.443600+0000","last_became_peered":"2024-04-15T15:19:26.443600+0000","last_unstale":"2024-04-15T15:19:27.725018+0000","last_undegraded":"2024-04-15T15:19:27.725018+0000","last_fullsized":"2024-04-15T15:19:27.725018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:07:04.317951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.420914+0000","last_change":"2024-04-15T15:19:26.445705+0000","last_active":"2024-04-15T15:19:27.420914+0000","last_peered":"2024-04-15T15:19:27.420914+0000","last_clean":"2024-04-15T15:19:27.420914+0000","last_became_active":"2024-04-15T15:19:26.445421+0000","last_became_peered":"2024-04-15T15:19:26.445421+0000","last_unstale":"2024-04-15T15:19:27.420914+0000","last_undegraded":"2024-04-15T15:19:27.420914+0000","last_fullsized":"2024-04-15T15:19:27.420914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:06:20.973059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.420805+0000","last_change":"2024-04-15T15:19:26.441881+0000","last_active":"2024-04-15T15:19:27.420805+0000","last_peered":"2024-04-15T15:19:27.420805+0000","last_clean":"2024-04-15T15:19:27.420805+0000","last_became_active":"2024-04-15T15:19:26.441610+0000","last_became_peered":"2024-04-15T15:19:26.441610+0000","last_unstale":"2024-04-15T15:19:27.420805+0000","last_undegraded":"2024-04-15T15:19:27.420805+0000","last_fullsized":"2024-04-15T15:19:27.420805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:55:46.969871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.421116+0000","last_change":"2024-04-15T15:19:26.443171+0000","last_active":"2024-04-15T15:19:27.421116+0000","last_peered":"2024-04-15T15:19:27.421116+0000","last_clean":"2024-04-15T15:19:27.421116+0000","last_became_active":"2024-04-15T15:19:26.442896+0000","last_became_peered":"2024-04-15T15:19:26.442896+0000","last_unstale":"2024-04-15T15:19:27.421116+0000","last_undegraded":"2024-04-15T15:19:27.421116+0000","last_fullsized":"2024-04-15T15:19:27.421116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:31:12.317442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.417978+0000","last_change":"2024-04-15T15:19:26.442097+0000","last_active":"2024-04-15T15:19:27.417978+0000","last_peered":"2024-04-15T15:19:27.417978+0000","last_clean":"2024-04-15T15:19:27.417978+0000","last_became_active":"2024-04-15T15:19:26.441823+0000","last_became_peered":"2024-04-15T15:19:26.441823+0000","last_unstale":"2024-04-15T15:19:27.417978+0000","last_undegraded":"2024-04-15T15:19:27.417978+0000","last_fullsized":"2024-04-15T15:19:27.417978+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:05:03.041116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.773787+0000","last_change":"2024-04-15T15:19:26.428713+0000","last_active":"2024-04-15T15:19:27.773787+0000","last_peered":"2024-04-15T15:19:27.773787+0000","last_clean":"2024-04-15T15:19:27.773787+0000","last_became_active":"2024-04-15T15:19:26.428434+0000","last_became_peered":"2024-04-15T15:19:26.428434+0000","last_unstale":"2024-04-15T15:19:27.773787+0000","last_undegraded":"2024-04-15T15:19:27.773787+0000","last_fullsized":"2024-04-15T15:19:27.773787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:03:15.458694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.439725+0000","last_change":"2024-04-15T15:19:26.436898+0000","last_active":"2024-04-15T15:19:27.439725+0000","last_peered":"2024-04-15T15:19:27.439725+0000","last_clean":"2024-04-15T15:19:27.439725+0000","last_became_active":"2024-04-15T15:19:26.436643+0000","last_became_peered":"2024-04-15T15:19:26.436643+0000","last_unstale":"2024-04-15T15:19:27.439725+0000","last_undegraded":"2024-04-15T15:19:27.439725+0000","last_fullsized":"2024-04-15T15:19:27.439725+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:57:45.149303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774246+0000","last_change":"2024-04-15T15:19:26.439598+0000","last_active":"2024-04-15T15:19:27.774246+0000","last_peered":"2024-04-15T15:19:27.774246+0000","last_clean":"2024-04-15T15:19:27.774246+0000","last_became_active":"2024-04-15T15:19:26.439335+0000","last_became_peered":"2024-04-15T15:19:26.439335+0000","last_unstale":"2024-04-15T15:19:27.774246+0000","last_undegraded":"2024-04-15T15:19:27.774246+0000","last_fullsized":"2024-04-15T15:19:27.774246+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:55:46.969871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774820+0000","last_change":"2024-04-15T15:19:26.445594+0000","last_active":"2024-04-15T15:19:27.774820+0000","last_peered":"2024-04-15T15:19:27.774820+0000","last_clean":"2024-04-15T15:19:27.774820+0000","last_became_active":"2024-04-15T15:19:26.445306+0000","last_became_peered":"2024-04-15T15:19:26.445306+0000","last_unstale":"2024-04-15T15:19:27.774820+0000","last_undegraded":"2024-04-15T15:19:27.774820+0000","last_fullsized":"2024-04-15T15:19:27.774820+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:47:07.377973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.554342+0000","last_change":"2024-04-15T15:19:26.424093+0000","last_active":"2024-04-15T15:19:27.554342+0000","last_peered":"2024-04-15T15:19:27.554342+0000","last_clean":"2024-04-15T15:19:27.554342+0000","last_became_active":"2024-04-15T15:19:26.423816+0000","last_became_peered":"2024-04-15T15:19:26.423816+0000","last_unstale":"2024-04-15T15:19:27.554342+0000","last_undegraded":"2024-04-15T15:19:27.554342+0000","last_fullsized":"2024-04-15T15:19:27.554342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:52:03.007433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.725002+0000","last_change":"2024-04-15T15:19:26.437633+0000","last_active":"2024-04-15T15:19:27.725002+0000","last_peered":"2024-04-15T15:19:27.725002+0000","last_clean":"2024-04-15T15:19:27.725002+0000","last_became_active":"2024-04-15T15:19:26.437000+0000","last_became_peered":"2024-04-15T15:19:26.437000+0000","last_unstale":"2024-04-15T15:19:27.725002+0000","last_undegraded":"2024-04-15T15:19:27.725002+0000","last_fullsized":"2024-04-15T15:19:27.725002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:49:01.189045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.420621+0000","last_change":"2024-04-15T15:19:26.431490+0000","last_active":"2024-04-15T15:19:27.420621+0000","last_peered":"2024-04-15T15:19:27.420621+0000","last_clean":"2024-04-15T15:19:27.420621+0000","last_became_active":"2024-04-15T15:19:26.431235+0000","last_became_peered":"2024-04-15T15:19:26.431235+0000","last_unstale":"2024-04-15T15:19:27.420621+0000","last_undegraded":"2024-04-15T15:19:27.420621+0000","last_fullsized":"2024-04-15T15:19:27.420621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:07:02.244933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774078+0000","last_change":"2024-04-15T15:19:26.433079+0000","last_active":"2024-04-15T15:19:27.774078+0000","last_peered":"2024-04-15T15:19:27.774078+0000","last_clean":"2024-04-15T15:19:27.774078+0000","last_became_active":"2024-04-15T15:19:26.432075+0000","last_became_peered":"2024-04-15T15:19:26.432075+0000","last_unstale":"2024-04-15T15:19:27.774078+0000","last_undegraded":"2024-04-15T15:19:27.774078+0000","last_fullsized":"2024-04-15T15:19:27.774078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:55:46.969871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774692+0000","last_change":"2024-04-15T15:19:26.442575+0000","last_active":"2024-04-15T15:19:27.774692+0000","last_peered":"2024-04-15T15:19:27.774692+0000","last_clean":"2024-04-15T15:19:27.774692+0000","last_became_active":"2024-04-15T15:19:26.441316+0000","last_became_peered":"2024-04-15T15:19:26.441316+0000","last_unstale":"2024-04-15T15:19:27.774692+0000","last_undegraded":"2024-04-15T15:19:27.774692+0000","last_fullsized":"2024-04-15T15:19:27.774692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T03:07:46.810235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.417731+0000","last_change":"2024-04-15T15:19:26.437545+0000","last_active":"2024-04-15T15:19:27.417731+0000","last_peered":"2024-04-15T15:19:27.417731+0000","last_clean":"2024-04-15T15:19:27.417731+0000","last_became_active":"2024-04-15T15:19:26.437018+0000","last_became_peered":"2024-04-15T15:19:26.437018+0000","last_unstale":"2024-04-15T15:19:27.417731+0000","last_undegraded":"2024-04-15T15:19:27.417731+0000","last_fullsized":"2024-04-15T15:19:27.417731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:57:45.149303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774315+0000","last_change":"2024-04-15T15:19:26.438924+0000","last_active":"2024-04-15T15:19:27.774315+0000","last_peered":"2024-04-15T15:19:27.774315+0000","last_clean":"2024-04-15T15:19:27.774315+0000","last_became_active":"2024-04-15T15:19:26.438615+0000","last_became_peered":"2024-04-15T15:19:26.438615+0000","last_unstale":"2024-04-15T15:19:27.774315+0000","last_undegraded":"2024-04-15T15:19:27.774315+0000","last_fullsized":"2024-04-15T15:19:27.774315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:06:20.973059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.417837+0000","last_change":"2024-04-15T15:19:26.446195+0000","last_active":"2024-04-15T15:19:27.417837+0000","last_peered":"2024-04-15T15:19:27.417837+0000","last_clean":"2024-04-15T15:19:27.417837+0000","last_became_active":"2024-04-15T15:19:26.445872+0000","last_became_peered":"2024-04-15T15:19:26.445872+0000","last_unstale":"2024-04-15T15:19:27.417837+0000","last_undegraded":"2024-04-15T15:19:27.417837+0000","last_fullsized":"2024-04-15T15:19:27.417837+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:07:02.244933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.421026+0000","last_change":"2024-04-15T15:19:26.429073+0000","last_active":"2024-04-15T15:19:27.421026+0000","last_peered":"2024-04-15T15:19:27.421026+0000","last_clean":"2024-04-15T15:19:27.421026+0000","last_became_active":"2024-04-15T15:19:26.428755+0000","last_became_peered":"2024-04-15T15:19:26.428755+0000","last_unstale":"2024-04-15T15:19:27.421026+0000","last_undegraded":"2024-04-15T15:19:27.421026+0000","last_fullsized":"2024-04-15T15:19:27.421026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:35:32.148665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.439805+0000","last_change":"2024-04-15T15:19:26.432582+0000","last_active":"2024-04-15T15:19:27.439805+0000","last_peered":"2024-04-15T15:19:27.439805+0000","last_clean":"2024-04-15T15:19:27.439805+0000","last_became_active":"2024-04-15T15:19:26.432334+0000","last_became_peered":"2024-04-15T15:19:26.432334+0000","last_unstale":"2024-04-15T15:19:27.439805+0000","last_undegraded":"2024-04-15T15:19:27.439805+0000","last_fullsized":"2024-04-15T15:19:27.439805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:07:02.244933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.418110+0000","last_change":"2024-04-15T15:19:26.436504+0000","last_active":"2024-04-15T15:19:27.418110+0000","last_peered":"2024-04-15T15:19:27.418110+0000","last_clean":"2024-04-15T15:19:27.418110+0000","last_became_active":"2024-04-15T15:19:26.436237+0000","last_became_peered":"2024-04-15T15:19:26.436237+0000","last_unstale":"2024-04-15T15:19:27.418110+0000","last_undegraded":"2024-04-15T15:19:27.418110+0000","last_fullsized":"2024-04-15T15:19:27.418110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:15:06.003211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774078+0000","last_change":"2024-04-15T15:19:26.429813+0000","last_active":"2024-04-15T15:19:27.774078+0000","last_peered":"2024-04-15T15:19:27.774078+0000","last_clean":"2024-04-15T15:19:27.774078+0000","last_became_active":"2024-04-15T15:19:26.429507+0000","last_became_peered":"2024-04-15T15:19:26.429507+0000","last_unstale":"2024-04-15T15:19:27.774078+0000","last_undegraded":"2024-04-15T15:19:27.774078+0000","last_fullsized":"2024-04-15T15:19:27.774078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:06:20.973059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774710+0000","last_change":"2024-04-15T15:19:26.442277+0000","last_active":"2024-04-15T15:19:27.774710+0000","last_peered":"2024-04-15T15:19:27.774710+0000","last_clean":"2024-04-15T15:19:27.774710+0000","last_became_active":"2024-04-15T15:19:26.441630+0000","last_became_peered":"2024-04-15T15:19:26.441630+0000","last_unstale":"2024-04-15T15:19:27.774710+0000","last_undegraded":"2024-04-15T15:19:27.774710+0000","last_fullsized":"2024-04-15T15:19:27.774710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:06:21.060519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.554197+0000","last_change":"2024-04-15T15:19:26.439042+0000","last_active":"2024-04-15T15:19:27.554197+0000","last_peered":"2024-04-15T15:19:27.554197+0000","last_clean":"2024-04-15T15:19:27.554197+0000","last_became_active":"2024-04-15T15:19:26.438709+0000","last_became_peered":"2024-04-15T15:19:26.438709+0000","last_unstale":"2024-04-15T15:19:27.554197+0000","last_undegraded":"2024-04-15T15:19:27.554197+0000","last_fullsized":"2024-04-15T15:19:27.554197+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:21:56.467880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.420350+0000","last_change":"2024-04-15T15:19:26.423461+0000","last_active":"2024-04-15T15:19:27.420350+0000","last_peered":"2024-04-15T15:19:27.420350+0000","last_clean":"2024-04-15T15:19:27.420350+0000","last_became_active":"2024-04-15T15:19:26.423171+0000","last_became_peered":"2024-04-15T15:19:26.423171+0000","last_unstale":"2024-04-15T15:19:27.420350+0000","last_undegraded":"2024-04-15T15:19:27.420350+0000","last_fullsized":"2024-04-15T15:19:27.420350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:57:45.149303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.420526+0000","last_change":"2024-04-15T15:19:26.431616+0000","last_active":"2024-04-15T15:19:27.420526+0000","last_peered":"2024-04-15T15:19:27.420526+0000","last_clean":"2024-04-15T15:19:27.420526+0000","last_became_active":"2024-04-15T15:19:26.430999+0000","last_became_peered":"2024-04-15T15:19:26.430999+0000","last_unstale":"2024-04-15T15:19:27.420526+0000","last_undegraded":"2024-04-15T15:19:27.420526+0000","last_fullsized":"2024-04-15T15:19:27.420526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:55:34.679094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.418188+0000","last_change":"2024-04-15T15:19:25.418290+0000","last_active":"2024-04-15T15:19:27.418188+0000","last_peered":"2024-04-15T15:19:27.418188+0000","last_clean":"2024-04-15T15:19:27.418188+0000","last_became_active":"2024-04-15T15:19:25.418036+0000","last_became_peered":"2024-04-15T15:19:25.418036+0000","last_unstale":"2024-04-15T15:19:27.418188+0000","last_undegraded":"2024-04-15T15:19:27.418188+0000","last_fullsized":"2024-04-15T15:19:27.418188+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:35:36.249861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774176+0000","last_change":"2024-04-15T15:19:26.431800+0000","last_active":"2024-04-15T15:19:27.774176+0000","last_peered":"2024-04-15T15:19:27.774176+0000","last_clean":"2024-04-15T15:19:27.774176+0000","last_became_active":"2024-04-15T15:19:26.431535+0000","last_became_peered":"2024-04-15T15:19:26.431535+0000","last_unstale":"2024-04-15T15:19:27.774176+0000","last_undegraded":"2024-04-15T15:19:27.774176+0000","last_fullsized":"2024-04-15T15:19:27.774176+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:05:03.041116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.421045+0000","last_change":"2024-04-15T15:19:25.422350+0000","last_active":"2024-04-15T15:19:27.421045+0000","last_peered":"2024-04-15T15:19:27.421045+0000","last_clean":"2024-04-15T15:19:27.421045+0000","last_became_active":"2024-04-15T15:19:25.422091+0000","last_became_peered":"2024-04-15T15:19:25.422091+0000","last_unstale":"2024-04-15T15:19:27.421045+0000","last_undegraded":"2024-04-15T15:19:27.421045+0000","last_fullsized":"2024-04-15T15:19:27.421045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:55:33.675151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.417601+0000","last_change":"2024-04-15T15:19:25.427262+0000","last_active":"2024-04-15T15:19:27.417601+0000","last_peered":"2024-04-15T15:19:27.417601+0000","last_clean":"2024-04-15T15:19:27.417601+0000","last_became_active":"2024-04-15T15:19:25.427057+0000","last_became_peered":"2024-04-15T15:19:25.427057+0000","last_unstale":"2024-04-15T15:19:27.417601+0000","last_undegraded":"2024-04-15T15:19:27.417601+0000","last_fullsized":"2024-04-15T15:19:27.417601+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:39:21.336486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.417662+0000","last_change":"2024-04-15T15:19:26.434134+0000","last_active":"2024-04-15T15:19:27.417662+0000","last_peered":"2024-04-15T15:19:27.417662+0000","last_clean":"2024-04-15T15:19:27.417662+0000","last_became_active":"2024-04-15T15:19:26.433852+0000","last_became_peered":"2024-04-15T15:19:26.433852+0000","last_unstale":"2024-04-15T15:19:27.417662+0000","last_undegraded":"2024-04-15T15:19:27.417662+0000","last_fullsized":"2024-04-15T15:19:27.417662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:17:45.061394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774370+0000","last_change":"2024-04-15T15:19:25.423330+0000","last_active":"2024-04-15T15:19:27.774370+0000","last_peered":"2024-04-15T15:19:27.774370+0000","last_clean":"2024-04-15T15:19:27.774370+0000","last_became_active":"2024-04-15T15:19:25.423045+0000","last_became_peered":"2024-04-15T15:19:25.423045+0000","last_unstale":"2024-04-15T15:19:27.774370+0000","last_undegraded":"2024-04-15T15:19:27.774370+0000","last_fullsized":"2024-04-15T15:19:27.774370+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:55:33.675151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.439704+0000","last_change":"2024-04-15T15:19:26.432451+0000","last_active":"2024-04-15T15:19:27.439704+0000","last_peered":"2024-04-15T15:19:27.439704+0000","last_clean":"2024-04-15T15:19:27.439704+0000","last_became_active":"2024-04-15T15:19:26.432197+0000","last_became_peered":"2024-04-15T15:19:26.432197+0000","last_unstale":"2024-04-15T15:19:27.439704+0000","last_undegraded":"2024-04-15T15:19:27.439704+0000","last_fullsized":"2024-04-15T15:19:27.439704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:17:45.061394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.420612+0000","last_change":"2024-04-15T15:19:25.423680+0000","last_active":"2024-04-15T15:19:27.420612+0000","last_peered":"2024-04-15T15:19:27.420612+0000","last_clean":"2024-04-15T15:19:27.420612+0000","last_became_active":"2024-04-15T15:19:25.423361+0000","last_became_peered":"2024-04-15T15:19:25.423361+0000","last_unstale":"2024-04-15T15:19:27.420612+0000","last_undegraded":"2024-04-15T15:19:27.420612+0000","last_fullsized":"2024-04-15T15:19:27.420612+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:30:23.903641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774647+0000","last_change":"2024-04-15T15:19:26.446276+0000","last_active":"2024-04-15T15:19:27.774647+0000","last_peered":"2024-04-15T15:19:27.774647+0000","last_clean":"2024-04-15T15:19:27.774647+0000","last_became_active":"2024-04-15T15:19:26.445956+0000","last_became_peered":"2024-04-15T15:19:26.445956+0000","last_unstale":"2024-04-15T15:19:27.774647+0000","last_undegraded":"2024-04-15T15:19:27.774647+0000","last_fullsized":"2024-04-15T15:19:27.774647+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:15:06.003211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774626+0000","last_change":"2024-04-15T15:19:25.427557+0000","last_active":"2024-04-15T15:19:27.774626+0000","last_peered":"2024-04-15T15:19:27.774626+0000","last_clean":"2024-04-15T15:19:27.774626+0000","last_became_active":"2024-04-15T15:19:25.427264+0000","last_became_peered":"2024-04-15T15:19:25.427264+0000","last_unstale":"2024-04-15T15:19:27.774626+0000","last_undegraded":"2024-04-15T15:19:27.774626+0000","last_fullsized":"2024-04-15T15:19:27.774626+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:30:23.903641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.773986+0000","last_change":"2024-04-15T15:19:26.443195+0000","last_active":"2024-04-15T15:19:27.773986+0000","last_peered":"2024-04-15T15:19:27.773986+0000","last_clean":"2024-04-15T15:19:27.773986+0000","last_became_active":"2024-04-15T15:19:26.442937+0000","last_became_peered":"2024-04-15T15:19:26.442937+0000","last_unstale":"2024-04-15T15:19:27.773986+0000","last_undegraded":"2024-04-15T15:19:27.773986+0000","last_fullsized":"2024-04-15T15:19:27.773986+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:56:54.270550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.439846+0000","last_change":"2024-04-15T15:19:25.419319+0000","last_active":"2024-04-15T15:19:27.439846+0000","last_peered":"2024-04-15T15:19:27.439846+0000","last_clean":"2024-04-15T15:19:27.439846+0000","last_became_active":"2024-04-15T15:19:25.419070+0000","last_became_peered":"2024-04-15T15:19:25.419070+0000","last_unstale":"2024-04-15T15:19:27.439846+0000","last_undegraded":"2024-04-15T15:19:27.439846+0000","last_fullsized":"2024-04-15T15:19:27.439846+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:21:02.027132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.421169+0000","last_change":"2024-04-15T15:19:26.428785+0000","last_active":"2024-04-15T15:19:27.421169+0000","last_peered":"2024-04-15T15:19:27.421169+0000","last_clean":"2024-04-15T15:19:27.421169+0000","last_became_active":"2024-04-15T15:19:26.428518+0000","last_became_peered":"2024-04-15T15:19:26.428518+0000","last_unstale":"2024-04-15T15:19:27.421169+0000","last_undegraded":"2024-04-15T15:19:27.421169+0000","last_fullsized":"2024-04-15T15:19:27.421169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:50:04.102480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.420737+0000","last_change":"2024-04-15T15:19:25.421107+0000","last_active":"2024-04-15T15:19:27.420737+0000","last_peered":"2024-04-15T15:19:27.420737+0000","last_clean":"2024-04-15T15:19:27.420737+0000","last_became_active":"2024-04-15T15:19:25.420203+0000","last_became_peered":"2024-04-15T15:19:25.420203+0000","last_unstale":"2024-04-15T15:19:27.420737+0000","last_undegraded":"2024-04-15T15:19:27.420737+0000","last_fullsized":"2024-04-15T15:19:27.420737+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:21:02.027132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.724752+0000","last_change":"2024-04-15T15:19:26.437300+0000","last_active":"2024-04-15T15:19:27.724752+0000","last_peered":"2024-04-15T15:19:27.724752+0000","last_clean":"2024-04-15T15:19:27.724752+0000","last_became_active":"2024-04-15T15:19:26.436626+0000","last_became_peered":"2024-04-15T15:19:26.436626+0000","last_unstale":"2024-04-15T15:19:27.724752+0000","last_undegraded":"2024-04-15T15:19:27.724752+0000","last_fullsized":"2024-04-15T15:19:27.724752+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:06:21.060519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774152+0000","last_change":"2024-04-15T15:19:25.423086+0000","last_active":"2024-04-15T15:19:27.774152+0000","last_peered":"2024-04-15T15:19:27.774152+0000","last_clean":"2024-04-15T15:19:27.774152+0000","last_became_active":"2024-04-15T15:19:25.422796+0000","last_became_peered":"2024-04-15T15:19:25.422796+0000","last_unstale":"2024-04-15T15:19:27.774152+0000","last_undegraded":"2024-04-15T15:19:27.774152+0000","last_fullsized":"2024-04-15T15:19:27.774152+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:20:16.625168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.773922+0000","last_change":"2024-04-15T15:19:26.433472+0000","last_active":"2024-04-15T15:19:27.773922+0000","last_peered":"2024-04-15T15:19:27.773922+0000","last_clean":"2024-04-15T15:19:27.773922+0000","last_became_active":"2024-04-15T15:19:26.432886+0000","last_became_peered":"2024-04-15T15:19:26.432886+0000","last_unstale":"2024-04-15T15:19:27.773922+0000","last_undegraded":"2024-04-15T15:19:27.773922+0000","last_fullsized":"2024-04-15T15:19:27.773922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:19:31.027967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774842+0000","last_change":"2024-04-15T15:19:25.419868+0000","last_active":"2024-04-15T15:19:27.774842+0000","last_peered":"2024-04-15T15:19:27.774842+0000","last_clean":"2024-04-15T15:19:27.774842+0000","last_became_active":"2024-04-15T15:19:25.419589+0000","last_became_peered":"2024-04-15T15:19:25.419589+0000","last_unstale":"2024-04-15T15:19:27.774842+0000","last_undegraded":"2024-04-15T15:19:27.774842+0000","last_fullsized":"2024-04-15T15:19:27.774842+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:08:25.359023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.724748+0000","last_change":"2024-04-15T15:19:26.446074+0000","last_active":"2024-04-15T15:19:27.724748+0000","last_peered":"2024-04-15T15:19:27.724748+0000","last_clean":"2024-04-15T15:19:27.724748+0000","last_became_active":"2024-04-15T15:19:26.445793+0000","last_became_peered":"2024-04-15T15:19:26.445793+0000","last_unstale":"2024-04-15T15:19:27.724748+0000","last_undegraded":"2024-04-15T15:19:27.724748+0000","last_fullsized":"2024-04-15T15:19:27.724748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:38:37.460808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.420477+0000","last_change":"2024-04-15T15:19:25.420817+0000","last_active":"2024-04-15T15:19:27.420477+0000","last_peered":"2024-04-15T15:19:27.420477+0000","last_clean":"2024-04-15T15:19:27.420477+0000","last_became_active":"2024-04-15T15:19:25.420546+0000","last_became_peered":"2024-04-15T15:19:25.420546+0000","last_unstale":"2024-04-15T15:19:27.420477+0000","last_undegraded":"2024-04-15T15:19:27.420477+0000","last_fullsized":"2024-04-15T15:19:27.420477+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:36:11.468459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.420351+0000","last_change":"2024-04-15T15:19:26.422633+0000","last_active":"2024-04-15T15:19:27.420351+0000","last_peered":"2024-04-15T15:19:27.420351+0000","last_clean":"2024-04-15T15:19:27.420351+0000","last_became_active":"2024-04-15T15:19:26.422315+0000","last_became_peered":"2024-04-15T15:19:26.422315+0000","last_unstale":"2024-04-15T15:19:27.420351+0000","last_undegraded":"2024-04-15T15:19:27.420351+0000","last_fullsized":"2024-04-15T15:19:27.420351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:56:54.270550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.554124+0000","last_change":"2024-04-15T15:19:25.414012+0000","last_active":"2024-04-15T15:19:27.554124+0000","last_peered":"2024-04-15T15:19:27.554124+0000","last_clean":"2024-04-15T15:19:27.554124+0000","last_became_active":"2024-04-15T15:19:25.413714+0000","last_became_peered":"2024-04-15T15:19:25.413714+0000","last_unstale":"2024-04-15T15:19:27.554124+0000","last_undegraded":"2024-04-15T15:19:27.554124+0000","last_fullsized":"2024-04-15T15:19:27.554124+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:39:21.336486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.724458+0000","last_change":"2024-04-15T15:19:26.442998+0000","last_active":"2024-04-15T15:19:27.724458+0000","last_peered":"2024-04-15T15:19:27.724458+0000","last_clean":"2024-04-15T15:19:27.724458+0000","last_became_active":"2024-04-15T15:19:26.442205+0000","last_became_peered":"2024-04-15T15:19:26.442205+0000","last_unstale":"2024-04-15T15:19:27.724458+0000","last_undegraded":"2024-04-15T15:19:27.724458+0000","last_fullsized":"2024-04-15T15:19:27.724458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:15:06.003211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.421043+0000","last_change":"2024-04-15T15:19:25.423057+0000","last_active":"2024-04-15T15:19:27.421043+0000","last_peered":"2024-04-15T15:19:27.421043+0000","last_clean":"2024-04-15T15:19:27.421043+0000","last_became_active":"2024-04-15T15:19:25.422760+0000","last_became_peered":"2024-04-15T15:19:25.422760+0000","last_unstale":"2024-04-15T15:19:27.421043+0000","last_undegraded":"2024-04-15T15:19:27.421043+0000","last_fullsized":"2024-04-15T15:19:27.421043+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:00:10.713962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.724934+0000","last_change":"2024-04-15T15:19:26.439301+0000","last_active":"2024-04-15T15:19:27.724934+0000","last_peered":"2024-04-15T15:19:27.724934+0000","last_clean":"2024-04-15T15:19:27.724934+0000","last_became_active":"2024-04-15T15:19:26.439016+0000","last_became_peered":"2024-04-15T15:19:26.439016+0000","last_unstale":"2024-04-15T15:19:27.724934+0000","last_undegraded":"2024-04-15T15:19:27.724934+0000","last_fullsized":"2024-04-15T15:19:27.724934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:47:07.377973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.554369+0000","last_change":"2024-04-15T15:19:25.416350+0000","last_active":"2024-04-15T15:19:27.554369+0000","last_peered":"2024-04-15T15:19:27.554369+0000","last_clean":"2024-04-15T15:19:27.554369+0000","last_became_active":"2024-04-15T15:19:25.415907+0000","last_became_peered":"2024-04-15T15:19:25.415907+0000","last_unstale":"2024-04-15T15:19:27.554369+0000","last_undegraded":"2024-04-15T15:19:27.554369+0000","last_fullsized":"2024-04-15T15:19:27.554369+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:55:33.675151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.725169+0000","last_change":"2024-04-15T15:19:26.445601+0000","last_active":"2024-04-15T15:19:27.725169+0000","last_peered":"2024-04-15T15:19:27.725169+0000","last_clean":"2024-04-15T15:19:27.725169+0000","last_became_active":"2024-04-15T15:19:26.445320+0000","last_became_peered":"2024-04-15T15:19:26.445320+0000","last_unstale":"2024-04-15T15:19:27.725169+0000","last_undegraded":"2024-04-15T15:19:27.725169+0000","last_fullsized":"2024-04-15T15:19:27.725169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:50:29.196579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.439918+0000","last_change":"2024-04-15T15:19:25.424681+0000","last_active":"2024-04-15T15:19:27.439918+0000","last_peered":"2024-04-15T15:19:27.439918+0000","last_clean":"2024-04-15T15:19:27.439918+0000","last_became_active":"2024-04-15T15:19:25.424420+0000","last_became_peered":"2024-04-15T15:19:25.424420+0000","last_unstale":"2024-04-15T15:19:27.439918+0000","last_undegraded":"2024-04-15T15:19:27.439918+0000","last_fullsized":"2024-04-15T15:19:27.439918+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:50:03.098538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774833+0000","last_change":"2024-04-15T15:19:26.441743+0000","last_active":"2024-04-15T15:19:27.774833+0000","last_peered":"2024-04-15T15:19:27.774833+0000","last_clean":"2024-04-15T15:19:27.774833+0000","last_became_active":"2024-04-15T15:19:26.441156+0000","last_became_peered":"2024-04-15T15:19:26.441156+0000","last_unstale":"2024-04-15T15:19:27.774833+0000","last_undegraded":"2024-04-15T15:19:27.774833+0000","last_fullsized":"2024-04-15T15:19:27.774833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:39:29.689447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.554294+0000","last_change":"2024-04-15T15:19:25.408454+0000","last_active":"2024-04-15T15:19:27.554294+0000","last_peered":"2024-04-15T15:19:27.554294+0000","last_clean":"2024-04-15T15:19:27.554294+0000","last_became_active":"2024-04-15T15:19:25.408188+0000","last_became_peered":"2024-04-15T15:19:25.408188+0000","last_unstale":"2024-04-15T15:19:27.554294+0000","last_undegraded":"2024-04-15T15:19:27.554294+0000","last_fullsized":"2024-04-15T15:19:27.554294+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:30:23.903641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.724769+0000","last_change":"2024-04-15T15:19:26.443981+0000","last_active":"2024-04-15T15:19:27.724769+0000","last_peered":"2024-04-15T15:19:27.724769+0000","last_clean":"2024-04-15T15:19:27.724769+0000","last_became_active":"2024-04-15T15:19:26.443177+0000","last_became_peered":"2024-04-15T15:19:26.443177+0000","last_unstale":"2024-04-15T15:19:27.724769+0000","last_undegraded":"2024-04-15T15:19:27.724769+0000","last_fullsized":"2024-04-15T15:19:27.724769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:32:49.335713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.724389+0000","last_change":"2024-04-15T15:19:14.878269+0000","last_active":"2024-04-15T15:19:27.724389+0000","last_peered":"2024-04-15T15:19:27.724389+0000","last_clean":"2024-04-15T15:19:27.724389+0000","last_became_active":"2024-04-15T15:19:14.877367+0000","last_became_peered":"2024-04-15T15:19:14.877367+0000","last_unstale":"2024-04-15T15:19:27.724389+0000","last_undegraded":"2024-04-15T15:19:27.724389+0000","last_fullsized":"2024-04-15T15:19:27.724389+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:13.827511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:13.827511+0000","last_clean_scrub_stamp":"2024-04-15T15:19:13.827511+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:32:55.902561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774286+0000","last_change":"2024-04-15T15:19:25.415380+0000","last_active":"2024-04-15T15:19:27.774286+0000","last_peered":"2024-04-15T15:19:27.774286+0000","last_clean":"2024-04-15T15:19:27.774286+0000","last_became_active":"2024-04-15T15:19:25.415098+0000","last_became_peered":"2024-04-15T15:19:25.415098+0000","last_unstale":"2024-04-15T15:19:27.774286+0000","last_undegraded":"2024-04-15T15:19:27.774286+0000","last_fullsized":"2024-04-15T15:19:27.774286+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:36:11.468459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.417927+0000","last_change":"2024-04-15T15:19:26.445122+0000","last_active":"2024-04-15T15:19:27.417927+0000","last_peered":"2024-04-15T15:19:27.417927+0000","last_clean":"2024-04-15T15:19:27.417927+0000","last_became_active":"2024-04-15T15:19:26.444748+0000","last_became_peered":"2024-04-15T15:19:26.444748+0000","last_unstale":"2024-04-15T15:19:27.417927+0000","last_undegraded":"2024-04-15T15:19:27.417927+0000","last_fullsized":"2024-04-15T15:19:27.417927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:55:46.969871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.724312+0000","last_change":"2024-04-15T15:19:25.422817+0000","last_active":"2024-04-15T15:19:27.724312+0000","last_peered":"2024-04-15T15:19:27.724312+0000","last_clean":"2024-04-15T15:19:27.724312+0000","last_became_active":"2024-04-15T15:19:25.422382+0000","last_became_peered":"2024-04-15T15:19:25.422382+0000","last_unstale":"2024-04-15T15:19:27.724312+0000","last_undegraded":"2024-04-15T15:19:27.724312+0000","last_fullsized":"2024-04-15T15:19:27.724312+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:21:02.027132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.773822+0000","last_change":"2024-04-15T15:19:26.431108+0000","last_active":"2024-04-15T15:19:27.773822+0000","last_peered":"2024-04-15T15:19:27.773822+0000","last_clean":"2024-04-15T15:19:27.773822+0000","last_became_active":"2024-04-15T15:19:26.430820+0000","last_became_peered":"2024-04-15T15:19:26.430820+0000","last_unstale":"2024-04-15T15:19:27.773822+0000","last_undegraded":"2024-04-15T15:19:27.773822+0000","last_fullsized":"2024-04-15T15:19:27.773822+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:23:17.976379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.554269+0000","last_change":"2024-04-15T15:19:25.416081+0000","last_active":"2024-04-15T15:19:27.554269+0000","last_peered":"2024-04-15T15:19:27.554269+0000","last_clean":"2024-04-15T15:19:27.554269+0000","last_became_active":"2024-04-15T15:19:25.415568+0000","last_became_peered":"2024-04-15T15:19:25.415568+0000","last_unstale":"2024-04-15T15:19:27.554269+0000","last_undegraded":"2024-04-15T15:19:27.554269+0000","last_fullsized":"2024-04-15T15:19:27.554269+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:36:11.468459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.439731+0000","last_change":"2024-04-15T15:19:26.429663+0000","last_active":"2024-04-15T15:19:27.439731+0000","last_peered":"2024-04-15T15:19:27.439731+0000","last_clean":"2024-04-15T15:19:27.439731+0000","last_became_active":"2024-04-15T15:19:26.429416+0000","last_became_peered":"2024-04-15T15:19:26.429416+0000","last_unstale":"2024-04-15T15:19:27.439731+0000","last_undegraded":"2024-04-15T15:19:27.439731+0000","last_fullsized":"2024-04-15T15:19:27.439731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:56:54.270550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774841+0000","last_change":"2024-04-15T15:19:25.414043+0000","last_active":"2024-04-15T15:19:27.774841+0000","last_peered":"2024-04-15T15:19:27.774841+0000","last_clean":"2024-04-15T15:19:27.774841+0000","last_became_active":"2024-04-15T15:19:25.413732+0000","last_became_peered":"2024-04-15T15:19:25.413732+0000","last_unstale":"2024-04-15T15:19:27.774841+0000","last_undegraded":"2024-04-15T15:19:27.774841+0000","last_fullsized":"2024-04-15T15:19:27.774841+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:21:02.027132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.420842+0000","last_change":"2024-04-15T15:19:26.432152+0000","last_active":"2024-04-15T15:19:27.420842+0000","last_peered":"2024-04-15T15:19:27.420842+0000","last_clean":"2024-04-15T15:19:27.420842+0000","last_became_active":"2024-04-15T15:19:26.431798+0000","last_became_peered":"2024-04-15T15:19:26.431798+0000","last_unstale":"2024-04-15T15:19:27.420842+0000","last_undegraded":"2024-04-15T15:19:27.420842+0000","last_fullsized":"2024-04-15T15:19:27.420842+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:21:03.031074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.439877+0000","last_change":"2024-04-15T15:19:25.420180+0000","last_active":"2024-04-15T15:19:27.439877+0000","last_peered":"2024-04-15T15:19:27.439877+0000","last_clean":"2024-04-15T15:19:27.439877+0000","last_became_active":"2024-04-15T15:19:25.419908+0000","last_became_peered":"2024-04-15T15:19:25.419908+0000","last_unstale":"2024-04-15T15:19:27.439877+0000","last_undegraded":"2024-04-15T15:19:27.439877+0000","last_fullsized":"2024-04-15T15:19:27.439877+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:36:11.468459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774793+0000","last_change":"2024-04-15T15:19:26.440109+0000","last_active":"2024-04-15T15:19:27.774793+0000","last_peered":"2024-04-15T15:19:27.774793+0000","last_clean":"2024-04-15T15:19:27.774793+0000","last_became_active":"2024-04-15T15:19:26.439411+0000","last_became_peered":"2024-04-15T15:19:26.439411+0000","last_unstale":"2024-04-15T15:19:27.774793+0000","last_undegraded":"2024-04-15T15:19:27.774793+0000","last_fullsized":"2024-04-15T15:19:27.774793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:32:49.335713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.725007+0000","last_change":"2024-04-15T15:19:25.422506+0000","last_active":"2024-04-15T15:19:27.725007+0000","last_peered":"2024-04-15T15:19:27.725007+0000","last_clean":"2024-04-15T15:19:27.725007+0000","last_became_active":"2024-04-15T15:19:25.422124+0000","last_became_peered":"2024-04-15T15:19:25.422124+0000","last_unstale":"2024-04-15T15:19:27.725007+0000","last_undegraded":"2024-04-15T15:19:27.725007+0000","last_fullsized":"2024-04-15T15:19:27.725007+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:52:02.003490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.439836+0000","last_change":"2024-04-15T15:19:26.429959+0000","last_active":"2024-04-15T15:19:27.439836+0000","last_peered":"2024-04-15T15:19:27.439836+0000","last_clean":"2024-04-15T15:19:27.439836+0000","last_became_active":"2024-04-15T15:19:26.429641+0000","last_became_peered":"2024-04-15T15:19:26.429641+0000","last_unstale":"2024-04-15T15:19:27.439836+0000","last_undegraded":"2024-04-15T15:19:27.439836+0000","last_fullsized":"2024-04-15T15:19:27.439836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:55:46.969871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.441601+0000","last_change":"2024-04-15T15:19:25.413501+0000","last_active":"2024-04-15T15:19:27.441601+0000","last_peered":"2024-04-15T15:19:27.441601+0000","last_clean":"2024-04-15T15:19:27.441601+0000","last_became_active":"2024-04-15T15:19:25.413246+0000","last_became_peered":"2024-04-15T15:19:25.413246+0000","last_unstale":"2024-04-15T15:19:27.441601+0000","last_undegraded":"2024-04-15T15:19:27.441601+0000","last_fullsized":"2024-04-15T15:19:27.441601+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:31:11.313499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.417952+0000","last_change":"2024-04-15T15:19:26.431760+0000","last_active":"2024-04-15T15:19:27.417952+0000","last_peered":"2024-04-15T15:19:27.417952+0000","last_clean":"2024-04-15T15:19:27.417952+0000","last_became_active":"2024-04-15T15:19:26.431488+0000","last_became_peered":"2024-04-15T15:19:26.431488+0000","last_unstale":"2024-04-15T15:19:27.417952+0000","last_undegraded":"2024-04-15T15:19:27.417952+0000","last_fullsized":"2024-04-15T15:19:27.417952+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:06:20.973059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.420725+0000","last_change":"2024-04-15T15:19:25.417327+0000","last_active":"2024-04-15T15:19:27.420725+0000","last_peered":"2024-04-15T15:19:27.420725+0000","last_clean":"2024-04-15T15:19:27.420725+0000","last_became_active":"2024-04-15T15:19:25.416474+0000","last_became_peered":"2024-04-15T15:19:25.416474+0000","last_unstale":"2024-04-15T15:19:27.420725+0000","last_undegraded":"2024-04-15T15:19:27.420725+0000","last_fullsized":"2024-04-15T15:19:27.420725+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:08:25.359023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774050+0000","last_change":"2024-04-15T15:19:26.427252+0000","last_active":"2024-04-15T15:19:27.774050+0000","last_peered":"2024-04-15T15:19:27.774050+0000","last_clean":"2024-04-15T15:19:27.774050+0000","last_became_active":"2024-04-15T15:19:26.427004+0000","last_became_peered":"2024-04-15T15:19:26.427004+0000","last_unstale":"2024-04-15T15:19:27.774050+0000","last_undegraded":"2024-04-15T15:19:27.774050+0000","last_fullsized":"2024-04-15T15:19:27.774050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:07:02.244933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.724316+0000","last_change":"2024-04-15T15:19:25.422257+0000","last_active":"2024-04-15T15:19:27.724316+0000","last_peered":"2024-04-15T15:19:27.724316+0000","last_clean":"2024-04-15T15:19:27.724316+0000","last_became_active":"2024-04-15T15:19:25.421856+0000","last_became_peered":"2024-04-15T15:19:25.421856+0000","last_unstale":"2024-04-15T15:19:27.724316+0000","last_undegraded":"2024-04-15T15:19:27.724316+0000","last_fullsized":"2024-04-15T15:19:27.724316+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:08:25.359023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.420621+0000","last_change":"2024-04-15T15:19:26.430222+0000","last_active":"2024-04-15T15:19:27.420621+0000","last_peered":"2024-04-15T15:19:27.420621+0000","last_clean":"2024-04-15T15:19:27.420621+0000","last_became_active":"2024-04-15T15:19:26.429923+0000","last_became_peered":"2024-04-15T15:19:26.429923+0000","last_unstale":"2024-04-15T15:19:27.420621+0000","last_undegraded":"2024-04-15T15:19:27.420621+0000","last_fullsized":"2024-04-15T15:19:27.420621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:08:26.362966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.439788+0000","last_change":"2024-04-15T15:19:25.418348+0000","last_active":"2024-04-15T15:19:27.439788+0000","last_peered":"2024-04-15T15:19:27.439788+0000","last_clean":"2024-04-15T15:19:27.439788+0000","last_became_active":"2024-04-15T15:19:25.418080+0000","last_became_peered":"2024-04-15T15:19:25.418080+0000","last_unstale":"2024-04-15T15:19:27.439788+0000","last_undegraded":"2024-04-15T15:19:27.439788+0000","last_fullsized":"2024-04-15T15:19:27.439788+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:55:33.675151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.417808+0000","last_change":"2024-04-15T15:19:26.428800+0000","last_active":"2024-04-15T15:19:27.417808+0000","last_peered":"2024-04-15T15:19:27.417808+0000","last_clean":"2024-04-15T15:19:27.417808+0000","last_became_active":"2024-04-15T15:19:26.428528+0000","last_became_peered":"2024-04-15T15:19:26.428528+0000","last_unstale":"2024-04-15T15:19:27.417808+0000","last_undegraded":"2024-04-15T15:19:27.417808+0000","last_fullsized":"2024-04-15T15:19:27.417808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:56:54.270550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.439924+0000","last_change":"2024-04-15T15:19:25.416097+0000","last_active":"2024-04-15T15:19:27.439924+0000","last_peered":"2024-04-15T15:19:27.439924+0000","last_clean":"2024-04-15T15:19:27.439924+0000","last_became_active":"2024-04-15T15:19:25.415550+0000","last_became_peered":"2024-04-15T15:19:25.415550+0000","last_unstale":"2024-04-15T15:19:27.439924+0000","last_undegraded":"2024-04-15T15:19:27.439924+0000","last_fullsized":"2024-04-15T15:19:27.439924+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:08:25.359023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.418104+0000","last_change":"2024-04-15T15:19:26.429710+0000","last_active":"2024-04-15T15:19:27.418104+0000","last_peered":"2024-04-15T15:19:27.418104+0000","last_clean":"2024-04-15T15:19:27.418104+0000","last_became_active":"2024-04-15T15:19:26.429429+0000","last_became_peered":"2024-04-15T15:19:26.429429+0000","last_unstale":"2024-04-15T15:19:27.418104+0000","last_undegraded":"2024-04-15T15:19:27.418104+0000","last_fullsized":"2024-04-15T15:19:27.418104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:52:38.985490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.724834+0000","last_change":"2024-04-15T15:19:25.425223+0000","last_active":"2024-04-15T15:19:27.724834+0000","last_peered":"2024-04-15T15:19:27.724834+0000","last_clean":"2024-04-15T15:19:27.724834+0000","last_became_active":"2024-04-15T15:19:25.424548+0000","last_became_peered":"2024-04-15T15:19:25.424548+0000","last_unstale":"2024-04-15T15:19:27.724834+0000","last_undegraded":"2024-04-15T15:19:27.724834+0000","last_fullsized":"2024-04-15T15:19:27.724834+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:50:03.098538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.420985+0000","last_change":"2024-04-15T15:19:26.430650+0000","last_active":"2024-04-15T15:19:27.420985+0000","last_peered":"2024-04-15T15:19:27.420985+0000","last_clean":"2024-04-15T15:19:27.420985+0000","last_became_active":"2024-04-15T15:19:26.430359+0000","last_became_peered":"2024-04-15T15:19:26.430359+0000","last_unstale":"2024-04-15T15:19:27.420985+0000","last_undegraded":"2024-04-15T15:19:27.420985+0000","last_fullsized":"2024-04-15T15:19:27.420985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:05:03.041116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774721+0000","last_change":"2024-04-15T15:19:26.445875+0000","last_active":"2024-04-15T15:19:27.774721+0000","last_peered":"2024-04-15T15:19:27.774721+0000","last_clean":"2024-04-15T15:19:27.774721+0000","last_became_active":"2024-04-15T15:19:26.444235+0000","last_became_peered":"2024-04-15T15:19:26.444235+0000","last_unstale":"2024-04-15T15:19:27.774721+0000","last_undegraded":"2024-04-15T15:19:27.774721+0000","last_fullsized":"2024-04-15T15:19:27.774721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:38:37.460808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.724639+0000","last_change":"2024-04-15T15:19:25.425067+0000","last_active":"2024-04-15T15:19:27.724639+0000","last_peered":"2024-04-15T15:19:27.724639+0000","last_clean":"2024-04-15T15:19:27.724639+0000","last_became_active":"2024-04-15T15:19:25.424761+0000","last_became_peered":"2024-04-15T15:19:25.424761+0000","last_unstale":"2024-04-15T15:19:27.724639+0000","last_undegraded":"2024-04-15T15:19:27.724639+0000","last_fullsized":"2024-04-15T15:19:27.724639+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:31:11.313499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774515+0000","last_change":"2024-04-15T15:19:26.444448+0000","last_active":"2024-04-15T15:19:27.774515+0000","last_peered":"2024-04-15T15:19:27.774515+0000","last_clean":"2024-04-15T15:19:27.774515+0000","last_became_active":"2024-04-15T15:19:26.444185+0000","last_became_peered":"2024-04-15T15:19:26.444185+0000","last_unstale":"2024-04-15T15:19:27.774515+0000","last_undegraded":"2024-04-15T15:19:27.774515+0000","last_fullsized":"2024-04-15T15:19:27.774515+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:52:38.985490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.724888+0000","last_change":"2024-04-15T15:19:25.427837+0000","last_active":"2024-04-15T15:19:27.724888+0000","last_peered":"2024-04-15T15:19:27.724888+0000","last_clean":"2024-04-15T15:19:27.724888+0000","last_became_active":"2024-04-15T15:19:25.427483+0000","last_became_peered":"2024-04-15T15:19:25.427483+0000","last_unstale":"2024-04-15T15:19:27.724888+0000","last_undegraded":"2024-04-15T15:19:27.724888+0000","last_fullsized":"2024-04-15T15:19:27.724888+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:21:55.463937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.773977+0000","last_change":"2024-04-15T15:19:26.443862+0000","last_active":"2024-04-15T15:19:27.773977+0000","last_peered":"2024-04-15T15:19:27.773977+0000","last_clean":"2024-04-15T15:19:27.773977+0000","last_became_active":"2024-04-15T15:19:26.442956+0000","last_became_peered":"2024-04-15T15:19:26.442956+0000","last_unstale":"2024-04-15T15:19:27.773977+0000","last_undegraded":"2024-04-15T15:19:27.773977+0000","last_fullsized":"2024-04-15T15:19:27.773977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:57:45.149303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.418121+0000","last_change":"2024-04-15T15:19:25.419914+0000","last_active":"2024-04-15T15:19:27.418121+0000","last_peered":"2024-04-15T15:19:27.418121+0000","last_clean":"2024-04-15T15:19:27.418121+0000","last_became_active":"2024-04-15T15:19:25.419638+0000","last_became_peered":"2024-04-15T15:19:25.419638+0000","last_unstale":"2024-04-15T15:19:27.418121+0000","last_undegraded":"2024-04-15T15:19:27.418121+0000","last_fullsized":"2024-04-15T15:19:27.418121+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:30:23.903641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.418082+0000","last_change":"2024-04-15T15:19:26.442167+0000","last_active":"2024-04-15T15:19:27.418082+0000","last_peered":"2024-04-15T15:19:27.418082+0000","last_clean":"2024-04-15T15:19:27.418082+0000","last_became_active":"2024-04-15T15:19:26.441866+0000","last_became_peered":"2024-04-15T15:19:26.441866+0000","last_unstale":"2024-04-15T15:19:27.418082+0000","last_undegraded":"2024-04-15T15:19:27.418082+0000","last_fullsized":"2024-04-15T15:19:27.418082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:35:32.148665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774091+0000","last_change":"2024-04-15T15:19:25.419672+0000","last_active":"2024-04-15T15:19:27.774091+0000","last_peered":"2024-04-15T15:19:27.774091+0000","last_clean":"2024-04-15T15:19:27.774091+0000","last_became_active":"2024-04-15T15:19:25.419412+0000","last_became_peered":"2024-04-15T15:19:25.419412+0000","last_unstale":"2024-04-15T15:19:27.774091+0000","last_undegraded":"2024-04-15T15:19:27.774091+0000","last_fullsized":"2024-04-15T15:19:27.774091+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:35:36.249861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-15T15:19:29.762 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-15T15:19:29.763 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-15T15:19:29.763 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-15T15:19:29.763 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-15T15:19:29.763 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-15T15:19:29.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.899+0000 7f41df288640 1 Processor -- start 2024-04-15T15:19:29.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.899+0000 7f41df288640 1 -- start start 2024-04-15T15:19:29.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.900+0000 7f41df288640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f41d80b3c80 0x7f41d8169ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:29.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.900+0000 7f41df288640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41d80c8270 con 0x7f41d806c0d0 2024-04-15T15:19:29.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.900+0000 7f41df288640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41d80d0920 con 0x7f41d80d0540 2024-04-15T15:19:29.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.900+0000 7f41df288640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f41d80c7cb0 con 0x7f41d80b3c80 2024-04-15T15:19:29.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.900+0000 7f41dc823640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f41d806c0d0 0x7f41d8166570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:40342/0 (socket says 172.21.15.105:40342) 2024-04-15T15:19:29.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.900+0000 7f41dc823640 1 -- 172.21.15.105:0/1055916688 learned_addr learned my addr 172.21.15.105:0/1055916688 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:19:29.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.900+0000 7f41dd825640 1 --2- 172.21.15.105:0/1055916688 >> v2:172.21.15.154:3300/0 conn(0x7f41d80b3c80 0x7f41d8169ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:29.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.900+0000 7f41cffff640 1 -- 172.21.15.105:0/1055916688 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2182921799 0 0) 0x7f41d80c8270 con 0x7f41d806c0d0 2024-04-15T15:19:29.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.900+0000 7f41cffff640 1 -- 172.21.15.105:0/1055916688 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f41c4003520 con 0x7f41d806c0d0 2024-04-15T15:19:29.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.901+0000 7f41cffff640 1 -- 172.21.15.105:0/1055916688 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1735343562 0 0) 0x7f41c4003520 con 0x7f41d806c0d0 2024-04-15T15:19:29.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.901+0000 7f41cffff640 1 -- 172.21.15.105:0/1055916688 >> v2:172.21.15.154:3300/0 conn(0x7f41d80b3c80 msgr2=0x7f41d8169ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:29.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.901+0000 7f41cffff640 1 --2- 172.21.15.105:0/1055916688 >> v2:172.21.15.154:3300/0 conn(0x7f41d80b3c80 0x7f41d8169ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:29.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.901+0000 7f41cffff640 1 -- 172.21.15.105:0/1055916688 >> v1:172.21.15.154:6789/0 conn(0x7f41d80d0540 legacy=0x7f41d806b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:29.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.901+0000 7f41cffff640 1 -- 172.21.15.105:0/1055916688 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41d80c7f90 con 0x7f41d806c0d0 2024-04-15T15:19:29.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.901+0000 7f41cffff640 1 -- 172.21.15.105:0/1055916688 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7f41c8001670 con 0x7f41d806c0d0 2024-04-15T15:19:29.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.901+0000 7f41cffff640 1 -- 172.21.15.105:0/1055916688 >> v1:172.21.15.105:6789/0 conn(0x7f41d806c0d0 legacy=0x7f41d8166570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:29.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.901+0000 7f41cffff640 1 --2- 172.21.15.105:0/1055916688 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41c40039e0 0x7f41c4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:29.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.901+0000 7f41cffff640 1 --2- 172.21.15.105:0/1055916688 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41c4004800 0x7f41c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:29.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.901+0000 7f41cffff640 1 --2- 172.21.15.105:0/1055916688 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f41c40051c0 0x7f41c400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:29.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.901+0000 7f41cffff640 1 -- 172.21.15.105:0/1055916688 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f41d80c7cb0 con 0x7f41c4004800 2024-04-15T15:19:29.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.901+0000 7f41cffff640 1 -- 172.21.15.105:0/1055916688 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f41c400c090 con 0x7f41c40039e0 2024-04-15T15:19:29.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.901+0000 7f41cffff640 1 -- 172.21.15.105:0/1055916688 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f41c400c290 con 0x7f41c40051c0 2024-04-15T15:19:29.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.901+0000 7f41dc823640 1 --2- 172.21.15.105:0/1055916688 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41c4004800 0x7f41c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:29.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.901+0000 7f41dd024640 1 --2- 172.21.15.105:0/1055916688 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41c40039e0 0x7f41c4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:29.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.901+0000 7f41dd825640 1 --2- 172.21.15.105:0/1055916688 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f41c40051c0 0x7f41c400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:29.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.901+0000 7f41dc823640 1 -- 172.21.15.105:0/1055916688 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f41c40051c0 msgr2=0x7f41c400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:29.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.901+0000 7f41dc823640 1 --2- 172.21.15.105:0/1055916688 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f41c40051c0 0x7f41c400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:29.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.901+0000 7f41dc823640 1 -- 172.21.15.105:0/1055916688 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41c40039e0 msgr2=0x7f41c4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:29.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.901+0000 7f41dc823640 1 --2- 172.21.15.105:0/1055916688 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41c40039e0 0x7f41c4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:29.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.901+0000 7f41dc823640 1 -- 172.21.15.105:0/1055916688 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41c4003520 con 0x7f41c4004800 2024-04-15T15:19:29.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.901+0000 7f41dd825640 1 -- 172.21.15.105:0/1055916688 reap_dead start 2024-04-15T15:19:29.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.902+0000 7f41dc823640 1 --2- 172.21.15.105:0/1055916688 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41c4004800 0x7f41c4004c80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f41c800c210 tx=0x7f41c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56652e4dbff2abc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:29.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.902+0000 7f41cffff640 1 -- 172.21.15.105:0/1055916688 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c8007d30 con 0x7f41c4004800 2024-04-15T15:19:29.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.902+0000 7f41cffff640 1 -- 172.21.15.105:0/1055916688 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41c8030070 con 0x7f41c4004800 2024-04-15T15:19:29.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.902+0000 7f41cffff640 1 -- 172.21.15.105:0/1055916688 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c8005d70 con 0x7f41c4004800 2024-04-15T15:19:29.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.902+0000 7f41df288640 1 -- 172.21.15.105:0/1055916688 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41c4004800 msgr2=0x7f41c4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:29.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.902+0000 7f41df288640 1 --2- 172.21.15.105:0/1055916688 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41c4004800 0x7f41c4004c80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f41c800c210 tx=0x7f41c4005830 comp rx=0 tx=0).stop 2024-04-15T15:19:29.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.903+0000 7f41df288640 1 -- 172.21.15.105:0/1055916688 shutdown_connections 2024-04-15T15:19:29.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.903+0000 7f41df288640 1 --2- 172.21.15.105:0/1055916688 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41c4004800 0x7f41c4004c80 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:29.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.903+0000 7f41df288640 1 -- 172.21.15.105:0/1055916688 >> 172.21.15.105:0/1055916688 conn(0x7f41d809aa40 msgr2=0x7f41d80b30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:29.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.903+0000 7f41df288640 1 -- 172.21.15.105:0/1055916688 shutdown_connections 2024-04-15T15:19:29.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.903+0000 7f41df288640 1 -- 172.21.15.105:0/1055916688 wait complete. 2024-04-15T15:19:29.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.904+0000 7f41df288640 1 Processor -- start 2024-04-15T15:19:29.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.904+0000 7f41df288640 1 -- start start 2024-04-15T15:19:29.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.905+0000 7f41df288640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f41c4004800 0x7f41d809cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:29.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.905+0000 7f41df288640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41d8152340 0x7f41d80b66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:29.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.905+0000 7f41df288640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41d80b6c20 0x7f41d8160060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:29.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.905+0000 7f41df288640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f41d8070e40 con 0x7f41d80b6c20 2024-04-15T15:19:29.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.905+0000 7f41df288640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f41d806f180 con 0x7f41d8152340 2024-04-15T15:19:29.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.905+0000 7f41df288640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f41d8070fc0 con 0x7f41c4004800 2024-04-15T15:19:29.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.905+0000 7f41dd825640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41d80b6c20 0x7f41d8160060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:29.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.905+0000 7f41dd825640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41d80b6c20 0x7f41d8160060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57258/0 (socket says 172.21.15.105:57258) 2024-04-15T15:19:29.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.905+0000 7f41dd825640 1 -- 172.21.15.105:0/3680186632 learned_addr learned my addr 172.21.15.105:0/3680186632 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:29.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.905+0000 7f41dd024640 1 --2- 172.21.15.105:0/3680186632 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f41c4004800 0x7f41d809cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:29.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.905+0000 7f41dc823640 1 --2- 172.21.15.105:0/3680186632 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41d8152340 0x7f41d80b66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:29.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.905+0000 7f41dd825640 1 -- 172.21.15.105:0/3680186632 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f41c4004800 msgr2=0x7f41d809cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:29.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.905+0000 7f41dd825640 1 --2- 172.21.15.105:0/3680186632 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f41c4004800 0x7f41d809cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:29.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.905+0000 7f41dd825640 1 -- 172.21.15.105:0/3680186632 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41d8152340 msgr2=0x7f41d80b66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:29.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.905+0000 7f41dd825640 1 --2- 172.21.15.105:0/3680186632 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41d8152340 0x7f41d80b66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:29.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.905+0000 7f41dd825640 1 -- 172.21.15.105:0/3680186632 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41d80b4360 con 0x7f41d80b6c20 2024-04-15T15:19:29.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.905+0000 7f41dd024640 1 --2- 172.21.15.105:0/3680186632 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f41c4004800 0x7f41d809cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:19:29.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.905+0000 7f41dd825640 1 --2- 172.21.15.105:0/3680186632 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41d80b6c20 0x7f41d8160060 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f41d8070030 tx=0x7f41d00115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:29.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.905+0000 7f41cdffb640 1 -- 172.21.15.105:0/3680186632 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d0011db0 con 0x7f41d80b6c20 2024-04-15T15:19:29.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.905+0000 7f41cdffb640 1 -- 172.21.15.105:0/3680186632 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41d001b070 con 0x7f41d80b6c20 2024-04-15T15:19:29.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.905+0000 7f41cdffb640 1 -- 172.21.15.105:0/3680186632 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d0003990 con 0x7f41d80b6c20 2024-04-15T15:19:29.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.905+0000 7f41df288640 1 -- 172.21.15.105:0/3680186632 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41d80b4640 con 0x7f41d80b6c20 2024-04-15T15:19:29.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.906+0000 7f41df288640 1 -- 172.21.15.105:0/3680186632 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41d80b4c00 con 0x7f41d80b6c20 2024-04-15T15:19:29.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.907+0000 7f41df288640 1 -- 172.21.15.105:0/3680186632 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41d80b4e00 con 0x7f41d80b6c20 2024-04-15T15:19:29.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.907+0000 7f41cdffb640 1 -- 172.21.15.105:0/3680186632 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f41d0003b30 con 0x7f41d80b6c20 2024-04-15T15:19:29.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.907+0000 7f41cdffb640 1 --2- 172.21.15.105:0/3680186632 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f41bc06c8f0 0x7f41bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:29.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.908+0000 7f41cdffb640 1 -- 172.21.15.105:0/3680186632 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f41d008c460 con 0x7f41d80b6c20 2024-04-15T15:19:29.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.908+0000 7f41dd024640 1 --2- 172.21.15.105:0/3680186632 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f41bc06c8f0 0x7f41bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:29.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.908+0000 7f41dd024640 1 --2- 172.21.15.105:0/3680186632 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f41bc06c8f0 0x7f41bc06edb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f41d4000c00 tx=0x7f41d4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:29.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:29.912+0000 7f41cdffb640 1 -- 172.21.15.105:0/3680186632 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f41d80b4e00 con 0x7f41d80b6c20 2024-04-15T15:19:30.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.137+0000 7f41df288640 1 -- 172.21.15.105:0/3680186632 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f41d8068a40 con 0x7f41d80b6c20 2024-04-15T15:19:30.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.139+0000 7f41cdffb640 1 -- 172.21.15.105:0/3680186632 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+174144 (secure 0 0 0) 0x7f41d0003e20 con 0x7f41d80b6c20 2024-04-15T15:19:30.143 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:19:30.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.142+0000 7f41df288640 1 -- 172.21.15.105:0/3680186632 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f41bc06c8f0 msgr2=0x7f41bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:30.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.142+0000 7f41df288640 1 --2- 172.21.15.105:0/3680186632 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f41bc06c8f0 0x7f41bc06edb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f41d4000c00 tx=0x7f41d4009040 comp rx=0 tx=0).stop 2024-04-15T15:19:30.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.142+0000 7f41df288640 1 -- 172.21.15.105:0/3680186632 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41d80b6c20 msgr2=0x7f41d8160060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:30.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.142+0000 7f41df288640 1 --2- 172.21.15.105:0/3680186632 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41d80b6c20 0x7f41d8160060 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f41d8070030 tx=0x7f41d00115a0 comp rx=0 tx=0).stop 2024-04-15T15:19:30.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.142+0000 7f41df288640 1 -- 172.21.15.105:0/3680186632 shutdown_connections 2024-04-15T15:19:30.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.142+0000 7f41df288640 1 --2- 172.21.15.105:0/3680186632 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f41bc06c8f0 0x7f41bc06edb0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:30.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.142+0000 7f41df288640 1 --2- 172.21.15.105:0/3680186632 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41d80b6c20 0x7f41d8160060 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:30.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.142+0000 7f41df288640 1 --2- 172.21.15.105:0/3680186632 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41d8152340 0x7f41d80b66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:30.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.142+0000 7f41df288640 1 --2- 172.21.15.105:0/3680186632 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f41c4004800 0x7f41d809cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:30.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.142+0000 7f41df288640 1 -- 172.21.15.105:0/3680186632 >> 172.21.15.105:0/3680186632 conn(0x7f41d809aa40 msgr2=0x7f41d80ae380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:30.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.142+0000 7f41df288640 1 -- 172.21.15.105:0/3680186632 shutdown_connections 2024-04-15T15:19:30.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.142+0000 7f41df288640 1 -- 172.21.15.105:0/3680186632 wait complete. 2024-04-15T15:19:30.155 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":5,"flags":0,"active_gid":4107,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":3506549322},{"type":"v1","addr":"172.21.15.105:6833","nonce":3506549322}]},"active_addr":"172.21.15.105:6833/3506549322","active_change":"2024-04-15T15:19:05.726306+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4100,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.105:0","nonce":1116253438}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.105:0","nonce":1849176489}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.105:0","nonce":4294803803}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.105:0","nonce":3397744087}]}]} 2024-04-15T15:19:30.158 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-15T15:19:30.158 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-15T15:19:30.158 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:19:30.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.259+0000 7f56d349a640 1 Processor -- start 2024-04-15T15:19:30.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.260+0000 7f56d349a640 1 -- start start 2024-04-15T15:19:30.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.260+0000 7f56d349a640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f56cc15e9d0 0x7f56cc0d3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:30.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.260+0000 7f56d349a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56cc15edb0 con 0x7f56cc163130 2024-04-15T15:19:30.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.260+0000 7f56d349a640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56cc0cff30 con 0x7f56cc0d0ef0 2024-04-15T15:19:30.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.260+0000 7f56d349a640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f56cc0cef90 con 0x7f56cc15e9d0 2024-04-15T15:19:30.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.261+0000 7f56d0a35640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f56cc163130 0x7f56cc0cf820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:40346/0 (socket says 172.21.15.105:40346) 2024-04-15T15:19:30.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.261+0000 7f56d0a35640 1 -- 172.21.15.105:0/4198869425 learned_addr learned my addr 172.21.15.105:0/4198869425 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:19:30.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.261+0000 7f56d1236640 1 --2- 172.21.15.105:0/4198869425 >> v2:172.21.15.154:3300/0 conn(0x7f56cc15e9d0 0x7f56cc0d3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:30.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.261+0000 7f56c3fff640 1 -- 172.21.15.105:0/4198869425 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3340759788 0 0) 0x7f56bc001110 con 0x7f56cc163130 2024-04-15T15:19:30.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.261+0000 7f56c3fff640 1 -- 172.21.15.105:0/4198869425 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f56b8003520 con 0x7f56cc163130 2024-04-15T15:19:30.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.261+0000 7f56c3fff640 1 -- 172.21.15.105:0/4198869425 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3952403436 0 0) 0x7f56cc0cff30 con 0x7f56cc0d0ef0 2024-04-15T15:19:30.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.261+0000 7f56c3fff640 1 -- 172.21.15.105:0/4198869425 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f56bc001110 con 0x7f56cc0d0ef0 2024-04-15T15:19:30.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.261+0000 7f56c3fff640 1 -- 172.21.15.105:0/4198869425 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1891849663 0 0) 0x7f56b8003520 con 0x7f56cc163130 2024-04-15T15:19:30.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.261+0000 7f56c3fff640 1 -- 172.21.15.105:0/4198869425 >> v2:172.21.15.154:3300/0 conn(0x7f56cc15e9d0 msgr2=0x7f56cc0d3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:30.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.261+0000 7f56c3fff640 1 --2- 172.21.15.105:0/4198869425 >> v2:172.21.15.154:3300/0 conn(0x7f56cc15e9d0 0x7f56cc0d3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:30.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.261+0000 7f56c3fff640 1 -- 172.21.15.105:0/4198869425 >> v1:172.21.15.154:6789/0 conn(0x7f56cc0d0ef0 legacy=0x7f56cc0d1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:30.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.261+0000 7f56c3fff640 1 -- 172.21.15.105:0/4198869425 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56cc0ba280 con 0x7f56cc163130 2024-04-15T15:19:30.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.261+0000 7f56c3fff640 1 -- 172.21.15.105:0/4198869425 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7f56bc001840 con 0x7f56cc163130 2024-04-15T15:19:30.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.261+0000 7f56c3fff640 1 -- 172.21.15.105:0/4198869425 >> v1:172.21.15.105:6789/0 conn(0x7f56cc163130 legacy=0x7f56cc0cf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:30.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.261+0000 7f56c3fff640 1 --2- 172.21.15.105:0/4198869425 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56b8003b90 0x7f56b8003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:30.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.261+0000 7f56c3fff640 1 --2- 172.21.15.105:0/4198869425 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56b8004800 0x7f56b8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:30.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.261+0000 7f56c3fff640 1 --2- 172.21.15.105:0/4198869425 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56b8005210 0x7f56b800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:30.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.261+0000 7f56c3fff640 1 -- 172.21.15.105:0/4198869425 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f56cc0cef90 con 0x7f56b8004800 2024-04-15T15:19:30.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.261+0000 7f56c3fff640 1 -- 172.21.15.105:0/4198869425 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f56b800c140 con 0x7f56b8003b90 2024-04-15T15:19:30.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.261+0000 7f56c3fff640 1 -- 172.21.15.105:0/4198869425 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f56b800c340 con 0x7f56b8005210 2024-04-15T15:19:30.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.261+0000 7f56d0a35640 1 --2- 172.21.15.105:0/4198869425 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56b8004800 0x7f56b8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:30.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.261+0000 7f56d1236640 1 --2- 172.21.15.105:0/4198869425 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56b8003b90 0x7f56b8003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:30.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.261+0000 7f56d1a37640 1 --2- 172.21.15.105:0/4198869425 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56b8005210 0x7f56b800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:30.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.262+0000 7f56d0a35640 1 -- 172.21.15.105:0/4198869425 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56b8005210 msgr2=0x7f56b800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:30.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.262+0000 7f56d0a35640 1 --2- 172.21.15.105:0/4198869425 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56b8005210 0x7f56b800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:30.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.262+0000 7f56d0a35640 1 -- 172.21.15.105:0/4198869425 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56b8003b90 msgr2=0x7f56b8003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:30.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.262+0000 7f56d0a35640 1 --2- 172.21.15.105:0/4198869425 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56b8003b90 0x7f56b8003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:30.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.262+0000 7f56d0a35640 1 -- 172.21.15.105:0/4198869425 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56b8003520 con 0x7f56b8004800 2024-04-15T15:19:30.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.262+0000 7f56d1a37640 1 -- 172.21.15.105:0/4198869425 reap_dead start 2024-04-15T15:19:30.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.262+0000 7f56d0a35640 1 --2- 172.21.15.105:0/4198869425 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56b8004800 0x7f56b8004c80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f56bc00c2a0 tx=0x7f56b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0eb4e2b0d0862b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:30.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.262+0000 7f56c3fff640 1 -- 172.21.15.105:0/4198869425 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56bc007de0 con 0x7f56b8004800 2024-04-15T15:19:30.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.262+0000 7f56c3fff640 1 -- 172.21.15.105:0/4198869425 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56bc030070 con 0x7f56b8004800 2024-04-15T15:19:30.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.262+0000 7f56c3fff640 1 -- 172.21.15.105:0/4198869425 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56bc02cdf0 con 0x7f56b8004800 2024-04-15T15:19:30.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.263+0000 7f56d349a640 1 -- 172.21.15.105:0/4198869425 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56b8004800 msgr2=0x7f56b8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:30.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.263+0000 7f56d349a640 1 --2- 172.21.15.105:0/4198869425 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56b8004800 0x7f56b8004c80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f56bc00c2a0 tx=0x7f56b8005830 comp rx=0 tx=0).stop 2024-04-15T15:19:30.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.263+0000 7f56d349a640 1 -- 172.21.15.105:0/4198869425 shutdown_connections 2024-04-15T15:19:30.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.263+0000 7f56d349a640 1 --2- 172.21.15.105:0/4198869425 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56b8004800 0x7f56b8004c80 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:30.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.263+0000 7f56d349a640 1 -- 172.21.15.105:0/4198869425 >> 172.21.15.105:0/4198869425 conn(0x7f56cc067720 msgr2=0x7f56cc067b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:30.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.263+0000 7f56d349a640 1 -- 172.21.15.105:0/4198869425 shutdown_connections 2024-04-15T15:19:30.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.263+0000 7f56d349a640 1 -- 172.21.15.105:0/4198869425 wait complete. 2024-04-15T15:19:30.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.264+0000 7f56d349a640 1 Processor -- start 2024-04-15T15:19:30.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.264+0000 7f56d349a640 1 -- start start 2024-04-15T15:19:30.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.265+0000 7f56d349a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56b8004800 0x7f56cc1f5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:30.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.265+0000 7f56d349a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56cc160bd0 0x7f56cc0b3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:30.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.265+0000 7f56d349a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56cc161050 0x7f56cc0b37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:30.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.265+0000 7f56d349a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f56cc0d1360 con 0x7f56cc160bd0 2024-04-15T15:19:30.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.265+0000 7f56d349a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f56cc0cf250 con 0x7f56cc161050 2024-04-15T15:19:30.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.265+0000 7f56d349a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f56cc0d14e0 con 0x7f56b8004800 2024-04-15T15:19:30.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.265+0000 7f56d0a35640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56cc160bd0 0x7f56cc0b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:30.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.265+0000 7f56d0a35640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56cc160bd0 0x7f56cc0b3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57274/0 (socket says 172.21.15.105:57274) 2024-04-15T15:19:30.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.265+0000 7f56d0a35640 1 -- 172.21.15.105:0/3671289979 learned_addr learned my addr 172.21.15.105:0/3671289979 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:30.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.265+0000 7f56d1a37640 1 --2- 172.21.15.105:0/3671289979 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56cc161050 0x7f56cc0b37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:30.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.265+0000 7f56d1236640 1 --2- 172.21.15.105:0/3671289979 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56b8004800 0x7f56cc1f5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:30.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.265+0000 7f56d0a35640 1 -- 172.21.15.105:0/3671289979 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56b8004800 msgr2=0x7f56cc1f5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:30.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.265+0000 7f56d0a35640 1 --2- 172.21.15.105:0/3671289979 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56b8004800 0x7f56cc1f5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:30.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.265+0000 7f56d0a35640 1 -- 172.21.15.105:0/3671289979 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56cc161050 msgr2=0x7f56cc0b37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:30.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.265+0000 7f56d0a35640 1 --2- 172.21.15.105:0/3671289979 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56cc161050 0x7f56cc0b37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:30.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.265+0000 7f56d0a35640 1 -- 172.21.15.105:0/3671289979 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56cc160010 con 0x7f56cc160bd0 2024-04-15T15:19:30.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.265+0000 7f56d1a37640 1 --2- 172.21.15.105:0/3671289979 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56cc161050 0x7f56cc0b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:19:30.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.266+0000 7f56d0a35640 1 --2- 172.21.15.105:0/3671289979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56cc160bd0 0x7f56cc0b3290 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f56bc000c00 tx=0x7f56bc02e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:30.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.266+0000 7f56c1ffb640 1 -- 172.21.15.105:0/3671289979 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56bc046070 con 0x7f56cc160bd0 2024-04-15T15:19:30.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.266+0000 7f56c1ffb640 1 -- 172.21.15.105:0/3671289979 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56bc0072b0 con 0x7f56cc160bd0 2024-04-15T15:19:30.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.266+0000 7f56d349a640 1 -- 172.21.15.105:0/3671289979 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56cc0b4400 con 0x7f56cc160bd0 2024-04-15T15:19:30.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.266+0000 7f56c1ffb640 1 -- 172.21.15.105:0/3671289979 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56bc007de0 con 0x7f56cc160bd0 2024-04-15T15:19:30.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.266+0000 7f56d349a640 1 -- 172.21.15.105:0/3671289979 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56cc0b49c0 con 0x7f56cc160bd0 2024-04-15T15:19:30.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.267+0000 7f56d349a640 1 -- 172.21.15.105:0/3671289979 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56cc0b41f0 con 0x7f56cc160bd0 2024-04-15T15:19:30.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.267+0000 7f56c1ffb640 1 -- 172.21.15.105:0/3671289979 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f56bc041480 con 0x7f56cc160bd0 2024-04-15T15:19:30.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.268+0000 7f56c1ffb640 1 --2- 172.21.15.105:0/3671289979 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f56b006c8f0 0x7f56b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:30.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.268+0000 7f56c1ffb640 1 -- 172.21.15.105:0/3671289979 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f56bc0ace40 con 0x7f56cc160bd0 2024-04-15T15:19:30.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.268+0000 7f56d1236640 1 --2- 172.21.15.105:0/3671289979 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f56b006c8f0 0x7f56b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:30.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.268+0000 7f56d1236640 1 --2- 172.21.15.105:0/3671289979 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f56b006c8f0 0x7f56b006edb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f56c8009aa0 tx=0x7f56c8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:30.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.272+0000 7f56c1ffb640 1 -- 172.21.15.105:0/3671289979 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f56cc15edb0 con 0x7f56cc160bd0 2024-04-15T15:19:30.349 INFO:tasks.ceph.mds.b.smithi154.stderr:2024-04-15T15:19:30.347+0000 7fe2b2e07640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-15T15:19:30.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.448+0000 7f56d349a640 1 -- 172.21.15.105:0/3671289979 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f56cc069150 con 0x7f56cc160bd0 2024-04-15T15:19:30.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.449+0000 7f56c1ffb640 1 -- 172.21.15.105:0/3671289979 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16653 (secure 0 0 0) 0x7f56bc004e20 con 0x7f56cc160bd0 2024-04-15T15:19:30.451 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:19:30.451 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":20,"fsid":"09e4d2a4-a324-4793-af77-644b25e520e1","created":"2024-04-15T15:19:01.524543+0000","modified":"2024-04-15T15:19:27.414890+0000","last_up_change":"2024-04-15T15:19:12.825815+0000","last_in_change":"2024-04-15T15:19:03.788126+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:19:13.748585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:19:23.789287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:19:24.722250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6f6292b-2d7e-4aa0-9d1c-6ddd0f97287c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6801","nonce":2873040054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6803","nonce":2873040054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6807","nonce":2873040054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6805","nonce":2873040054}]},"public_addr":"172.21.15.105:6801/2873040054","cluster_addr":"172.21.15.105:6803/2873040054","heartbeat_back_addr":"172.21.15.105:6807/2873040054","heartbeat_front_addr":"172.21.15.105:6805/2873040054","state":["exists","up"]},{"osd":1,"uuid":"8900966a-b393-41e3-8f93-e5a43bd23622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6818","nonce":2494994479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6821","nonce":2494994479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6828","nonce":2494994479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6823","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6825","nonce":2494994479}]},"public_addr":"172.21.15.105:6818/2494994479","cluster_addr":"172.21.15.105:6821/2494994479","heartbeat_back_addr":"172.21.15.105:6828/2494994479","heartbeat_front_addr":"172.21.15.105:6825/2494994479","state":["exists","up"]},{"osd":2,"uuid":"81a5dc7a-9325-4ff5-941d-a2d5fba21f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6809","nonce":3798408699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6811","nonce":3798408699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6815","nonce":3798408699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6813","nonce":3798408699}]},"public_addr":"172.21.15.105:6809/3798408699","cluster_addr":"172.21.15.105:6811/3798408699","heartbeat_back_addr":"172.21.15.105:6815/3798408699","heartbeat_front_addr":"172.21.15.105:6813/3798408699","state":["exists","up"]},{"osd":3,"uuid":"df766ed2-5c71-4805-96f4-3d4970edc960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6819","nonce":1920720186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6824","nonce":1920720186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6831","nonce":1920720186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6827","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6829","nonce":1920720186}]},"public_addr":"172.21.15.105:6819/1920720186","cluster_addr":"172.21.15.105:6824/1920720186","heartbeat_back_addr":"172.21.15.105:6831/1920720186","heartbeat_front_addr":"172.21.15.105:6829/1920720186","state":["exists","up"]},{"osd":4,"uuid":"e1039e03-28b8-49df-b951-c91930b93c9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6810","nonce":3516494456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6813","nonce":3516494456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6820","nonce":3516494456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6817","nonce":3516494456}]},"public_addr":"172.21.15.154:6810/3516494456","cluster_addr":"172.21.15.154:6813/3516494456","heartbeat_back_addr":"172.21.15.154:6820/3516494456","heartbeat_front_addr":"172.21.15.154:6817/3516494456","state":["exists","up"]},{"osd":5,"uuid":"a9f5eaf1-66ff-405a-a9e4-ec3340812a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6801","nonce":540367751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6803","nonce":540367751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6807","nonce":540367751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6805","nonce":540367751}]},"public_addr":"172.21.15.154:6801/540367751","cluster_addr":"172.21.15.154:6803/540367751","heartbeat_back_addr":"172.21.15.154:6807/540367751","heartbeat_front_addr":"172.21.15.154:6805/540367751","state":["exists","up"]},{"osd":6,"uuid":"267d02ea-675f-428f-aae8-e0375e4d4feb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6825","nonce":1550829005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6827","nonce":1550829005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6831","nonce":1550829005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6829","nonce":1550829005}]},"public_addr":"172.21.15.154:6825/1550829005","cluster_addr":"172.21.15.154:6827/1550829005","heartbeat_back_addr":"172.21.15.154:6831/1550829005","heartbeat_front_addr":"172.21.15.154:6829/1550829005","state":["exists","up"]},{"osd":7,"uuid":"759ac6cc-896a-423c-885f-9fe90d8dcb14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6809","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6811","nonce":3920089903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6816","nonce":3920089903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6823","nonce":3920089903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6819","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6821","nonce":3920089903}]},"public_addr":"172.21.15.154:6811/3920089903","cluster_addr":"172.21.15.154:6816/3920089903","heartbeat_back_addr":"172.21.15.154:6823/3920089903","heartbeat_front_addr":"172.21.15.154:6821/3920089903","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.949509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.984072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.907278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.947391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.975447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.998896+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:09.028739+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.894119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:19:30.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.451+0000 7f56d349a640 1 -- 172.21.15.105:0/3671289979 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f56b006c8f0 msgr2=0x7f56b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:30.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.451+0000 7f56d349a640 1 --2- 172.21.15.105:0/3671289979 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f56b006c8f0 0x7f56b006edb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f56c8009aa0 tx=0x7f56c8008040 comp rx=0 tx=0).stop 2024-04-15T15:19:30.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.451+0000 7f56d349a640 1 -- 172.21.15.105:0/3671289979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56cc160bd0 msgr2=0x7f56cc0b3290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:30.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.451+0000 7f56d349a640 1 --2- 172.21.15.105:0/3671289979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56cc160bd0 0x7f56cc0b3290 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f56bc000c00 tx=0x7f56bc02e070 comp rx=0 tx=0).stop 2024-04-15T15:19:30.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.451+0000 7f56d349a640 1 -- 172.21.15.105:0/3671289979 shutdown_connections 2024-04-15T15:19:30.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.451+0000 7f56d349a640 1 --2- 172.21.15.105:0/3671289979 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f56b006c8f0 0x7f56b006edb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:30.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.451+0000 7f56d349a640 1 --2- 172.21.15.105:0/3671289979 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56cc161050 0x7f56cc0b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:30.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.451+0000 7f56d349a640 1 --2- 172.21.15.105:0/3671289979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56cc160bd0 0x7f56cc0b3290 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:30.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.451+0000 7f56d349a640 1 --2- 172.21.15.105:0/3671289979 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56b8004800 0x7f56cc1f5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:30.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.451+0000 7f56d349a640 1 -- 172.21.15.105:0/3671289979 >> 172.21.15.105:0/3671289979 conn(0x7f56cc067720 msgr2=0x7f56cc0b78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:30.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.451+0000 7f56d349a640 1 -- 172.21.15.105:0/3671289979 shutdown_connections 2024-04-15T15:19:30.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.451+0000 7f56d349a640 1 -- 172.21.15.105:0/3671289979 wait complete. 2024-04-15T15:19:30.464 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-15T15:19:30.464 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:19:30.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.566+0000 7f87c7250640 1 Processor -- start 2024-04-15T15:19:30.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.566+0000 7f87c7250640 1 -- start start 2024-04-15T15:19:30.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.567+0000 7f87c7250640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f87c00d0540 0x7f87c006b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:30.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.567+0000 7f87c7250640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f87c00b3c80 0x7f87c006bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:30.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.567+0000 7f87c7250640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87c00c8270 con 0x7f87c00ce800 2024-04-15T15:19:30.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.567+0000 7f87c7250640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f87c00d0920 con 0x7f87c00d0540 2024-04-15T15:19:30.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.567+0000 7f87c7250640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f87c00c7cb0 con 0x7f87c00b3c80 2024-04-15T15:19:30.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.567+0000 7f87c4fec640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f87c00d0540 0x7f87c006b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:30.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.567+0000 7f87c4fec640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f87c00d0540 0x7f87c006b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39392/0 (socket says 172.21.15.105:39392) 2024-04-15T15:19:30.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.567+0000 7f87c4fec640 1 -- 172.21.15.105:0/2593947302 learned_addr learned my addr 172.21.15.105:0/2593947302 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:30.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.567+0000 7f87c4fec640 1 -- 172.21.15.105:0/2593947302 >> v2:172.21.15.154:3300/0 conn(0x7f87c00b3c80 msgr2=0x7f87c006bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:30.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.567+0000 7f87c4fec640 1 --2- 172.21.15.105:0/2593947302 >> v2:172.21.15.154:3300/0 conn(0x7f87c00b3c80 0x7f87c006bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:30.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.567+0000 7f87c4fec640 1 -- 172.21.15.105:0/2593947302 >> v1:172.21.15.154:6789/0 conn(0x7f87c00ce800 legacy=0x7f87c00c6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:30.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.567+0000 7f87c4fec640 1 -- 172.21.15.105:0/2593947302 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87c00c7f90 con 0x7f87c00d0540 2024-04-15T15:19:30.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.568+0000 7f87c4fec640 1 --2- 172.21.15.105:0/2593947302 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87c00d0540 0x7f87c006b9c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f87bc009e50 tx=0x7f87bc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30a01d847d640afa server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:30.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.568+0000 7f87b77fe640 1 -- 172.21.15.105:0/2593947302 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87bc02ca80 con 0x7f87c00d0540 2024-04-15T15:19:30.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.568+0000 7f87b77fe640 1 -- 172.21.15.105:0/2593947302 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87bc02cc20 con 0x7f87c00d0540 2024-04-15T15:19:30.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.568+0000 7f87b77fe640 1 -- 172.21.15.105:0/2593947302 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87bc0343f0 con 0x7f87c00d0540 2024-04-15T15:19:30.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.569+0000 7f87c7250640 1 -- 172.21.15.105:0/2593947302 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87c00d0540 msgr2=0x7f87c006b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:30.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.569+0000 7f87c7250640 1 --2- 172.21.15.105:0/2593947302 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87c00d0540 0x7f87c006b9c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f87bc009e50 tx=0x7f87bc02bee0 comp rx=0 tx=0).stop 2024-04-15T15:19:30.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.569+0000 7f87c7250640 1 -- 172.21.15.105:0/2593947302 shutdown_connections 2024-04-15T15:19:30.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.569+0000 7f87c7250640 1 --2- 172.21.15.105:0/2593947302 >> v2:172.21.15.154:3300/0 conn(0x7f87c00b3c80 0x7f87c006bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:30.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.569+0000 7f87c7250640 1 --2- 172.21.15.105:0/2593947302 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87c00d0540 0x7f87c006b9c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:30.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.569+0000 7f87c7250640 1 -- 172.21.15.105:0/2593947302 >> 172.21.15.105:0/2593947302 conn(0x7f87c009aa40 msgr2=0x7f87c00b30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:30.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.569+0000 7f87c7250640 1 -- 172.21.15.105:0/2593947302 shutdown_connections 2024-04-15T15:19:30.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.569+0000 7f87c7250640 1 -- 172.21.15.105:0/2593947302 wait complete. 2024-04-15T15:19:30.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.570+0000 7f87c7250640 1 Processor -- start 2024-04-15T15:19:30.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.570+0000 7f87c7250640 1 -- start start 2024-04-15T15:19:30.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.571+0000 7f87c7250640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87c00b3c80 0x7f87c0159660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:30.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.571+0000 7f87c7250640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87c00ce800 0x7f87c009b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:30.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.571+0000 7f87c7250640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f87c00d0540 0x7f87c00b4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:30.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.571+0000 7f87c7250640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f87c0070e40 con 0x7f87c00b3c80 2024-04-15T15:19:30.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.571+0000 7f87c7250640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f87c006f180 con 0x7f87c00ce800 2024-04-15T15:19:30.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.571+0000 7f87c7250640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f87c0070fc0 con 0x7f87c00d0540 2024-04-15T15:19:30.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.571+0000 7f87c4fec640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87c00b3c80 0x7f87c0159660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:30.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.571+0000 7f87c4fec640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87c00b3c80 0x7f87c0159660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39398/0 (socket says 172.21.15.105:39398) 2024-04-15T15:19:30.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.571+0000 7f87c4fec640 1 -- 172.21.15.105:0/801948807 learned_addr learned my addr 172.21.15.105:0/801948807 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:30.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.571+0000 7f87b7fff640 1 --2- 172.21.15.105:0/801948807 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87c00ce800 0x7f87c009b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:30.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.571+0000 7f87c57ed640 1 --2- 172.21.15.105:0/801948807 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f87c00d0540 0x7f87c00b4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:30.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.571+0000 7f87c4fec640 1 -- 172.21.15.105:0/801948807 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f87c00d0540 msgr2=0x7f87c00b4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:30.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.571+0000 7f87c4fec640 1 --2- 172.21.15.105:0/801948807 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f87c00d0540 0x7f87c00b4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:30.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.571+0000 7f87c4fec640 1 -- 172.21.15.105:0/801948807 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87c00ce800 msgr2=0x7f87c009b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:30.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.571+0000 7f87c4fec640 1 --2- 172.21.15.105:0/801948807 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87c00ce800 0x7f87c009b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:30.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.571+0000 7f87c4fec640 1 -- 172.21.15.105:0/801948807 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87c009bee0 con 0x7f87c00b3c80 2024-04-15T15:19:30.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.571+0000 7f87b7fff640 1 --2- 172.21.15.105:0/801948807 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87c00ce800 0x7f87c009b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:19:30.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.571+0000 7f87c57ed640 1 --2- 172.21.15.105:0/801948807 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f87c00d0540 0x7f87c00b4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:19:30.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.572+0000 7f87c4fec640 1 --2- 172.21.15.105:0/801948807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87c00b3c80 0x7f87c0159660 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f87bc009e50 tx=0x7f87bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:30.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.572+0000 7f87b57fa640 1 -- 172.21.15.105:0/801948807 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87bc043020 con 0x7f87c00b3c80 2024-04-15T15:19:30.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.572+0000 7f87b57fa640 1 -- 172.21.15.105:0/801948807 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87bc03e450 con 0x7f87c00b3c80 2024-04-15T15:19:30.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.572+0000 7f87c7250640 1 -- 172.21.15.105:0/801948807 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87c009c1c0 con 0x7f87c00b3c80 2024-04-15T15:19:30.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.572+0000 7f87b57fa640 1 -- 172.21.15.105:0/801948807 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87bc03e840 con 0x7f87c00b3c80 2024-04-15T15:19:30.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.572+0000 7f87c7250640 1 -- 172.21.15.105:0/801948807 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87c00b5b30 con 0x7f87c00b3c80 2024-04-15T15:19:30.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.573+0000 7f87c7250640 1 -- 172.21.15.105:0/801948807 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87c0068a40 con 0x7f87c00b3c80 2024-04-15T15:19:30.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.573+0000 7f87b57fa640 1 -- 172.21.15.105:0/801948807 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f87bc046020 con 0x7f87c00b3c80 2024-04-15T15:19:30.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.573+0000 7f87b57fa640 1 --2- 172.21.15.105:0/801948807 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f87ac06c8f0 0x7f87ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:30.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.574+0000 7f87b7fff640 1 --2- 172.21.15.105:0/801948807 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f87ac06c8f0 0x7f87ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:30.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.574+0000 7f87b57fa640 1 -- 172.21.15.105:0/801948807 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f87bc0ae500 con 0x7f87c00b3c80 2024-04-15T15:19:30.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.574+0000 7f87b7fff640 1 --2- 172.21.15.105:0/801948807 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f87ac06c8f0 0x7f87ac06edb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f87c00c7b90 tx=0x7f87b0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:30.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.578+0000 7f87b57fa640 1 -- 172.21.15.105:0/801948807 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f87bc04d030 con 0x7f87c00b3c80 2024-04-15T15:19:30.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.754+0000 7f87c7250640 1 -- 172.21.15.105:0/801948807 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f87c00c78e0 con 0x7f87c00b3c80 2024-04-15T15:19:30.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.755+0000 7f87b57fa640 1 -- 172.21.15.105:0/801948807 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16653 (secure 0 0 0) 0x7f87bc07c930 con 0x7f87c00b3c80 2024-04-15T15:19:30.757 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:19:30.757 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":20,"fsid":"09e4d2a4-a324-4793-af77-644b25e520e1","created":"2024-04-15T15:19:01.524543+0000","modified":"2024-04-15T15:19:27.414890+0000","last_up_change":"2024-04-15T15:19:12.825815+0000","last_in_change":"2024-04-15T15:19:03.788126+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:19:13.748585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:19:23.789287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:19:24.722250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6f6292b-2d7e-4aa0-9d1c-6ddd0f97287c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6801","nonce":2873040054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6803","nonce":2873040054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6807","nonce":2873040054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6805","nonce":2873040054}]},"public_addr":"172.21.15.105:6801/2873040054","cluster_addr":"172.21.15.105:6803/2873040054","heartbeat_back_addr":"172.21.15.105:6807/2873040054","heartbeat_front_addr":"172.21.15.105:6805/2873040054","state":["exists","up"]},{"osd":1,"uuid":"8900966a-b393-41e3-8f93-e5a43bd23622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6818","nonce":2494994479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6821","nonce":2494994479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6828","nonce":2494994479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6823","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6825","nonce":2494994479}]},"public_addr":"172.21.15.105:6818/2494994479","cluster_addr":"172.21.15.105:6821/2494994479","heartbeat_back_addr":"172.21.15.105:6828/2494994479","heartbeat_front_addr":"172.21.15.105:6825/2494994479","state":["exists","up"]},{"osd":2,"uuid":"81a5dc7a-9325-4ff5-941d-a2d5fba21f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6809","nonce":3798408699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6811","nonce":3798408699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6815","nonce":3798408699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6813","nonce":3798408699}]},"public_addr":"172.21.15.105:6809/3798408699","cluster_addr":"172.21.15.105:6811/3798408699","heartbeat_back_addr":"172.21.15.105:6815/3798408699","heartbeat_front_addr":"172.21.15.105:6813/3798408699","state":["exists","up"]},{"osd":3,"uuid":"df766ed2-5c71-4805-96f4-3d4970edc960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6819","nonce":1920720186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6824","nonce":1920720186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6831","nonce":1920720186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6827","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6829","nonce":1920720186}]},"public_addr":"172.21.15.105:6819/1920720186","cluster_addr":"172.21.15.105:6824/1920720186","heartbeat_back_addr":"172.21.15.105:6831/1920720186","heartbeat_front_addr":"172.21.15.105:6829/1920720186","state":["exists","up"]},{"osd":4,"uuid":"e1039e03-28b8-49df-b951-c91930b93c9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6810","nonce":3516494456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6813","nonce":3516494456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6820","nonce":3516494456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6817","nonce":3516494456}]},"public_addr":"172.21.15.154:6810/3516494456","cluster_addr":"172.21.15.154:6813/3516494456","heartbeat_back_addr":"172.21.15.154:6820/3516494456","heartbeat_front_addr":"172.21.15.154:6817/3516494456","state":["exists","up"]},{"osd":5,"uuid":"a9f5eaf1-66ff-405a-a9e4-ec3340812a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6801","nonce":540367751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6803","nonce":540367751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6807","nonce":540367751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6805","nonce":540367751}]},"public_addr":"172.21.15.154:6801/540367751","cluster_addr":"172.21.15.154:6803/540367751","heartbeat_back_addr":"172.21.15.154:6807/540367751","heartbeat_front_addr":"172.21.15.154:6805/540367751","state":["exists","up"]},{"osd":6,"uuid":"267d02ea-675f-428f-aae8-e0375e4d4feb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6825","nonce":1550829005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6827","nonce":1550829005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6831","nonce":1550829005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6829","nonce":1550829005}]},"public_addr":"172.21.15.154:6825/1550829005","cluster_addr":"172.21.15.154:6827/1550829005","heartbeat_back_addr":"172.21.15.154:6831/1550829005","heartbeat_front_addr":"172.21.15.154:6829/1550829005","state":["exists","up"]},{"osd":7,"uuid":"759ac6cc-896a-423c-885f-9fe90d8dcb14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6809","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6811","nonce":3920089903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6816","nonce":3920089903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6823","nonce":3920089903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6819","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6821","nonce":3920089903}]},"public_addr":"172.21.15.154:6811/3920089903","cluster_addr":"172.21.15.154:6816/3920089903","heartbeat_back_addr":"172.21.15.154:6823/3920089903","heartbeat_front_addr":"172.21.15.154:6821/3920089903","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.949509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.984072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.907278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.947391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.975447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.998896+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:09.028739+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.894119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:19:30.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.757+0000 7f87c7250640 1 -- 172.21.15.105:0/801948807 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f87ac06c8f0 msgr2=0x7f87ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:30.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.757+0000 7f87c7250640 1 --2- 172.21.15.105:0/801948807 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f87ac06c8f0 0x7f87ac06edb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f87c00c7b90 tx=0x7f87b0008040 comp rx=0 tx=0).stop 2024-04-15T15:19:30.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.757+0000 7f87c7250640 1 -- 172.21.15.105:0/801948807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87c00b3c80 msgr2=0x7f87c0159660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:30.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.757+0000 7f87c7250640 1 --2- 172.21.15.105:0/801948807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87c00b3c80 0x7f87c0159660 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f87bc009e50 tx=0x7f87bc004a40 comp rx=0 tx=0).stop 2024-04-15T15:19:30.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.758+0000 7f87c7250640 1 -- 172.21.15.105:0/801948807 shutdown_connections 2024-04-15T15:19:30.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.758+0000 7f87c7250640 1 --2- 172.21.15.105:0/801948807 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f87c00d0540 0x7f87c00b4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:30.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.758+0000 7f87c7250640 1 --2- 172.21.15.105:0/801948807 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f87ac06c8f0 0x7f87ac06edb0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:30.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.758+0000 7f87c7250640 1 --2- 172.21.15.105:0/801948807 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87c00ce800 0x7f87c009b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:30.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.758+0000 7f87c7250640 1 --2- 172.21.15.105:0/801948807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87c00b3c80 0x7f87c0159660 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:30.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.758+0000 7f87c7250640 1 -- 172.21.15.105:0/801948807 >> 172.21.15.105:0/801948807 conn(0x7f87c009aa40 msgr2=0x7f87c015b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:30.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.758+0000 7f87c7250640 1 -- 172.21.15.105:0/801948807 shutdown_connections 2024-04-15T15:19:30.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.758+0000 7f87c7250640 1 -- 172.21.15.105:0/801948807 wait complete. 2024-04-15T15:19:30.771 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-15T15:19:30.771 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-15T15:19:30.771 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-15T15:19:30.772 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-15T15:19:30.772 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-15T15:19:30.773 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-15T15:19:30.773 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-15T15:19:30.773 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-15T15:19:30.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.944+0000 7f758989e640 1 Processor -- start 2024-04-15T15:19:30.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.945+0000 7f758989e640 1 -- start start 2024-04-15T15:19:30.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.945+0000 7f7bf6814640 1 Processor -- start 2024-04-15T15:19:30.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.946+0000 7f7bf6814640 1 -- start start 2024-04-15T15:19:30.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.946+0000 7f758989e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f757c0dfef0 0x7f757c0085d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:30.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.946+0000 7f6c0f509640 1 Processor -- start 2024-04-15T15:19:30.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.946+0000 7f758989e640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f757c056590 con 0x7f757c0e42d0 2024-04-15T15:19:30.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.946+0000 7f758989e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f757c056870 con 0x7f757c0e02d0 2024-04-15T15:19:30.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.946+0000 7f758989e640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f757c0094b0 con 0x7f757c0dfef0 2024-04-15T15:19:30.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.946+0000 7f7583fff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f757c0e02d0 0x7f757c0e3bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.105:36742/0 (socket says 172.21.15.105:36742) 2024-04-15T15:19:30.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.946+0000 7f7583fff640 1 -- 172.21.15.105:0/1744676739 learned_addr learned my addr 172.21.15.105:0/1744676739 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:19:30.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.946+0000 7f6c0f509640 1 -- start start 2024-04-15T15:19:30.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.947+0000 7f758889c640 1 --2- 172.21.15.105:0/1744676739 >> v2:172.21.15.105:3300/0 conn(0x7f757c0dfef0 0x7f757c0085d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:30.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.947+0000 7f75837fe640 1 -- 172.21.15.105:0/1744676739 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1772869788 0 0) 0x7f757c056870 con 0x7f757c0e02d0 2024-04-15T15:19:30.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.947+0000 7f75837fe640 1 -- 172.21.15.105:0/1744676739 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7574003520 con 0x7f757c0e02d0 2024-04-15T15:19:30.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.947+0000 7f75837fe640 1 -- 172.21.15.105:0/1744676739 <== mon.1 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1886913601 0 0) 0x7f7574003520 con 0x7f757c0e02d0 2024-04-15T15:19:30.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.947+0000 7f75837fe640 1 -- 172.21.15.105:0/1744676739 >> v2:172.21.15.105:3300/0 conn(0x7f757c0dfef0 msgr2=0x7f757c0085d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:30.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.947+0000 7f75837fe640 1 --2- 172.21.15.105:0/1744676739 >> v2:172.21.15.105:3300/0 conn(0x7f757c0dfef0 0x7f757c0085d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:30.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.947+0000 7f75837fe640 1 -- 172.21.15.105:0/1744676739 >> v1:172.21.15.105:6789/0 conn(0x7f757c0e42d0 legacy=0x7f757c0539f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:30.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.947+0000 7f75837fe640 1 -- 172.21.15.105:0/1744676739 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f757c008e60 con 0x7f757c0e02d0 2024-04-15T15:19:30.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.947+0000 7f75837fe640 1 -- 172.21.15.105:0/1744676739 <== mon.1 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7f7578001670 con 0x7f757c0e02d0 2024-04-15T15:19:30.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.947+0000 7f75837fe640 1 -- 172.21.15.105:0/1744676739 >> v1:172.21.15.154:6789/0 conn(0x7f757c0e02d0 legacy=0x7f757c0e3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:30.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.947+0000 7f75837fe640 1 --2- 172.21.15.105:0/1744676739 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75740039b0 0x7f7574003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:30.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.947+0000 7f75837fe640 1 --2- 172.21.15.105:0/1744676739 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7574004800 0x7f7574004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:30.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.947+0000 7f75837fe640 1 --2- 172.21.15.105:0/1744676739 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75740051c0 0x7f757400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:30.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.947+0000 7f75837fe640 1 -- 172.21.15.105:0/1744676739 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f757c0094b0 con 0x7f75740039b0 2024-04-15T15:19:30.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.947+0000 7f75837fe640 1 -- 172.21.15.105:0/1744676739 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f757400c040 con 0x7f75740051c0 2024-04-15T15:19:30.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.947+0000 7f75837fe640 1 -- 172.21.15.105:0/1744676739 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f757400c240 con 0x7f7574004800 2024-04-15T15:19:30.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.947+0000 7f758889c640 1 --2- 172.21.15.105:0/1744676739 >> v2:172.21.15.105:3300/0 conn(0x7f757c0dfef0 0x7f757c0085d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:19:30.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.948+0000 7f758889c640 1 --2- 172.21.15.105:0/1744676739 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75740039b0 0x7f7574003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:30.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.948+0000 7f758909d640 1 --2- 172.21.15.105:0/1744676739 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75740051c0 0x7f757400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:30.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.948+0000 7f7583fff640 1 --2- 172.21.15.105:0/1744676739 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7574004800 0x7f7574004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:30.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.947+0000 7f955759e640 1 Processor -- start 2024-04-15T15:19:30.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.947+0000 7f955759e640 1 -- start start 2024-04-15T15:19:30.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.947+0000 7f7bf6814640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7bf0151a50 0x7f7bf015aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:30.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.948+0000 7f7bf6814640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7bf006deb0 0x7f7bf0071eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:30.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.947+0000 7f2907cd9640 1 Processor -- start 2024-04-15T15:19:30.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.947+0000 7f785b304640 1 Processor -- start 2024-04-15T15:19:30.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.948+0000 7f785b304640 1 -- start start 2024-04-15T15:19:30.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.948+0000 7f758889c640 1 -- 172.21.15.105:0/1744676739 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7574004800 msgr2=0x7f7574004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:30.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.948+0000 7f758889c640 1 --2- 172.21.15.105:0/1744676739 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7574004800 0x7f7574004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:30.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.948+0000 7f758889c640 1 -- 172.21.15.105:0/1744676739 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75740051c0 msgr2=0x7f757400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:30.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.948+0000 7f758889c640 1 --2- 172.21.15.105:0/1744676739 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75740051c0 0x7f757400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:30.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.948+0000 7f758889c640 1 -- 172.21.15.105:0/1744676739 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7574003520 con 0x7f75740039b0 2024-04-15T15:19:30.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.947+0000 7f2907cd9640 1 -- start start 2024-04-15T15:19:30.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.947+0000 7f6c0f509640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6c08072500 0x7f6c080ce580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:30.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.947+0000 7f6c0f509640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c0806e610 con 0x7f6c0806fee0 2024-04-15T15:19:30.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.947+0000 7f6c0f509640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c0806e8f0 con 0x7f6c0806de50 2024-04-15T15:19:30.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.948+0000 7f955759e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f955806b9c0 0x7f955806bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:30.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.948+0000 7f955759e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f95580b5d90 0x7f955806f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:30.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.948+0000 7f955759e640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f955806c2e0 con 0x7f95580b4330 2024-04-15T15:19:30.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.948+0000 7f955759e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f95580ae9b0 con 0x7f955806b9c0 2024-04-15T15:19:30.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.948+0000 7f955759e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f9558070800 con 0x7f95580b5d90 2024-04-15T15:19:30.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.947+0000 7f6c0f509640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f6c0806ebd0 con 0x7f6c08072500 2024-04-15T15:19:30.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.948+0000 7f6c0caa4640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6c08072500 0x7f6c080ce580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:30.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.949+0000 7f7bf6814640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bf0151e30 con 0x7f7bf00723f0 2024-04-15T15:19:30.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.948+0000 7f6c0caa4640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6c08072500 0x7f6c080ce580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.105:56752/0 (socket says 172.21.15.105:56752) 2024-04-15T15:19:30.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.948+0000 7f6c0caa4640 1 -- 172.21.15.105:0/2981386822 learned_addr learned my addr 172.21.15.105:0/2981386822 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:30.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.949+0000 7f7bf6814640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f7bf0072840 con 0x7f7bf0151a50 2024-04-15T15:19:30.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.949+0000 7f7bf6814640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f7bf006e690 con 0x7f7bf006deb0 2024-04-15T15:19:30.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.949+0000 7f7bf4db1640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f7bf00723f0 0x7f7bf015e4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.105:36746/0 (socket says 172.21.15.105:36746) 2024-04-15T15:19:30.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.949+0000 7f7bf4db1640 1 -- 172.21.15.105:0/179605387 learned_addr learned my addr 172.21.15.105:0/179605387 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:19:30.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.949+0000 7f7beffff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7bf0151a50 0x7f7bf015aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:30.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.949+0000 7fec80263640 1 Processor -- start 2024-04-15T15:19:30.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.948+0000 7f6c0caa4640 1 -- 172.21.15.105:0/2981386822 >> v1:172.21.15.154:6789/0 conn(0x7f6c0806de50 legacy=0x7f6c08071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:30.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.948+0000 7f6c0caa4640 1 -- 172.21.15.105:0/2981386822 >> v1:172.21.15.105:6789/0 conn(0x7f6c0806fee0 legacy=0x7f6c0815a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:30.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.948+0000 7f6c0caa4640 1 -- 172.21.15.105:0/2981386822 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c08162370 con 0x7f6c08072500 2024-04-15T15:19:30.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.949+0000 7f7bef7fe640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7bf006deb0 0x7f7bf0071eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:30.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.949+0000 7f785b304640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7854162e10 0x7f78540d0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:30.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.949+0000 7f785b304640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f78540d09c0 0x7f78540b6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:30.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.949+0000 7f7beeffd640 1 -- 172.21.15.105:0/179605387 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2975247096 0 0) 0x7f7bf0151e30 con 0x7f7bf00723f0 2024-04-15T15:19:30.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.949+0000 7fec80263640 1 -- start start 2024-04-15T15:19:30.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.949+0000 7f7beeffd640 1 -- 172.21.15.105:0/179605387 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7bd8003520 con 0x7f7bf00723f0 2024-04-15T15:19:30.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.949+0000 7f78590a0640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7854162e10 0x7f78540d0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:30.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.949+0000 7f78590a0640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7854162e10 0x7f78540d0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.105:56780/0 (socket says 172.21.15.105:56780) 2024-04-15T15:19:30.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.949+0000 7f758909d640 1 -- 172.21.15.105:0/1744676739 reap_dead start 2024-04-15T15:19:30.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.949+0000 7f785889f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f78540d09c0 0x7f78540b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:30.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.950+0000 7f785b304640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f78540b6780 0x7f78540d2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:30.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.949+0000 7f9555d9b640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f95580b4330 0x7f95580ae2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:40966/0 (socket says 172.21.15.105:40966) 2024-04-15T15:19:30.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.950+0000 7f785889f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f78540d09c0 0x7f78540b6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39444/0 (socket says 172.21.15.105:39444) 2024-04-15T15:19:30.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.950+0000 7f785889f640 1 -- 172.21.15.105:0/3968999020 learned_addr learned my addr 172.21.15.105:0/3968999020 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:30.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.950+0000 7f7beeffd640 1 -- 172.21.15.105:0/179605387 <== mon.1 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3662847415 0 0) 0x7f7bd8003520 con 0x7f7bf00723f0 2024-04-15T15:19:30.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.950+0000 7f785b304640 1 -- 172.21.15.105:0/3968999020 --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f7854166950 con 0x7f78540d09c0 2024-04-15T15:19:30.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.950+0000 7f785b304640 1 -- 172.21.15.105:0/3968999020 --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f78540d0ed0 con 0x7f78540b6780 2024-04-15T15:19:30.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.950+0000 7f785b304640 1 -- 172.21.15.105:0/3968999020 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f7854166ba0 con 0x7f7854162e10 2024-04-15T15:19:30.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.950+0000 7f955659c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f955806b9c0 0x7f955806bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:30.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.950+0000 7f955659c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f955806b9c0 0x7f955806bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.105:44904/0 (socket says 172.21.15.105:44904) 2024-04-15T15:19:30.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.950+0000 7f955659c640 1 -- 172.21.15.105:0/3785885387 learned_addr learned my addr 172.21.15.105:0/3785885387 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:30.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.950+0000 7f9556d9d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f95580b5d90 0x7f955806f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:30.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.950+0000 7f7beeffd640 1 -- 172.21.15.105:0/179605387 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7bf006deb0 msgr2=0x7f7bf0071eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:30.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.950+0000 7f7beeffd640 1 --2- 172.21.15.105:0/179605387 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7bf006deb0 0x7f7bf0071eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:30.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.950+0000 7f7beeffd640 1 -- 172.21.15.105:0/179605387 >> v2:172.21.15.105:3300/0 conn(0x7f7bf0151a50 msgr2=0x7f7bf015aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:30.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.950+0000 7f7beeffd640 1 --2- 172.21.15.105:0/179605387 >> v2:172.21.15.105:3300/0 conn(0x7f7bf0151a50 0x7f7bf015aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:30.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.950+0000 7f7beeffd640 1 -- 172.21.15.105:0/179605387 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bf015be40 con 0x7f7bf00723f0 2024-04-15T15:19:30.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.950+0000 7f7beeffd640 1 -- 172.21.15.105:0/179605387 <== mon.1 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7f7be8001670 con 0x7f7bf00723f0 2024-04-15T15:19:30.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.950+0000 7f7beeffd640 1 -- 172.21.15.105:0/179605387 >> v1:172.21.15.154:6789/0 conn(0x7f7bf00723f0 legacy=0x7f7bf015e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:30.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.950+0000 7f7beeffd640 1 --2- 172.21.15.105:0/179605387 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7bd8003b80 0x7f7bd8005fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:30.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.950+0000 7f7beeffd640 1 --2- 172.21.15.105:0/179605387 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bd8006530 0x7f7bd80089c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:30.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.950+0000 7f7beeffd640 1 --2- 172.21.15.105:0/179605387 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7bd8009030 0x7f7bd800b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:30.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.950+0000 7f7beeffd640 1 -- 172.21.15.105:0/179605387 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf0072840 con 0x7f7bd8006530 2024-04-15T15:19:30.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.950+0000 7f7beeffd640 1 -- 172.21.15.105:0/179605387 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf006e690 con 0x7f7bd8009030 2024-04-15T15:19:30.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.950+0000 7f7beeffd640 1 -- 172.21.15.105:0/179605387 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f7bd800be20 con 0x7f7bd8003b80 2024-04-15T15:19:30.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.950+0000 7f78598a1640 1 --2- 172.21.15.105:0/3968999020 >> v2:172.21.15.154:3300/0 conn(0x7f78540b6780 0x7f78540d2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:30.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.950+0000 7f7beffff640 1 --2- 172.21.15.105:0/179605387 >> v2:172.21.15.105:3300/0 conn(0x7f7bf0151a50 0x7f7bf015aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:19:30.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.950+0000 7f758889c640 1 --2- 172.21.15.105:0/1744676739 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75740039b0 0x7f7574003d90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f75840654e0 tx=0x7f7574005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c83d9f73ea6bd16b server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:30.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.950+0000 7f75837fe640 1 -- 172.21.15.105:0/1744676739 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75840502c0 con 0x7f75740039b0 2024-04-15T15:19:30.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.950+0000 7f75837fe640 1 -- 172.21.15.105:0/1744676739 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75840ad040 con 0x7f75740039b0 2024-04-15T15:19:30.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.950+0000 7f75837fe640 1 -- 172.21.15.105:0/1744676739 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75840b26a0 con 0x7f75740039b0 2024-04-15T15:19:30.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.950+0000 7f9556d9d640 1 -- 172.21.15.105:0/3785885387 >> v2:172.21.15.154:3300/0 conn(0x7f955806b9c0 msgr2=0x7f955806bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:30.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.950+0000 7f9556d9d640 1 --2- 172.21.15.105:0/3785885387 >> v2:172.21.15.154:3300/0 conn(0x7f955806b9c0 0x7f955806bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:30.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.950+0000 7f7bf4db1640 1 --2- 172.21.15.105:0/179605387 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7bd8009030 0x7f7bd800b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:30.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.949+0000 7f2907cd9640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2900163130 0x7f29000cf820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:30.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.949+0000 7f2907cd9640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f29000d0ef0 0x7f29000d1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:30.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.949+0000 7f2907cd9640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f290015edb0 con 0x7f290015e9d0 2024-04-15T15:19:30.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.949+0000 7f2907cd9640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f29000cef90 con 0x7f2900163130 2024-04-15T15:19:30.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.949+0000 7f2907cd9640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f29000d18a0 con 0x7f29000d0ef0 2024-04-15T15:19:30.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.949+0000 7f2905a75640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f290015e9d0 0x7f29000d3f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.105:36750/0 (socket says 172.21.15.105:36750) 2024-04-15T15:19:30.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.949+0000 7f2905a75640 1 -- 172.21.15.105:0/163871938 learned_addr learned my addr 172.21.15.105:0/163871938 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:19:30.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.949+0000 7f2905274640 1 --2- 172.21.15.105:0/163871938 >> v2:172.21.15.105:3300/0 conn(0x7f2900163130 0x7f29000cf820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:30.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.949+0000 7f2906276640 1 --2- 172.21.15.105:0/163871938 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f29000d0ef0 0x7f29000d1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:30.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.949+0000 7f2904a73640 1 -- 172.21.15.105:0/163871938 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3985587269 0 0) 0x7f28f8001490 con 0x7f290015e9d0 2024-04-15T15:19:30.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.949+0000 7f2904a73640 1 -- 172.21.15.105:0/163871938 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f28ec003520 con 0x7f290015e9d0 2024-04-15T15:19:30.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.950+0000 7f2905274640 1 -- 172.21.15.105:0/163871938 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f29000d0ef0 msgr2=0x7f29000d1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:30.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.950+0000 7f2905274640 1 --2- 172.21.15.105:0/163871938 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f29000d0ef0 0x7f29000d1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:30.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.950+0000 7f2905274640 1 -- 172.21.15.105:0/163871938 >> v1:172.21.15.154:6789/0 conn(0x7f290015e9d0 legacy=0x7f29000d3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:30.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.950+0000 7f2905274640 1 -- 172.21.15.105:0/163871938 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29000ba280 con 0x7f2900163130 2024-04-15T15:19:30.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.950+0000 7f78598a1640 1 -- 172.21.15.105:0/3968999020 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7854162e10 msgr2=0x7f78540d0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:30.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.950+0000 7f78598a1640 1 --2- 172.21.15.105:0/3968999020 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7854162e10 0x7f78540d0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:30.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.950+0000 7f78598a1640 1 -- 172.21.15.105:0/3968999020 >> v2:172.21.15.105:3300/0 conn(0x7f78540d09c0 msgr2=0x7f78540b6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:30.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.950+0000 7f78598a1640 1 --2- 172.21.15.105:0/3968999020 >> v2:172.21.15.105:3300/0 conn(0x7f78540d09c0 0x7f78540b6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:30.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.950+0000 7f78598a1640 1 -- 172.21.15.105:0/3968999020 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7854166e80 con 0x7f78540b6780 2024-04-15T15:19:30.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.950+0000 7f7beffff640 1 --2- 172.21.15.105:0/179605387 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7bd8003b80 0x7f7bd8005fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:30.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.951+0000 7f7bef7fe640 1 --2- 172.21.15.105:0/179605387 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bd8006530 0x7f7bd80089c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:30.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.951+0000 7fec80263640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fec7815e250 0x7fec7806feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:30.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.951+0000 7fec80263640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec7806e840 con 0x7fec7815a7d0 2024-04-15T15:19:30.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.951+0000 7fec80263640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec7806eb20 con 0x7fec780703f0 2024-04-15T15:19:30.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.951+0000 7fec80263640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fec780d0210 con 0x7fec7815e250 2024-04-15T15:19:30.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.951+0000 7fe06b5c8640 1 Processor -- start 2024-04-15T15:19:30.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.951+0000 7f758989e640 1 -- 172.21.15.105:0/1744676739 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75740039b0 msgr2=0x7f7574003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:30.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.951+0000 7f758989e640 1 --2- 172.21.15.105:0/1744676739 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75740039b0 0x7f7574003d90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f75840654e0 tx=0x7f7574005830 comp rx=0 tx=0).stop 2024-04-15T15:19:30.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.951+0000 7f9556d9d640 1 -- 172.21.15.105:0/3785885387 >> v1:172.21.15.105:6789/0 conn(0x7f95580b4330 legacy=0x7f95580ae2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:30.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.951+0000 7f9556d9d640 1 -- 172.21.15.105:0/3785885387 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9558070a50 con 0x7f95580b5d90 2024-04-15T15:19:30.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.951+0000 7f7bf4db1640 1 -- 172.21.15.105:0/179605387 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7bd8003b80 msgr2=0x7f7bd8005fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:30.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.951+0000 7f7bf4db1640 1 --2- 172.21.15.105:0/179605387 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7bd8003b80 0x7f7bd8005fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:30.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.951+0000 7f7bf4db1640 1 -- 172.21.15.105:0/179605387 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bd8006530 msgr2=0x7f7bd80089c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:30.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.951+0000 7f7bf4db1640 1 --2- 172.21.15.105:0/179605387 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bd8006530 0x7f7bd80089c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:30.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.951+0000 7f7bf4db1640 1 -- 172.21.15.105:0/179605387 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bf015be40 con 0x7f7bd8009030 2024-04-15T15:19:30.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.951+0000 7f2905274640 1 --2- 172.21.15.105:0/163871938 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2900163130 0x7f29000cf820 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f28f00133b0 tx=0x7f28f002ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=655090d8836a5840 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:30.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.951+0000 7fec7f001640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fec780703f0 0x7fec780b3470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.105:36754/0 (socket says 172.21.15.105:36754) 2024-04-15T15:19:30.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.951+0000 7fec7f001640 1 -- 172.21.15.105:0/3584445408 learned_addr learned my addr 172.21.15.105:0/3584445408 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:19:30.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.951+0000 7fe06b5c8640 1 -- start start 2024-04-15T15:19:30.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.951+0000 7f758989e640 1 -- 172.21.15.105:0/1744676739 shutdown_connections 2024-04-15T15:19:30.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.951+0000 7f758989e640 1 --2- 172.21.15.105:0/1744676739 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75740039b0 0x7f7574003d90 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:30.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.951+0000 7f758989e640 1 -- 172.21.15.105:0/1744676739 >> 172.21.15.105:0/1744676739 conn(0x7f757c01ef40 msgr2=0x7f757c055c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:30.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.951+0000 7f6c0caa4640 1 --2- 172.21.15.105:0/2981386822 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6c08072500 0x7f6c080ce580 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6c04013180 tx=0x7f6c040341a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3193ff6d907d944e server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:30.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.951+0000 7f2904a73640 1 -- 172.21.15.105:0/163871938 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f0033070 con 0x7f2900163130 2024-04-15T15:19:30.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.951+0000 7f2904a73640 1 -- 172.21.15.105:0/163871938 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28f00112d0 con 0x7f2900163130 2024-04-15T15:19:30.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.951+0000 7f2904a73640 1 -- 172.21.15.105:0/163871938 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f00116a0 con 0x7f2900163130 2024-04-15T15:19:30.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.952+0000 7f78598a1640 1 --2- 172.21.15.105:0/3968999020 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78540b6780 0x7f78540d2390 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f78540d3c70 tx=0x7f784c02bdc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d883c12701f038da server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:30.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.952+0000 7f758989e640 1 -- 172.21.15.105:0/1744676739 shutdown_connections 2024-04-15T15:19:30.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.952+0000 7fec7d7fe640 1 --2- 172.21.15.105:0/3584445408 >> v2:172.21.15.105:3300/0 conn(0x7fec7815e250 0x7fec7806feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:30.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.952+0000 7f6bfffff640 1 -- 172.21.15.105:0/2981386822 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c040110d0 con 0x7f6c08072500 2024-04-15T15:19:30.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.952+0000 7fec7cffd640 1 -- 172.21.15.105:0/3584445408 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3006700592 0 0) 0x7fec7806eb20 con 0x7fec780703f0 2024-04-15T15:19:30.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.952+0000 7fec7cffd640 1 -- 172.21.15.105:0/3584445408 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec64003520 con 0x7fec780703f0 2024-04-15T15:19:30.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.952+0000 7f9556d9d640 1 --2- 172.21.15.105:0/3785885387 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f95580b5d90 0x7f955806f5a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9550009d90 tx=0x7f955002bc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fa3bd31ef287ac server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:30.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.952+0000 7f6bfffff640 1 -- 172.21.15.105:0/2981386822 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c04011270 con 0x7f6c08072500 2024-04-15T15:19:30.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.952+0000 7f2907cd9640 1 -- 172.21.15.105:0/163871938 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2900163130 msgr2=0x7f29000cf820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:30.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.952+0000 7f2907cd9640 1 --2- 172.21.15.105:0/163871938 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2900163130 0x7f29000cf820 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f28f00133b0 tx=0x7f28f002ba90 comp rx=0 tx=0).stop 2024-04-15T15:19:30.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.953+0000 7f7bf4db1640 1 --2- 172.21.15.105:0/179605387 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7bd8009030 0x7f7bd800b4c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7be800c340 tx=0x7f7bd8003810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad99fb5ffa278903 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:30.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.953+0000 7fec7cffd640 1 -- 172.21.15.105:0/3584445408 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 121924751 0 0) 0x7fec7806e840 con 0x7fec7815a7d0 2024-04-15T15:19:30.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.953+0000 7f784bfff640 1 -- 172.21.15.105:0/3968999020 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784c039070 con 0x7f78540b6780 2024-04-15T15:19:30.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.953+0000 7f784bfff640 1 -- 172.21.15.105:0/3968999020 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f784c02cb70 con 0x7f78540b6780 2024-04-15T15:19:30.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.953+0000 7fe06b5c8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe05c04b3b0 0x7fe05c034010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:30.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.953+0000 7fe06b5c8640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe05c006a40 0x7fe05c006ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:30.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.953+0000 7fe06b5c8640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe05c054c00 con 0x7fe05c04f4c0 2024-04-15T15:19:30.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.953+0000 7fe06b5c8640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fe05c0062d0 con 0x7fe05c006a40 2024-04-15T15:19:30.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.953+0000 7fe06b5c8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fe05c053a40 con 0x7fe05c04b3b0 2024-04-15T15:19:30.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.953+0000 7fec7cffd640 1 -- 172.21.15.105:0/3584445408 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec7806eb20 con 0x7fec7815a7d0 2024-04-15T15:19:30.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.953+0000 7f784bfff640 1 -- 172.21.15.105:0/3968999020 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784c034440 con 0x7f78540b6780 2024-04-15T15:19:30.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.953+0000 7fec7cffd640 1 -- 172.21.15.105:0/3584445408 <== mon.1 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 824106628 0 0) 0x7fec64003520 con 0x7fec780703f0 2024-04-15T15:19:30.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.953+0000 7fe069364640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe05c04b3b0 0x7fe05c034010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:30.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.953+0000 7fe069b65640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe05c006a40 0x7fe05c006ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:30.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.953+0000 7fec7d7fe640 1 -- 172.21.15.105:0/3584445408 >> v1:172.21.15.154:6789/0 conn(0x7fec780703f0 legacy=0x7fec780b3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:30.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.953+0000 7fec7d7fe640 1 -- 172.21.15.105:0/3584445408 >> v1:172.21.15.105:6789/0 conn(0x7fec7815a7d0 legacy=0x7fec7806e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:30.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.953+0000 7fe069b65640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe05c006a40 0x7fe05c006ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39484/0 (socket says 172.21.15.105:39484) 2024-04-15T15:19:30.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.953+0000 7fec7d7fe640 1 -- 172.21.15.105:0/3584445408 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec7806ee00 con 0x7fec7815e250 2024-04-15T15:19:30.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.953+0000 7fe069b65640 1 -- 172.21.15.105:0/3516516396 learned_addr learned my addr 172.21.15.105:0/3516516396 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:30.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.953+0000 7fe068b63640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fe05c04f4c0 0x7fe05c04f940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.105:36760/0 (socket says 172.21.15.105:36760) 2024-04-15T15:19:30.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.953+0000 7f7beeffd640 1 -- 172.21.15.105:0/179605387 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be802cc20 con 0x7f7bd8009030 2024-04-15T15:19:30.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.953+0000 7f7beeffd640 1 -- 172.21.15.105:0/179605387 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7be8030070 con 0x7f7bd8009030 2024-04-15T15:19:30.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.953+0000 7f7beeffd640 1 -- 172.21.15.105:0/179605387 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be80058d0 con 0x7f7bd8009030 2024-04-15T15:19:30.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.952+0000 7f758989e640 1 -- 172.21.15.105:0/1744676739 wait complete. 2024-04-15T15:19:30.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.953+0000 7f6bfffff640 1 -- 172.21.15.105:0/2981386822 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c04011640 con 0x7f6c08072500 2024-04-15T15:19:30.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.953+0000 7f6c0f509640 1 -- 172.21.15.105:0/2981386822 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6c08072500 msgr2=0x7f6c080ce580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:30.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.953+0000 7f6c0f509640 1 --2- 172.21.15.105:0/2981386822 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6c08072500 0x7f6c080ce580 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6c04013180 tx=0x7f6c040341a0 comp rx=0 tx=0).stop 2024-04-15T15:19:30.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.953+0000 7fe069b65640 1 -- 172.21.15.105:0/3516516396 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe05c04b3b0 msgr2=0x7fe05c034010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:30.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.953+0000 7fe069b65640 1 --2- 172.21.15.105:0/3516516396 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe05c04b3b0 0x7fe05c034010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:30.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.953+0000 7fe069b65640 1 -- 172.21.15.105:0/3516516396 >> v1:172.21.15.154:6789/0 conn(0x7fe05c04f4c0 legacy=0x7fe05c04f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:30.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.953+0000 7fe069b65640 1 -- 172.21.15.105:0/3516516396 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe05c053d20 con 0x7fe05c006a40 2024-04-15T15:19:30.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.953+0000 7f758989e640 1 Processor -- start 2024-04-15T15:19:30.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.953+0000 7f955559a640 1 -- 172.21.15.105:0/3785885387 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9550039070 con 0x7f95580b5d90 2024-04-15T15:19:30.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.953+0000 7fe069364640 1 --2- 172.21.15.105:0/3516516396 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe05c04b3b0 0x7fe05c034010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:19:30.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.953+0000 7f6c0f509640 1 -- 172.21.15.105:0/2981386822 shutdown_connections 2024-04-15T15:19:30.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.953+0000 7f6c0f509640 1 --2- 172.21.15.105:0/2981386822 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6c08072500 0x7f6c080ce580 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:30.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.954+0000 7f6c0f509640 1 -- 172.21.15.105:0/2981386822 >> 172.21.15.105:0/2981386822 conn(0x7f6c08067b70 msgr2=0x7f6c0806b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:30.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.953+0000 7f955559a640 1 -- 172.21.15.105:0/3785885387 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f955002c980 con 0x7f95580b5d90 2024-04-15T15:19:30.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.953+0000 7f955559a640 1 -- 172.21.15.105:0/3785885387 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f955002cd50 con 0x7f95580b5d90 2024-04-15T15:19:30.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.954+0000 7f2907cd9640 1 -- 172.21.15.105:0/163871938 shutdown_connections 2024-04-15T15:19:30.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.954+0000 7f2907cd9640 1 --2- 172.21.15.105:0/163871938 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f29000d0ef0 0x7f29000d1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:30.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.954+0000 7f2907cd9640 1 --2- 172.21.15.105:0/163871938 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2900163130 0x7f29000cf820 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:30.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.954+0000 7f2907cd9640 1 -- 172.21.15.105:0/163871938 >> 172.21.15.105:0/163871938 conn(0x7f2900067720 msgr2=0x7f2900067b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:30.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.954+0000 7f785b304640 1 -- 172.21.15.105:0/3968999020 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78540b6780 msgr2=0x7f78540d2390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:30.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.954+0000 7f2907cd9640 1 -- 172.21.15.105:0/163871938 shutdown_connections 2024-04-15T15:19:30.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.954+0000 7f785b304640 1 --2- 172.21.15.105:0/3968999020 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78540b6780 0x7f78540d2390 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f78540d3c70 tx=0x7f784c02bdc0 comp rx=0 tx=0).stop 2024-04-15T15:19:30.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.954+0000 7f785b304640 1 -- 172.21.15.105:0/3968999020 shutdown_connections 2024-04-15T15:19:30.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.954+0000 7f785b304640 1 --2- 172.21.15.105:0/3968999020 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78540b6780 0x7f78540d2390 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:30.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.954+0000 7f785b304640 1 --2- 172.21.15.105:0/3968999020 >> v2:172.21.15.105:3300/0 conn(0x7f78540d09c0 0x7f78540b6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:30.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.954+0000 7f785b304640 1 --2- 172.21.15.105:0/3968999020 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7854162e10 0x7f78540d0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:30.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.954+0000 7f785b304640 1 -- 172.21.15.105:0/3968999020 >> 172.21.15.105:0/3968999020 conn(0x7f78540d1240 msgr2=0x7f78540d1650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:30.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.954+0000 7fec7d7fe640 1 --2- 172.21.15.105:0/3584445408 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec7815e250 0x7fec7806feb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fec680081e0 tx=0x7fec6802dc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f72239a4f367e854 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:30.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.954+0000 7f785b304640 1 -- 172.21.15.105:0/3968999020 shutdown_connections 2024-04-15T15:19:30.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.954+0000 7f785b304640 1 -- 172.21.15.105:0/3968999020 wait complete. 2024-04-15T15:19:30.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.954+0000 7f2907cd9640 1 -- 172.21.15.105:0/163871938 wait complete. 2024-04-15T15:19:30.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.954+0000 7f955759e640 1 -- 172.21.15.105:0/3785885387 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f95580b5d90 msgr2=0x7f955806f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:30.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.954+0000 7f955759e640 1 --2- 172.21.15.105:0/3785885387 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f95580b5d90 0x7f955806f5a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9550009d90 tx=0x7f955002bc50 comp rx=0 tx=0).stop 2024-04-15T15:19:30.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.954+0000 7fec7cffd640 1 -- 172.21.15.105:0/3584445408 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec740015f0 con 0x7fec7815e250 2024-04-15T15:19:30.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.954+0000 7fec7cffd640 1 -- 172.21.15.105:0/3584445408 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec68008bf0 con 0x7fec7815e250 2024-04-15T15:19:30.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.954+0000 7fec7cffd640 1 -- 172.21.15.105:0/3584445408 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6800a0d0 con 0x7fec7815e250 2024-04-15T15:19:30.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.955+0000 7f7bf6814640 1 -- 172.21.15.105:0/179605387 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7bd8009030 msgr2=0x7f7bd800b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:30.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.955+0000 7f7bf6814640 1 --2- 172.21.15.105:0/179605387 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7bd8009030 0x7f7bd800b4c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7be800c340 tx=0x7f7bd8003810 comp rx=0 tx=0).stop 2024-04-15T15:19:30.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.955+0000 7fe069b65640 1 --2- 172.21.15.105:0/3516516396 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe05c006a40 0x7fe05c006ed0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe060009870 tx=0x7fe06002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30fe389ac074635c server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:30.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.955+0000 7f7bf4db1640 1 -- 172.21.15.105:0/179605387 reap_dead start 2024-04-15T15:19:30.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.955+0000 7f955759e640 1 -- 172.21.15.105:0/3785885387 shutdown_connections 2024-04-15T15:19:30.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.955+0000 7f7bf6814640 1 -- 172.21.15.105:0/179605387 shutdown_connections 2024-04-15T15:19:30.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.955+0000 7f7bf6814640 1 -- 172.21.15.105:0/179605387 >> 172.21.15.105:0/179605387 conn(0x7f7bf0067e10 msgr2=0x7f7bf00b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:30.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.955+0000 7fe057fff640 1 -- 172.21.15.105:0/3516516396 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe060034070 con 0x7fe05c006a40 2024-04-15T15:19:30.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.955+0000 7fe057fff640 1 -- 172.21.15.105:0/3516516396 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe060030840 con 0x7fe05c006a40 2024-04-15T15:19:30.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.955+0000 7fe057fff640 1 -- 172.21.15.105:0/3516516396 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe060030c10 con 0x7fe05c006a40 2024-04-15T15:19:30.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.955+0000 7f6c0f509640 1 -- 172.21.15.105:0/2981386822 shutdown_connections 2024-04-15T15:19:30.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.954+0000 7f758989e640 1 -- start start 2024-04-15T15:19:30.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.955+0000 7f955759e640 1 --2- 172.21.15.105:0/3785885387 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f95580b5d90 0x7f955806f5a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:30.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.955+0000 7f955759e640 1 --2- 172.21.15.105:0/3785885387 >> v2:172.21.15.154:3300/0 conn(0x7f955806b9c0 0x7f955806bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:30.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.955+0000 7f758989e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75740039b0 0x7f757c0087c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:30.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.955+0000 7f758989e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f757c0da3e0 0x7f757c052270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:30.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.955+0000 7f758989e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f757c0527b0 0x7f757c007350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:30.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.955+0000 7f758989e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f757c0e4910 con 0x7f757c0527b0 2024-04-15T15:19:30.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.955+0000 7f758989e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f757c056b50 con 0x7f75740039b0 2024-04-15T15:19:30.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.955+0000 7f758989e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f757c0e3bc0 con 0x7f757c0da3e0 2024-04-15T15:19:30.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.955+0000 7f955759e640 1 -- 172.21.15.105:0/3785885387 >> 172.21.15.105:0/3785885387 conn(0x7f95580b67e0 msgr2=0x7f95580b6bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:30.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.955+0000 7f955759e640 1 -- 172.21.15.105:0/3785885387 shutdown_connections 2024-04-15T15:19:30.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.955+0000 7f955759e640 1 -- 172.21.15.105:0/3785885387 wait complete. 2024-04-15T15:19:30.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.955+0000 7f7583fff640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f757c0da3e0 0x7f757c052270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:30.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.955+0000 7f7583fff640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f757c0da3e0 0x7f757c052270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.105:56816/0 (socket says 172.21.15.105:56816) 2024-04-15T15:19:30.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.955+0000 7f7583fff640 1 -- 172.21.15.105:0/1498572065 learned_addr learned my addr 172.21.15.105:0/1498572065 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:30.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.955+0000 7f758909d640 1 --2- 172.21.15.105:0/1498572065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f757c0527b0 0x7f757c007350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:30.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.955+0000 7f758889c640 1 --2- 172.21.15.105:0/1498572065 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75740039b0 0x7f757c0087c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:30.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.955+0000 7f6c0f509640 1 -- 172.21.15.105:0/2981386822 wait complete. 2024-04-15T15:19:30.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.956+0000 7f7583fff640 1 -- 172.21.15.105:0/1498572065 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75740039b0 msgr2=0x7f757c0087c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:30.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.956+0000 7f7583fff640 1 --2- 172.21.15.105:0/1498572065 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75740039b0 0x7f757c0087c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:30.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.956+0000 7f7583fff640 1 -- 172.21.15.105:0/1498572065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f757c0527b0 msgr2=0x7f757c007350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:30.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.956+0000 7f7583fff640 1 --2- 172.21.15.105:0/1498572065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f757c0527b0 0x7f757c007350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:30.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.956+0000 7f7583fff640 1 -- 172.21.15.105:0/1498572065 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f757c0506d0 con 0x7f757c0da3e0 2024-04-15T15:19:30.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.956+0000 7f7bf6814640 1 -- 172.21.15.105:0/179605387 shutdown_connections 2024-04-15T15:19:30.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.956+0000 7fec80263640 1 -- 172.21.15.105:0/3584445408 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec7815e250 msgr2=0x7fec7806feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:30.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.956+0000 7f7bf6814640 1 -- 172.21.15.105:0/179605387 wait complete. 2024-04-15T15:19:30.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.956+0000 7f7583fff640 1 --2- 172.21.15.105:0/1498572065 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f757c0da3e0 0x7f757c052270 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7578001050 tx=0x7f757800ec10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:30.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.956+0000 7f2907cd9640 1 Processor -- start 2024-04-15T15:19:30.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.956+0000 7f2907cd9640 1 -- start start 2024-04-15T15:19:30.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.956+0000 7fe06b5c8640 1 -- 172.21.15.105:0/3516516396 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe05c006a40 msgr2=0x7fe05c006ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:30.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.956+0000 7fe06b5c8640 1 --2- 172.21.15.105:0/3516516396 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe05c006a40 0x7fe05c006ed0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe060009870 tx=0x7fe06002b980 comp rx=0 tx=0).stop 2024-04-15T15:19:30.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.956+0000 7f785b304640 1 Processor -- start 2024-04-15T15:19:30.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.956+0000 7f785b304640 1 -- start start 2024-04-15T15:19:30.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.956+0000 7fec80263640 1 --2- 172.21.15.105:0/3584445408 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec7815e250 0x7fec7806feb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fec680081e0 tx=0x7fec6802dc70 comp rx=0 tx=0).stop 2024-04-15T15:19:30.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.956+0000 7fec80263640 1 -- 172.21.15.105:0/3584445408 shutdown_connections 2024-04-15T15:19:30.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.956+0000 7fec80263640 1 --2- 172.21.15.105:0/3584445408 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec7815e250 0x7fec7806feb0 secure :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fec680081e0 tx=0x7fec6802dc70 comp rx=0 tx=0).stop 2024-04-15T15:19:30.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.956+0000 7fec80263640 1 -- 172.21.15.105:0/3584445408 >> 172.21.15.105:0/3584445408 conn(0x7fec780b2170 msgr2=0x7fec780b2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:30.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.956+0000 7fe06b5c8640 1 -- 172.21.15.105:0/3516516396 shutdown_connections 2024-04-15T15:19:30.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.956+0000 7fe06b5c8640 1 --2- 172.21.15.105:0/3516516396 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe05c006a40 0x7fe05c006ed0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:30.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.956+0000 7fe06b5c8640 1 --2- 172.21.15.105:0/3516516396 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe05c04b3b0 0x7fe05c034010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:30.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.956+0000 7fe06b5c8640 1 -- 172.21.15.105:0/3516516396 >> 172.21.15.105:0/3516516396 conn(0x7fe05c00a3f0 msgr2=0x7fe05c00a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:30.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.956+0000 7fe06b5c8640 1 -- 172.21.15.105:0/3516516396 shutdown_connections 2024-04-15T15:19:30.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.956+0000 7f75817fa640 1 -- 172.21.15.105:0/1498572065 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757800ee00 con 0x7f757c0da3e0 2024-04-15T15:19:30.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.957+0000 7f75817fa640 1 -- 172.21.15.105:0/1498572065 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7578018070 con 0x7f757c0da3e0 2024-04-15T15:19:30.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.957+0000 7f758989e640 1 -- 172.21.15.105:0/1498572065 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f757c0509b0 con 0x7f757c0da3e0 2024-04-15T15:19:30.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.957+0000 7f758989e640 1 -- 172.21.15.105:0/1498572065 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f757c050f70 con 0x7f757c0da3e0 2024-04-15T15:19:30.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.957+0000 7f2907cd9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29000d0ef0 0x7f29000b41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:30.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.957+0000 7f75817fa640 1 -- 172.21.15.105:0/1498572065 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757800cb10 con 0x7f757c0da3e0 2024-04-15T15:19:30.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.957+0000 7f955759e640 1 Processor -- start 2024-04-15T15:19:30.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.957+0000 7f2907cd9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f290015e9d0 0x7f2900161840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:30.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.957+0000 7f2907cd9640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2900163130 0x7f2900157980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:30.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.957+0000 7f2907cd9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f29000cfd60 con 0x7f29000d0ef0 2024-04-15T15:19:30.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.957+0000 7f785b304640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78540b6780 0x7f78540ca460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:30.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.957+0000 7f2907cd9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f29000cf250 con 0x7f290015e9d0 2024-04-15T15:19:30.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.957+0000 7f2907cd9640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f29000cfee0 con 0x7f2900163130 2024-04-15T15:19:30.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.957+0000 7f785b304640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78540d09c0 0x7f78540ce110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:30.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.957+0000 7f2905a75640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29000d0ef0 0x7f29000b41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:30.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.957+0000 7f78590a0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78540b6780 0x7f78540ca460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:30.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.957+0000 7f78590a0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78540b6780 0x7f78540ca460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.105:44928/0 (socket says 172.21.15.105:44928) 2024-04-15T15:19:30.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.957+0000 7f2905a75640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29000d0ef0 0x7f29000b41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39490/0 (socket says 172.21.15.105:39490) 2024-04-15T15:19:30.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.957+0000 7f785889f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78540d09c0 0x7f78540ce110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:30.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.957+0000 7f2905a75640 1 -- 172.21.15.105:0/2973168466 learned_addr learned my addr 172.21.15.105:0/2973168466 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:31.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.957+0000 7f785889f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78540d09c0 0x7f78540ce110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39500/0 (socket says 172.21.15.105:39500) 2024-04-15T15:19:31.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.958+0000 7f78590a0640 1 -- 172.21.15.105:0/4219357178 learned_addr learned my addr 172.21.15.105:0/4219357178 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:31.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.958+0000 7f785b304640 1 --2- 172.21.15.105:0/4219357178 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7854162e10 0x7f78540cc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.958+0000 7f2905a75640 1 -- 172.21.15.105:0/2973168466 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2900163130 msgr2=0x7f2900157980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-15T15:19:31.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.958+0000 7f2905a75640 1 --2- 172.21.15.105:0/2973168466 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2900163130 0x7f2900157980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.958+0000 7f2905a75640 1 -- 172.21.15.105:0/2973168466 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f290015e9d0 msgr2=0x7f2900161840 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-15T15:19:31.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.958+0000 7f2905a75640 1 --2- 172.21.15.105:0/2973168466 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f290015e9d0 0x7f2900161840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.958+0000 7f2905a75640 1 -- 172.21.15.105:0/2973168466 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29000b6f00 con 0x7f29000d0ef0 2024-04-15T15:19:31.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.958+0000 7f785b304640 1 -- 172.21.15.105:0/4219357178 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7854167680 con 0x7f78540d09c0 2024-04-15T15:19:31.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.958+0000 7f2905a75640 1 --2- 172.21.15.105:0/2973168466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29000d0ef0 0x7f29000b41e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f28f800cb50 tx=0x7f28f800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:31.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.958+0000 7f758989e640 1 -- 172.21.15.105:0/1498572065 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f757c0057e0 con 0x7f757c0da3e0 2024-04-15T15:19:31.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.958+0000 7f78598a1640 1 --2- 172.21.15.105:0/4219357178 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7854162e10 0x7f78540cc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.958+0000 7f785b304640 1 -- 172.21.15.105:0/4219357178 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f78541631f0 con 0x7f78540b6780 2024-04-15T15:19:31.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.958+0000 7f785b304640 1 -- 172.21.15.105:0/4219357178 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f78541659b0 con 0x7f7854162e10 2024-04-15T15:19:31.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.958+0000 7f7bf6814640 1 Processor -- start 2024-04-15T15:19:31.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.957+0000 7f955759e640 1 -- start start 2024-04-15T15:19:31.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.958+0000 7f955759e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f955806b9c0 0x7f9558156780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.958+0000 7f955759e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95580b4330 0x7f9558160be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.958+0000 7f955759e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f95580b5d90 0x7f9558155e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.958+0000 7f955759e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9558162f60 con 0x7f955806b9c0 2024-04-15T15:19:31.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.958+0000 7f955759e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f955806f080 con 0x7f95580b4330 2024-04-15T15:19:31.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.958+0000 7f955759e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f95581630e0 con 0x7f95580b5d90 2024-04-15T15:19:31.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.956+0000 7fe06b5c8640 1 -- 172.21.15.105:0/3516516396 wait complete. 2024-04-15T15:19:31.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.958+0000 7fe06b5c8640 1 Processor -- start 2024-04-15T15:19:31.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.958+0000 7f7bf6814640 1 -- start start 2024-04-15T15:19:31.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.958+0000 7fec80263640 1 -- 172.21.15.105:0/3584445408 shutdown_connections 2024-04-15T15:19:31.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.958+0000 7f9555d9b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95580b4330 0x7f9558160be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.959+0000 7f9555d9b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95580b4330 0x7f9558160be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.105:44940/0 (socket says 172.21.15.105:44940) 2024-04-15T15:19:31.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.958+0000 7f78590a0640 1 -- 172.21.15.105:0/4219357178 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7854162e10 msgr2=0x7f78540cc0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.958+0000 7f78590a0640 1 --2- 172.21.15.105:0/4219357178 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7854162e10 0x7f78540cc0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.958+0000 7f78590a0640 1 -- 172.21.15.105:0/4219357178 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78540d09c0 msgr2=0x7f78540ce110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.958+0000 7f78590a0640 1 --2- 172.21.15.105:0/4219357178 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78540d09c0 0x7f78540ce110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.958+0000 7f78590a0640 1 -- 172.21.15.105:0/4219357178 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7854165c90 con 0x7f78540b6780 2024-04-15T15:19:31.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.959+0000 7f9555d9b640 1 -- 172.21.15.105:0/651811173 learned_addr learned my addr 172.21.15.105:0/651811173 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:31.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.959+0000 7f955659c640 1 --2- 172.21.15.105:0/651811173 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f955806b9c0 0x7f9558156780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.959+0000 7f9556d9d640 1 --2- 172.21.15.105:0/651811173 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f95580b5d90 0x7f9558155e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.957+0000 7f6c0f509640 1 Processor -- start 2024-04-15T15:19:31.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.957+0000 7f6c0f509640 1 -- start start 2024-04-15T15:19:31.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.958+0000 7f6c0f509640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6c0806de50 0x7f6c08144000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.958+0000 7f6c0f509640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c0806fee0 0x7f6c081448c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.959+0000 7f6c0f509640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c08144e00 0x7f6c081533c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.959+0000 7f6c0f509640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6c0815acd0 con 0x7f6c08144e00 2024-04-15T15:19:31.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.959+0000 7f6c0f509640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6c0815ab50 con 0x7f6c0806fee0 2024-04-15T15:19:31.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.959+0000 7f6c0f509640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f6c0815ae50 con 0x7f6c0806de50 2024-04-15T15:19:31.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.959+0000 7f6c0caa4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c0806fee0 0x7f6c081448c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.959+0000 7f6c0caa4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c0806fee0 0x7f6c081448c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.105:44950/0 (socket says 172.21.15.105:44950) 2024-04-15T15:19:31.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.959+0000 7f6c0caa4640 1 -- 172.21.15.105:0/3051101491 learned_addr learned my addr 172.21.15.105:0/3051101491 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:31.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.958+0000 7fe06b5c8640 1 -- start start 2024-04-15T15:19:31.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.959+0000 7f78590a0640 1 --2- 172.21.15.105:0/4219357178 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78540b6780 0x7f78540ca460 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f785000d800 tx=0x7f785000dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:31.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.959+0000 7f9555d9b640 1 -- 172.21.15.105:0/651811173 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f95580b5d90 msgr2=0x7f9558155e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.959+0000 7f9555d9b640 1 --2- 172.21.15.105:0/651811173 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f95580b5d90 0x7f9558155e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.959+0000 7f9555d9b640 1 -- 172.21.15.105:0/651811173 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f955806b9c0 msgr2=0x7f9558156780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.958+0000 7fec80263640 1 -- 172.21.15.105:0/3584445408 wait complete. 2024-04-15T15:19:31.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.959+0000 7f9555d9b640 1 --2- 172.21.15.105:0/651811173 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f955806b9c0 0x7f9558156780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.959+0000 7f9555d9b640 1 -- 172.21.15.105:0/651811173 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9558161310 con 0x7f95580b4330 2024-04-15T15:19:31.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.959+0000 7f7849ffb640 1 -- 172.21.15.105:0/4219357178 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785000b6c0 con 0x7f78540b6780 2024-04-15T15:19:31.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.959+0000 7f7849ffb640 1 -- 172.21.15.105:0/4219357178 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7850019070 con 0x7f78540b6780 2024-04-15T15:19:31.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.959+0000 7f7849ffb640 1 -- 172.21.15.105:0/4219357178 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785000ba60 con 0x7f78540b6780 2024-04-15T15:19:31.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.959+0000 7f9555d9b640 1 --2- 172.21.15.105:0/651811173 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95580b4330 0x7f9558160be0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f954c00c570 tx=0x7f954c00ca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:31.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.960+0000 7f9546ffd640 1 -- 172.21.15.105:0/651811173 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954c018070 con 0x7f95580b4330 2024-04-15T15:19:31.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.960+0000 7f9546ffd640 1 -- 172.21.15.105:0/651811173 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f954c0074e0 con 0x7f95580b4330 2024-04-15T15:19:31.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.960+0000 7f7bf6814640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bd8009030 0x7f7bf013cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.960+0000 7f28f67fc640 1 -- 172.21.15.105:0/2973168466 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f8017070 con 0x7f29000d0ef0 2024-04-15T15:19:31.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.960+0000 7f28f67fc640 1 -- 172.21.15.105:0/2973168466 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28f800ad40 con 0x7f29000d0ef0 2024-04-15T15:19:31.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.960+0000 7f28f67fc640 1 -- 172.21.15.105:0/2973168466 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f8012620 con 0x7f29000d0ef0 2024-04-15T15:19:31.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.960+0000 7f2907cd9640 1 -- 172.21.15.105:0/2973168466 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f290015f130 con 0x7f29000d0ef0 2024-04-15T15:19:31.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.960+0000 7fec80263640 1 Processor -- start 2024-04-15T15:19:31.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.960+0000 7f785b304640 1 -- 172.21.15.105:0/4219357178 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7854165f70 con 0x7f78540b6780 2024-04-15T15:19:31.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.960+0000 7f9546ffd640 1 -- 172.21.15.105:0/651811173 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954c004a00 con 0x7f95580b4330 2024-04-15T15:19:31.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.960+0000 7fec80263640 1 -- start start 2024-04-15T15:19:31.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.960+0000 7f75817fa640 1 -- 172.21.15.105:0/1498572065 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f757800ccb0 con 0x7f757c0da3e0 2024-04-15T15:19:31.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.961+0000 7f785b304640 1 -- 172.21.15.105:0/4219357178 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78540ca9a0 con 0x7f78540b6780 2024-04-15T15:19:31.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.961+0000 7f7bf6814640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7bf015cca0 0x7f7bf015d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.961+0000 7f2907cd9640 1 -- 172.21.15.105:0/2973168466 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2900161d80 con 0x7f29000d0ef0 2024-04-15T15:19:31.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.961+0000 7f7bf6814640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7bf00abe10 0x7f7bf00ac290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.961+0000 7f7bf6814640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf00ce9d0 con 0x7f7bd8009030 2024-04-15T15:19:31.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.961+0000 7f7bf6814640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf006ec50 con 0x7f7bf00abe10 2024-04-15T15:19:31.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.961+0000 7f7bf6814640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f7bf00ce6a0 con 0x7f7bf015cca0 2024-04-15T15:19:31.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.961+0000 7f7beffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bd8009030 0x7f7bf013cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.961+0000 7f75817fa640 1 --2- 172.21.15.105:0/1498572065 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f756806c9c0 0x7f756806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.961+0000 7f75817fa640 1 -- 172.21.15.105:0/1498572065 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f757808bae0 con 0x7f757c0da3e0 2024-04-15T15:19:31.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.961+0000 7f7bef7fe640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7bf015cca0 0x7f7bf015d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.961+0000 7f7bef7fe640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7bf015cca0 0x7f7bf015d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.105:56866/0 (socket says 172.21.15.105:56866) 2024-04-15T15:19:31.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.961+0000 7f7bef7fe640 1 -- 172.21.15.105:0/1671744519 learned_addr learned my addr 172.21.15.105:0/1671744519 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:31.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.961+0000 7fec80263640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec780703f0 0x7fec78145d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.961+0000 7fec80263640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec7815a7d0 0x7fec78146520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.959+0000 7f6c0daa6640 1 --2- 172.21.15.105:0/3051101491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c08144e00 0x7f6c081533c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.959+0000 7f6c0caa4640 1 -- 172.21.15.105:0/3051101491 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6c0806de50 msgr2=0x7f6c08144000 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-15T15:19:31.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.961+0000 7f6c0caa4640 1 --2- 172.21.15.105:0/3051101491 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6c0806de50 0x7f6c08144000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.961+0000 7f6c0caa4640 1 -- 172.21.15.105:0/3051101491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c08144e00 msgr2=0x7f6c081533c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.961+0000 7fec80263640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec78146a60 0x7fec780b0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.962+0000 7fec80263640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fec78072e30 con 0x7fec780703f0 2024-04-15T15:19:31.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.962+0000 7fec80263640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fec7806f3c0 con 0x7fec7815a7d0 2024-04-15T15:19:31.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.962+0000 7fec80263640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fec78072fb0 con 0x7fec78146a60 2024-04-15T15:19:31.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.962+0000 7fec7d7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec7815a7d0 0x7fec78146520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.962+0000 7f6c0caa4640 1 --2- 172.21.15.105:0/3051101491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c08144e00 0x7f6c081533c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.962+0000 7f6c0caa4640 1 -- 172.21.15.105:0/3051101491 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c0814c1a0 con 0x7f6c0806fee0 2024-04-15T15:19:31.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.962+0000 7f6c0daa6640 1 --2- 172.21.15.105:0/3051101491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c08144e00 0x7f6c081533c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:31.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.962+0000 7fec7d7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec7815a7d0 0x7fec78146520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.105:44966/0 (socket says 172.21.15.105:44966) 2024-04-15T15:19:31.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.962+0000 7fec7d7fe640 1 -- 172.21.15.105:0/2294086990 learned_addr learned my addr 172.21.15.105:0/2294086990 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:31.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.961+0000 7f75817fa640 1 --2- 172.21.15.105:0/1498572065 >> [v2:172.21.15.105:6800/2873040054,v1:172.21.15.105:6801/2873040054] conn(0x7f7568074be0 0x7f7568077000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.961+0000 7f75817fa640 1 -- 172.21.15.105:0/1498572065 --> [v2:172.21.15.105:6800/2873040054,v1:172.21.15.105:6801/2873040054] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f757800bf30 con 0x7f7568074be0 2024-04-15T15:19:31.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.961+0000 7f75817fa640 1 -- 172.21.15.105:0/1498572065 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f757808bed0 con 0x7f757c0da3e0 2024-04-15T15:19:31.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.962+0000 7f758909d640 1 --2- 172.21.15.105:0/1498572065 >> [v2:172.21.15.105:6800/2873040054,v1:172.21.15.105:6801/2873040054] conn(0x7f7568074be0 0x7f7568077000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.962+0000 7fec7f001640 1 --2- 172.21.15.105:0/2294086990 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec78146a60 0x7fec780b0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.962+0000 7fec7dfff640 1 --2- 172.21.15.105:0/2294086990 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec780703f0 0x7fec78145d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.962+0000 7f2907cd9640 1 -- 172.21.15.105:0/2973168466 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f29000b4720 con 0x7f29000d0ef0 2024-04-15T15:19:31.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.962+0000 7f758889c640 1 --2- 172.21.15.105:0/1498572065 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f756806c9c0 0x7f756806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.962+0000 7f7bf4db1640 1 --2- 172.21.15.105:0/1671744519 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7bf00abe10 0x7f7bf00ac290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.962+0000 7f6c0caa4640 1 --2- 172.21.15.105:0/3051101491 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c0806fee0 0x7f6c081448c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6c0403e040 tx=0x7f6c040046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:31.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.962+0000 7f7bef7fe640 1 -- 172.21.15.105:0/1671744519 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7bf00abe10 msgr2=0x7f7bf00ac290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.962+0000 7f7bef7fe640 1 --2- 172.21.15.105:0/1671744519 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7bf00abe10 0x7f7bf00ac290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.959+0000 7fe06b5c8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe05c006a40 0x7fe05c17ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.959+0000 7fe06b5c8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe05c04b3b0 0x7fe05c04e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.960+0000 7fe06b5c8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe05c04f4c0 0x7fe05c039bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.960+0000 7fe069364640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe05c006a40 0x7fe05c17ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.960+0000 7fe069b65640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe05c04f4c0 0x7fe05c039bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.960+0000 7fe069b65640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe05c04f4c0 0x7fe05c039bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39532/0 (socket says 172.21.15.105:39532) 2024-04-15T15:19:31.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.960+0000 7fe068b63640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe05c04b3b0 0x7fe05c04e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.960+0000 7fe068b63640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe05c04b3b0 0x7fe05c04e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.105:44954/0 (socket says 172.21.15.105:44954) 2024-04-15T15:19:31.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.960+0000 7fe069364640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe05c006a40 0x7fe05c17ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.105:56858/0 (socket says 172.21.15.105:56858) 2024-04-15T15:19:31.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.960+0000 7fe069b65640 1 -- 172.21.15.105:0/770634128 learned_addr learned my addr 172.21.15.105:0/770634128 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:31.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.960+0000 7fe06b5c8640 1 -- 172.21.15.105:0/770634128 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe05c050050 con 0x7fe05c04f4c0 2024-04-15T15:19:31.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.960+0000 7fe06b5c8640 1 -- 172.21.15.105:0/770634128 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe05c04b790 con 0x7fe05c04b3b0 2024-04-15T15:19:31.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.960+0000 7fe06b5c8640 1 -- 172.21.15.105:0/770634128 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fe05c054820 con 0x7fe05c006a40 2024-04-15T15:19:31.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.962+0000 7fec7d7fe640 1 -- 172.21.15.105:0/2294086990 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec78146a60 msgr2=0x7fec780b0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.961+0000 7fe069b65640 1 -- 172.21.15.105:0/770634128 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe05c006a40 msgr2=0x7fe05c17ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.961+0000 7fe069b65640 1 --2- 172.21.15.105:0/770634128 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe05c006a40 0x7fe05c17ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.961+0000 7fe069b65640 1 -- 172.21.15.105:0/770634128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe05c04b3b0 msgr2=0x7fe05c04e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.961+0000 7fe069b65640 1 --2- 172.21.15.105:0/770634128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe05c04b3b0 0x7fe05c04e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.961+0000 7fe069b65640 1 -- 172.21.15.105:0/770634128 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe05c04bf40 con 0x7fe05c04f4c0 2024-04-15T15:19:31.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.961+0000 7fe068b63640 1 --2- 172.21.15.105:0/770634128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe05c04b3b0 0x7fe05c04e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:19:31.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.962+0000 7fe069364640 1 --2- 172.21.15.105:0/770634128 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe05c006a40 0x7fe05c17ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:31.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.962+0000 7f758909d640 1 --2- 172.21.15.105:0/1498572065 >> [v2:172.21.15.105:6800/2873040054,v1:172.21.15.105:6801/2873040054] conn(0x7f7568074be0 0x7f7568077000 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:31.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.962+0000 7fec7d7fe640 1 --2- 172.21.15.105:0/2294086990 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec78146a60 0x7fec780b0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.962+0000 7fec7d7fe640 1 -- 172.21.15.105:0/2294086990 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec780703f0 msgr2=0x7fec78145d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.962+0000 7fec7d7fe640 1 --2- 172.21.15.105:0/2294086990 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec780703f0 0x7fec78145d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.962+0000 7fec7d7fe640 1 -- 172.21.15.105:0/2294086990 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec78153f20 con 0x7fec7815a7d0 2024-04-15T15:19:31.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.962+0000 7f7bef7fe640 1 -- 172.21.15.105:0/1671744519 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bd8009030 msgr2=0x7f7bf013cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.962+0000 7f7bef7fe640 1 --2- 172.21.15.105:0/1671744519 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bd8009030 0x7f7bf013cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.962+0000 7f6bfdffb640 1 -- 172.21.15.105:0/3051101491 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c040110d0 con 0x7f6c0806fee0 2024-04-15T15:19:31.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.962+0000 7fec7dfff640 1 --2- 172.21.15.105:0/2294086990 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec780703f0 0x7fec78145d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:19:31.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.962+0000 7f6bfdffb640 1 -- 172.21.15.105:0/3051101491 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c0403cac0 con 0x7f6c0806fee0 2024-04-15T15:19:31.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.962+0000 7f6c0f509640 1 -- 172.21.15.105:0/3051101491 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c080b3c10 con 0x7f6c0806fee0 2024-04-15T15:19:31.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.963+0000 7f6bfdffb640 1 -- 172.21.15.105:0/3051101491 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c04045400 con 0x7f6c0806fee0 2024-04-15T15:19:31.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.962+0000 7f7bef7fe640 1 -- 172.21.15.105:0/1671744519 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bf013f510 con 0x7f7bf015cca0 2024-04-15T15:19:31.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.963+0000 7f955759e640 1 -- 172.21.15.105:0/651811173 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95581615f0 con 0x7f95580b4330 2024-04-15T15:19:31.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.963+0000 7f955759e640 1 -- 172.21.15.105:0/651811173 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9558161bb0 con 0x7f95580b4330 2024-04-15T15:19:31.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.963+0000 7fec7d7fe640 1 --2- 172.21.15.105:0/2294086990 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec7815a7d0 0x7fec78146520 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fec6802e200 tx=0x7fec680046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:31.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.963+0000 7f6c0f509640 1 -- 172.21.15.105:0/3051101491 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c080b41d0 con 0x7f6c0806fee0 2024-04-15T15:19:31.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.963+0000 7f7bef7fe640 1 --2- 172.21.15.105:0/1671744519 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7bf015cca0 0x7f7bf015d120 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7be4009ec0 tx=0x7f7be400f580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:31.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.962+0000 7f78377fe640 1 -- 172.21.15.105:0/4219357178 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7830001030 con 0x7f78540b6780 2024-04-15T15:19:31.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.963+0000 7fe069b65640 1 --2- 172.21.15.105:0/770634128 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe05c04f4c0 0x7fe05c039bd0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe060009870 tx=0x7fe060004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:31.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.963+0000 7fe055ffb640 1 -- 172.21.15.105:0/770634128 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe060048070 con 0x7fe05c04f4c0 2024-04-15T15:19:31.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.963+0000 7fe055ffb640 1 -- 172.21.15.105:0/770634128 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe060030640 con 0x7fe05c04f4c0 2024-04-15T15:19:31.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.963+0000 7fe055ffb640 1 -- 172.21.15.105:0/770634128 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe060034040 con 0x7fe05c04f4c0 2024-04-15T15:19:31.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.963+0000 7fe06b5c8640 1 -- 172.21.15.105:0/770634128 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe05c0386a0 con 0x7fe05c04f4c0 2024-04-15T15:19:31.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.963+0000 7f75817fa640 1 -- 172.21.15.105:0/1498572065 <== osd.0 v2:172.21.15.105:6800/2873040054 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f757800bf30 con 0x7f7568074be0 2024-04-15T15:19:31.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.964+0000 7f758889c640 1 --2- 172.21.15.105:0/1498572065 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f756806c9c0 0x7f756806ee80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f757400c010 tx=0x7f75840b2000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:31.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.964+0000 7fec6e7fc640 1 -- 172.21.15.105:0/2294086990 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec68044070 con 0x7fec7815a7d0 2024-04-15T15:19:31.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.964+0000 7f955759e640 1 -- 172.21.15.105:0/651811173 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9558069150 con 0x7f95580b4330 2024-04-15T15:19:31.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.964+0000 7fec80263640 1 -- 172.21.15.105:0/2294086990 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec7815c5d0 con 0x7fec7815a7d0 2024-04-15T15:19:31.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.964+0000 7fec80263640 1 -- 172.21.15.105:0/2294086990 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec7815cb90 con 0x7fec7815a7d0 2024-04-15T15:19:31.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.964+0000 7fec6e7fc640 1 -- 172.21.15.105:0/2294086990 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec6800a850 con 0x7fec7815a7d0 2024-04-15T15:19:31.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.964+0000 7fec6e7fc640 1 -- 172.21.15.105:0/2294086990 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6800d080 con 0x7fec7815a7d0 2024-04-15T15:19:31.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.963+0000 7f7849ffb640 1 -- 172.21.15.105:0/4219357178 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f785000bc00 con 0x7f78540b6780 2024-04-15T15:19:31.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.964+0000 7f7849ffb640 1 --2- 172.21.15.105:0/4219357178 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f783806c9c0 0x7f783806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.964+0000 7f7849ffb640 1 -- 172.21.15.105:0/4219357178 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f785008c0f0 con 0x7f78540b6780 2024-04-15T15:19:31.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.964+0000 7f7849ffb640 1 --2- 172.21.15.105:0/4219357178 >> [v2:172.21.15.154:6824/1550829005,v1:172.21.15.154:6825/1550829005] conn(0x7f7838074be0 0x7f7838077000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.964+0000 7f7849ffb640 1 -- 172.21.15.105:0/4219357178 --> [v2:172.21.15.154:6824/1550829005,v1:172.21.15.154:6825/1550829005] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f785000fe50 con 0x7f7838074be0 2024-04-15T15:19:31.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.964+0000 7f7849ffb640 1 -- 172.21.15.105:0/4219357178 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f785008c4e0 con 0x7f78540b6780 2024-04-15T15:19:31.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.964+0000 7f785889f640 1 --2- 172.21.15.105:0/4219357178 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f783806c9c0 0x7f783806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.965+0000 7f6bf37fe640 1 -- 172.21.15.105:0/3051101491 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6c08068a40 con 0x7f6c0806fee0 2024-04-15T15:19:31.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.965+0000 7fe06b5c8640 1 -- 172.21.15.105:0/770634128 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe05c038c60 con 0x7fe05c04f4c0 2024-04-15T15:19:31.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.965+0000 7f28f67fc640 1 -- 172.21.15.105:0/2973168466 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f28f801a070 con 0x7f29000d0ef0 2024-04-15T15:19:31.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.965+0000 7f7becff9640 1 -- 172.21.15.105:0/1671744519 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be4015430 con 0x7f7bf015cca0 2024-04-15T15:19:31.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.965+0000 7f7becff9640 1 -- 172.21.15.105:0/1671744519 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7be40155d0 con 0x7f7bf015cca0 2024-04-15T15:19:31.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.965+0000 7f7becff9640 1 -- 172.21.15.105:0/1671744519 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be40159c0 con 0x7f7bf015cca0 2024-04-15T15:19:31.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.965+0000 7f7bf6814640 1 -- 172.21.15.105:0/1671744519 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bf013f7f0 con 0x7f7bf015cca0 2024-04-15T15:19:31.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.965+0000 7f9546ffd640 1 -- 172.21.15.105:0/651811173 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f954c004ba0 con 0x7f95580b4330 2024-04-15T15:19:31.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.965+0000 7f785889f640 1 --2- 172.21.15.105:0/4219357178 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f783806c9c0 0x7f783806ee80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f78541666f0 tx=0x7f7844009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:31.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.966+0000 7f78598a1640 1 --2- 172.21.15.105:0/4219357178 >> [v2:172.21.15.154:6824/1550829005,v1:172.21.15.154:6825/1550829005] conn(0x7f7838074be0 0x7f7838077000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.966+0000 7f6bfdffb640 1 -- 172.21.15.105:0/3051101491 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6c0403b3a0 con 0x7f6c0806fee0 2024-04-15T15:19:31.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.966+0000 7f6bfdffb640 1 --2- 172.21.15.105:0/3051101491 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f6bec06c920 0x7f6bec06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.966+0000 7fec80263640 1 -- 172.21.15.105:0/2294086990 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fec60001030 con 0x7fec7815a7d0 2024-04-15T15:19:31.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.966+0000 7f9546ffd640 1 --2- 172.21.15.105:0/651811173 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f953806c8f0 0x7f953806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.966+0000 7f9546ffd640 1 -- 172.21.15.105:0/651811173 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f954c08af20 con 0x7f95580b4330 2024-04-15T15:19:31.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.966+0000 7f9546ffd640 1 --2- 172.21.15.105:0/651811173 >> [v2:172.21.15.105:6816/2494994479,v1:172.21.15.105:6818/2494994479] conn(0x7f9538074b10 0x7f9538076f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.966+0000 7f9546ffd640 1 -- 172.21.15.105:0/651811173 --> [v2:172.21.15.105:6816/2494994479,v1:172.21.15.105:6818/2494994479] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f95380775e0 con 0x7f9538074b10 2024-04-15T15:19:31.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.967+0000 7fec6e7fc640 1 -- 172.21.15.105:0/2294086990 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fec68047070 con 0x7fec7815a7d0 2024-04-15T15:19:31.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.966+0000 7f9546ffd640 1 -- 172.21.15.105:0/651811173 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f954c0904f0 con 0x7f95580b4330 2024-04-15T15:19:31.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.966+0000 7f28f67fc640 1 --2- 172.21.15.105:0/2973168466 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f28e406c940 0x7f28e406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.966+0000 7f2905274640 1 --2- 172.21.15.105:0/2973168466 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f28e406c940 0x7f28e406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.966+0000 7f955659c640 1 --2- 172.21.15.105:0/651811173 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f953806c8f0 0x7f953806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.967+0000 7f9556d9d640 1 --2- 172.21.15.105:0/651811173 >> [v2:172.21.15.105:6816/2494994479,v1:172.21.15.105:6818/2494994479] conn(0x7f9538074b10 0x7f9538076f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.967+0000 7f28f67fc640 1 -- 172.21.15.105:0/2973168466 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f28f808ba50 con 0x7f29000d0ef0 2024-04-15T15:19:31.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.967+0000 7f28f67fc640 1 --2- 172.21.15.105:0/2973168466 >> [v2:172.21.15.154:6808/3516494456,v1:172.21.15.154:6810/3516494456] conn(0x7f28e4074b60 0x7f28e4076f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.967+0000 7f28f67fc640 1 -- 172.21.15.105:0/2973168466 --> [v2:172.21.15.154:6808/3516494456,v1:172.21.15.154:6810/3516494456] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f28e4077630 con 0x7f28e4074b60 2024-04-15T15:19:31.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.967+0000 7f28f67fc640 1 -- 172.21.15.105:0/2973168466 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f28f80902a0 con 0x7f29000d0ef0 2024-04-15T15:19:31.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.967+0000 7f955659c640 1 --2- 172.21.15.105:0/651811173 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f953806c8f0 0x7f953806edb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9548009b30 tx=0x7f9548009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:31.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.967+0000 7f7bf6814640 1 -- 172.21.15.105:0/1671744519 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bf013fdb0 con 0x7f7bf015cca0 2024-04-15T15:19:31.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.966+0000 7f6bfdffb640 1 -- 172.21.15.105:0/3051101491 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6c040ad600 con 0x7f6c0806fee0 2024-04-15T15:19:31.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.967+0000 7f6bfdffb640 1 --2- 172.21.15.105:0/3051101491 >> [v2:172.21.15.105:6817/1920720186,v1:172.21.15.105:6819/1920720186] conn(0x7f6bec074ac0 0x7f6bec076ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.967+0000 7f6c0d2a5640 1 --2- 172.21.15.105:0/3051101491 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f6bec06c920 0x7f6bec06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.967+0000 7f6bfdffb640 1 -- 172.21.15.105:0/3051101491 --> [v2:172.21.15.105:6817/1920720186,v1:172.21.15.105:6819/1920720186] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6c04011270 con 0x7f6bec074ac0 2024-04-15T15:19:31.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.967+0000 7f6bfdffb640 1 -- 172.21.15.105:0/3051101491 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f6c040b4050 con 0x7f6c0806fee0 2024-04-15T15:19:31.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.967+0000 7f6c0daa6640 1 --2- 172.21.15.105:0/3051101491 >> [v2:172.21.15.105:6817/1920720186,v1:172.21.15.105:6819/1920720186] conn(0x7f6bec074ac0 0x7f6bec076ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.967+0000 7f78598a1640 1 --2- 172.21.15.105:0/4219357178 >> [v2:172.21.15.154:6824/1550829005,v1:172.21.15.154:6825/1550829005] conn(0x7f7838074be0 0x7f7838077000 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:31.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.966+0000 7fe055ffb640 1 -- 172.21.15.105:0/770634128 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe06004b070 con 0x7fe05c04f4c0 2024-04-15T15:19:31.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.967+0000 7fe055ffb640 1 --2- 172.21.15.105:0/770634128 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fe04806c970 0x7fe04806ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.967+0000 7fe055ffb640 1 -- 172.21.15.105:0/770634128 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe0600ad8c0 con 0x7fe05c04f4c0 2024-04-15T15:19:31.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.967+0000 7fe06b5c8640 1 -- 172.21.15.105:0/770634128 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe05c0057d0 con 0x7fe05c04f4c0 2024-04-15T15:19:31.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.967+0000 7fe055ffb640 1 --2- 172.21.15.105:0/770634128 >> [v2:172.21.15.154:6809/3920089903,v1:172.21.15.154:6811/3920089903] conn(0x7fe048074b10 0x7fe048076f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.967+0000 7fe055ffb640 1 -- 172.21.15.105:0/770634128 --> [v2:172.21.15.154:6809/3920089903,v1:172.21.15.154:6811/3920089903] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe06002de50 con 0x7fe048074b10 2024-04-15T15:19:31.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.967+0000 7fe069364640 1 --2- 172.21.15.105:0/770634128 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fe04806c970 0x7fe04806ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.967+0000 7fe068b63640 1 --2- 172.21.15.105:0/770634128 >> [v2:172.21.15.154:6809/3920089903,v1:172.21.15.154:6811/3920089903] conn(0x7fe048074b10 0x7fe048076f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.968+0000 7fec6e7fc640 1 --2- 172.21.15.105:0/2294086990 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fec5c06c9c0 0x7fec5c06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.968+0000 7f9556d9d640 1 --2- 172.21.15.105:0/651811173 >> [v2:172.21.15.105:6816/2494994479,v1:172.21.15.105:6818/2494994479] conn(0x7f9538074b10 0x7f9538076f30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:31.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.968+0000 7fec6e7fc640 1 -- 172.21.15.105:0/2294086990 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fec680ad110 con 0x7fec7815a7d0 2024-04-15T15:19:31.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.968+0000 7f6c0daa6640 1 --2- 172.21.15.105:0/3051101491 >> [v2:172.21.15.105:6817/1920720186,v1:172.21.15.105:6819/1920720186] conn(0x7f6bec074ac0 0x7f6bec076ee0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:31.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.968+0000 7fe055ffb640 1 -- 172.21.15.105:0/770634128 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fe060077880 con 0x7fe05c04f4c0 2024-04-15T15:19:31.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.968+0000 7fec6e7fc640 1 --2- 172.21.15.105:0/2294086990 >> [v2:172.21.15.154:6800/540367751,v1:172.21.15.154:6801/540367751] conn(0x7fec5c074be0 0x7fec5c077000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.968+0000 7f7849ffb640 1 -- 172.21.15.105:0/4219357178 <== osd.6 v2:172.21.15.154:6824/1550829005 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f785000fe50 con 0x7f7838074be0 2024-04-15T15:19:31.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.968+0000 7fec6e7fc640 1 -- 172.21.15.105:0/2294086990 --> [v2:172.21.15.154:6800/540367751,v1:172.21.15.154:6801/540367751] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fec68001d30 con 0x7fec5c074be0 2024-04-15T15:19:31.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.968+0000 7fec6e7fc640 1 -- 172.21.15.105:0/2294086990 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fec680b4050 con 0x7fec7815a7d0 2024-04-15T15:19:31.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.968+0000 7fe068b63640 1 --2- 172.21.15.105:0/770634128 >> [v2:172.21.15.154:6809/3920089903,v1:172.21.15.154:6811/3920089903] conn(0x7fe048074b10 0x7fe048076f30 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:31.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.969+0000 7f9546ffd640 1 -- 172.21.15.105:0/651811173 <== osd.1 v2:172.21.15.105:6816/2494994479 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f95380775e0 con 0x7f9538074b10 2024-04-15T15:19:31.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.969+0000 7f7bf6814640 1 -- 172.21.15.105:0/1671744519 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7bf0068a40 con 0x7f7bf015cca0 2024-04-15T15:19:31.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.969+0000 7fec7dfff640 1 --2- 172.21.15.105:0/2294086990 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fec5c06c9c0 0x7fec5c06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.969+0000 7fec7f001640 1 --2- 172.21.15.105:0/2294086990 >> [v2:172.21.15.154:6800/540367751,v1:172.21.15.154:6801/540367751] conn(0x7fec5c074be0 0x7fec5c077000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.969+0000 7f2906276640 1 --2- 172.21.15.105:0/2973168466 >> [v2:172.21.15.154:6808/3516494456,v1:172.21.15.154:6810/3516494456] conn(0x7f28e4074b60 0x7f28e4076f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.969+0000 7f6bfdffb640 1 -- 172.21.15.105:0/3051101491 <== osd.3 v2:172.21.15.105:6817/1920720186 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6c04011270 con 0x7f6bec074ac0 2024-04-15T15:19:31.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.969+0000 7f6c0d2a5640 1 --2- 172.21.15.105:0/3051101491 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f6bec06c920 0x7f6bec06ede0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6bf8003c50 tx=0x7f6bf80079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:31.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.969+0000 7fe055ffb640 1 -- 172.21.15.105:0/770634128 <== osd.7 v2:172.21.15.154:6809/3920089903 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe06002de50 con 0x7fe048074b10 2024-04-15T15:19:31.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.969+0000 7fec7dfff640 1 --2- 172.21.15.105:0/2294086990 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fec5c06c9c0 0x7fec5c06ee80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fec70000b60 tx=0x7fec70009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:31.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.969+0000 7fec7f001640 1 --2- 172.21.15.105:0/2294086990 >> [v2:172.21.15.154:6800/540367751,v1:172.21.15.154:6801/540367751] conn(0x7fec5c074be0 0x7fec5c077000 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:31.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.969+0000 7f2905274640 1 --2- 172.21.15.105:0/2973168466 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f28e406c940 0x7f28e406ee00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f28f0035000 tx=0x7f28f0011000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:31.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.969+0000 7f2906276640 1 --2- 172.21.15.105:0/2973168466 >> [v2:172.21.15.154:6808/3516494456,v1:172.21.15.154:6810/3516494456] conn(0x7f28e4074b60 0x7f28e4076f80 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:31.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.970+0000 7fe069364640 1 --2- 172.21.15.105:0/770634128 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fe04806c970 0x7fe04806ee30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe06402db70 tx=0x7fe06406b7d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:31.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.970+0000 7f7becff9640 1 -- 172.21.15.105:0/1671744519 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7be4015b60 con 0x7f7bf015cca0 2024-04-15T15:19:31.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.970+0000 7f28f67fc640 1 -- 172.21.15.105:0/2973168466 <== osd.4 v2:172.21.15.154:6808/3516494456 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f28e4077630 con 0x7f28e4074b60 2024-04-15T15:19:31.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.970+0000 7f7becff9640 1 --2- 172.21.15.105:0/1671744519 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f7bdc06c940 0x7f7bdc06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.971+0000 7f7becff9640 1 -- 172.21.15.105:0/1671744519 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7be408c280 con 0x7f7bf015cca0 2024-04-15T15:19:31.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.971+0000 7f7becff9640 1 --2- 172.21.15.105:0/1671744519 >> [v2:172.21.15.105:6808/3798408699,v1:172.21.15.105:6809/3798408699] conn(0x7f7bdc074b60 0x7f7bdc076f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.971+0000 7f7beffff640 1 --2- 172.21.15.105:0/1671744519 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f7bdc06c940 0x7f7bdc06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.971+0000 7f7becff9640 1 -- 172.21.15.105:0/1671744519 --> [v2:172.21.15.105:6808/3798408699,v1:172.21.15.105:6809/3798408699] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7be4015e50 con 0x7f7bdc074b60 2024-04-15T15:19:31.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.971+0000 7f7becff9640 1 -- 172.21.15.105:0/1671744519 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f7be4091060 con 0x7f7bf015cca0 2024-04-15T15:19:31.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.970+0000 7fec6e7fc640 1 -- 172.21.15.105:0/2294086990 <== osd.5 v2:172.21.15.154:6800/540367751 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fec68001d30 con 0x7fec5c074be0 2024-04-15T15:19:31.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.971+0000 7f7beffff640 1 --2- 172.21.15.105:0/1671744519 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f7bdc06c940 0x7f7bdc06ee00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7be00099a0 tx=0x7f7be0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:31.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.975+0000 7f7bf4db1640 1 --2- 172.21.15.105:0/1671744519 >> [v2:172.21.15.105:6808/3798408699,v1:172.21.15.105:6809/3798408699] conn(0x7f7bdc074b60 0x7f7bdc076f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.975+0000 7f7bf4db1640 1 --2- 172.21.15.105:0/1671744519 >> [v2:172.21.15.105:6808/3798408699,v1:172.21.15.105:6809/3798408699] conn(0x7f7bdc074b60 0x7f7bdc076f80 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:31.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.976+0000 7f7becff9640 1 -- 172.21.15.105:0/1671744519 <== osd.2 v2:172.21.15.105:6808/3798408699 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7be4015e50 con 0x7f7bdc074b60 2024-04-15T15:19:31.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.995+0000 7f758989e640 1 -- 172.21.15.105:0/1498572065 --> [v2:172.21.15.105:6800/2873040054,v1:172.21.15.105:6801/2873040054] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f757c052b90 con 0x7f7568074be0 2024-04-15T15:19:31.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.996+0000 7f75817fa640 1 -- 172.21.15.105:0/1498572065 <== osd.0 v2:172.21.15.105:6800/2873040054 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f757c052b90 con 0x7f7568074be0 2024-04-15T15:19:31.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.996+0000 7f758989e640 1 -- 172.21.15.105:0/1498572065 >> [v2:172.21.15.105:6800/2873040054,v1:172.21.15.105:6801/2873040054] conn(0x7f7568074be0 msgr2=0x7f7568077000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.996+0000 7f758989e640 1 --2- 172.21.15.105:0/1498572065 >> [v2:172.21.15.105:6800/2873040054,v1:172.21.15.105:6801/2873040054] conn(0x7f7568074be0 0x7f7568077000 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.996+0000 7f758989e640 1 -- 172.21.15.105:0/1498572065 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f756806c9c0 msgr2=0x7f756806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.996+0000 7f758989e640 1 --2- 172.21.15.105:0/1498572065 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f756806c9c0 0x7f756806ee80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f757400c010 tx=0x7f75840b2000 comp rx=0 tx=0).stop 2024-04-15T15:19:31.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.996+0000 7f758989e640 1 -- 172.21.15.105:0/1498572065 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f757c0da3e0 msgr2=0x7f757c052270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.996+0000 7f758989e640 1 --2- 172.21.15.105:0/1498572065 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f757c0da3e0 0x7f757c052270 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7578001050 tx=0x7f757800ec10 comp rx=0 tx=0).stop 2024-04-15T15:19:31.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.997+0000 7f758909d640 1 -- 172.21.15.105:0/1498572065 reap_dead start 2024-04-15T15:19:31.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.997+0000 7f758989e640 1 -- 172.21.15.105:0/1498572065 shutdown_connections 2024-04-15T15:19:31.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.997+0000 7f758989e640 1 -- 172.21.15.105:0/1498572065 >> 172.21.15.105:0/1498572065 conn(0x7f757c01ef40 msgr2=0x7f757c00b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:31.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.997+0000 7f758989e640 1 -- 172.21.15.105:0/1498572065 shutdown_connections 2024-04-15T15:19:31.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.997+0000 7f758989e640 1 -- 172.21.15.105:0/1498572065 wait complete. 2024-04-15T15:19:31.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.998+0000 7f2907cd9640 1 -- 172.21.15.105:0/2973168466 --> [v2:172.21.15.154:6808/3516494456,v1:172.21.15.154:6810/3516494456] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2900069150 con 0x7f28e4074b60 2024-04-15T15:19:31.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.998+0000 7f28f67fc640 1 -- 172.21.15.105:0/2973168466 <== osd.4 v2:172.21.15.154:6808/3516494456 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2900069150 con 0x7f28e4074b60 2024-04-15T15:19:31.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.999+0000 7f78377fe640 1 -- 172.21.15.105:0/4219357178 --> [v2:172.21.15.154:6824/1550829005,v1:172.21.15.154:6825/1550829005] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7830002dd0 con 0x7f7838074be0 2024-04-15T15:19:31.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.999+0000 7f28e3fff640 1 -- 172.21.15.105:0/2973168466 >> [v2:172.21.15.154:6808/3516494456,v1:172.21.15.154:6810/3516494456] conn(0x7f28e4074b60 msgr2=0x7f28e4076f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:30.999+0000 7f28e3fff640 1 --2- 172.21.15.105:0/2973168466 >> [v2:172.21.15.154:6808/3516494456,v1:172.21.15.154:6810/3516494456] conn(0x7f28e4074b60 0x7f28e4076f80 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.000+0000 7f28e3fff640 1 -- 172.21.15.105:0/2973168466 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f28e406c940 msgr2=0x7f28e406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.000+0000 7f28e3fff640 1 --2- 172.21.15.105:0/2973168466 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f28e406c940 0x7f28e406ee00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f28f0035000 tx=0x7f28f0011000 comp rx=0 tx=0).stop 2024-04-15T15:19:31.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.000+0000 7f28e3fff640 1 -- 172.21.15.105:0/2973168466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29000d0ef0 msgr2=0x7f29000b41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.000+0000 7f28e3fff640 1 --2- 172.21.15.105:0/2973168466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29000d0ef0 0x7f29000b41e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f28f800cb50 tx=0x7f28f800a430 comp rx=0 tx=0).stop 2024-04-15T15:19:31.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.000+0000 7f2906276640 1 -- 172.21.15.105:0/2973168466 reap_dead start 2024-04-15T15:19:31.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.000+0000 7f7849ffb640 1 -- 172.21.15.105:0/4219357178 <== osd.6 v2:172.21.15.154:6824/1550829005 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7830002dd0 con 0x7f7838074be0 2024-04-15T15:19:31.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.000+0000 7f78377fe640 1 -- 172.21.15.105:0/4219357178 >> [v2:172.21.15.154:6824/1550829005,v1:172.21.15.154:6825/1550829005] conn(0x7f7838074be0 msgr2=0x7f7838077000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.000+0000 7f78377fe640 1 --2- 172.21.15.105:0/4219357178 >> [v2:172.21.15.154:6824/1550829005,v1:172.21.15.154:6825/1550829005] conn(0x7f7838074be0 0x7f7838077000 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.001+0000 7f28e3fff640 1 -- 172.21.15.105:0/2973168466 shutdown_connections 2024-04-15T15:19:31.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.001+0000 7f28e3fff640 1 -- 172.21.15.105:0/2973168466 >> 172.21.15.105:0/2973168466 conn(0x7f2900067720 msgr2=0x7f2900067b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:31.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.001+0000 7f78377fe640 1 -- 172.21.15.105:0/4219357178 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f783806c9c0 msgr2=0x7f783806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.001+0000 7f78377fe640 1 --2- 172.21.15.105:0/4219357178 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f783806c9c0 0x7f783806ee80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f78541666f0 tx=0x7f7844009240 comp rx=0 tx=0).stop 2024-04-15T15:19:31.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.001+0000 7f78377fe640 1 -- 172.21.15.105:0/4219357178 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78540b6780 msgr2=0x7f78540ca460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.001+0000 7f78377fe640 1 --2- 172.21.15.105:0/4219357178 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78540b6780 0x7f78540ca460 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f785000d800 tx=0x7f785000dcc0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.001+0000 7f78598a1640 1 -- 172.21.15.105:0/4219357178 reap_dead start 2024-04-15T15:19:31.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.001+0000 7f955759e640 1 -- 172.21.15.105:0/651811173 --> [v2:172.21.15.105:6816/2494994479,v1:172.21.15.105:6818/2494994479] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f95580ae4b0 con 0x7f9538074b10 2024-04-15T15:19:31.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.001+0000 7f28e3fff640 1 -- 172.21.15.105:0/2973168466 shutdown_connections 2024-04-15T15:19:31.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.001+0000 7f28e3fff640 1 -- 172.21.15.105:0/2973168466 wait complete. 2024-04-15T15:19:31.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.001+0000 7f78377fe640 1 -- 172.21.15.105:0/4219357178 shutdown_connections 2024-04-15T15:19:31.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.001+0000 7f78377fe640 1 -- 172.21.15.105:0/4219357178 >> 172.21.15.105:0/4219357178 conn(0x7f78540d1240 msgr2=0x7f78540b4120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:31.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.002+0000 7fe06b5c8640 1 -- 172.21.15.105:0/770634128 --> [v2:172.21.15.154:6809/3920089903,v1:172.21.15.154:6811/3920089903] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe05c0399b0 con 0x7fe048074b10 2024-04-15T15:19:31.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.002+0000 7f9546ffd640 1 -- 172.21.15.105:0/651811173 <== osd.1 v2:172.21.15.105:6816/2494994479 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f95580ae4b0 con 0x7f9538074b10 2024-04-15T15:19:31.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.002+0000 7fec80263640 1 -- 172.21.15.105:0/2294086990 --> [v2:172.21.15.154:6800/540367751,v1:172.21.15.154:6801/540367751] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fec60002dd0 con 0x7fec5c074be0 2024-04-15T15:19:31.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.002+0000 7f78377fe640 1 -- 172.21.15.105:0/4219357178 shutdown_connections 2024-04-15T15:19:31.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.002+0000 7f78377fe640 1 -- 172.21.15.105:0/4219357178 wait complete. 2024-04-15T15:19:31.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.002+0000 7f955759e640 1 -- 172.21.15.105:0/651811173 >> [v2:172.21.15.105:6816/2494994479,v1:172.21.15.105:6818/2494994479] conn(0x7f9538074b10 msgr2=0x7f9538076f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.002+0000 7f955759e640 1 --2- 172.21.15.105:0/651811173 >> [v2:172.21.15.105:6816/2494994479,v1:172.21.15.105:6818/2494994479] conn(0x7f9538074b10 0x7f9538076f30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.002+0000 7fec6e7fc640 1 -- 172.21.15.105:0/2294086990 <== osd.5 v2:172.21.15.154:6800/540367751 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fec60002dd0 con 0x7fec5c074be0 2024-04-15T15:19:31.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.002+0000 7fe055ffb640 1 -- 172.21.15.105:0/770634128 <== osd.7 v2:172.21.15.154:6809/3920089903 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe05c0399b0 con 0x7fe048074b10 2024-04-15T15:19:31.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.002+0000 7f955759e640 1 -- 172.21.15.105:0/651811173 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f953806c8f0 msgr2=0x7f953806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.003+0000 7f955759e640 1 --2- 172.21.15.105:0/651811173 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f953806c8f0 0x7f953806edb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9548009b30 tx=0x7f9548009240 comp rx=0 tx=0).stop 2024-04-15T15:19:31.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.003+0000 7f955759e640 1 -- 172.21.15.105:0/651811173 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95580b4330 msgr2=0x7f9558160be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.003+0000 7f955759e640 1 --2- 172.21.15.105:0/651811173 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95580b4330 0x7f9558160be0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f954c00c570 tx=0x7f954c00ca30 comp rx=0 tx=0).stop 2024-04-15T15:19:31.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.003+0000 7f9556d9d640 1 -- 172.21.15.105:0/651811173 reap_dead start 2024-04-15T15:19:31.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.003+0000 7f6bf37fe640 1 -- 172.21.15.105:0/3051101491 --> [v2:172.21.15.105:6817/1920720186,v1:172.21.15.105:6819/1920720186] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6c080ceb00 con 0x7f6bec074ac0 2024-04-15T15:19:31.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.003+0000 7f955759e640 1 -- 172.21.15.105:0/651811173 shutdown_connections 2024-04-15T15:19:31.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.003+0000 7f955759e640 1 -- 172.21.15.105:0/651811173 >> 172.21.15.105:0/651811173 conn(0x7f95580b67e0 msgr2=0x7f955815a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:31.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.003+0000 7f955759e640 1 -- 172.21.15.105:0/651811173 shutdown_connections 2024-04-15T15:19:31.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.003+0000 7fec80263640 1 -- 172.21.15.105:0/2294086990 >> [v2:172.21.15.154:6800/540367751,v1:172.21.15.154:6801/540367751] conn(0x7fec5c074be0 msgr2=0x7fec5c077000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.003+0000 7fec80263640 1 --2- 172.21.15.105:0/2294086990 >> [v2:172.21.15.154:6800/540367751,v1:172.21.15.154:6801/540367751] conn(0x7fec5c074be0 0x7fec5c077000 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.003+0000 7f955759e640 1 -- 172.21.15.105:0/651811173 wait complete. 2024-04-15T15:19:31.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.003+0000 7fec80263640 1 -- 172.21.15.105:0/2294086990 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fec5c06c9c0 msgr2=0x7fec5c06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.003+0000 7fec80263640 1 --2- 172.21.15.105:0/2294086990 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fec5c06c9c0 0x7fec5c06ee80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fec70000b60 tx=0x7fec70009040 comp rx=0 tx=0).stop 2024-04-15T15:19:31.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.003+0000 7fec80263640 1 -- 172.21.15.105:0/2294086990 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec7815a7d0 msgr2=0x7fec78146520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.003+0000 7fec80263640 1 --2- 172.21.15.105:0/2294086990 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec7815a7d0 0x7fec78146520 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fec6802e200 tx=0x7fec680046c0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.003+0000 7fec7f001640 1 -- 172.21.15.105:0/2294086990 reap_dead start 2024-04-15T15:19:31.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.003+0000 7fec80263640 1 -- 172.21.15.105:0/2294086990 shutdown_connections 2024-04-15T15:19:31.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.003+0000 7fec80263640 1 -- 172.21.15.105:0/2294086990 >> 172.21.15.105:0/2294086990 conn(0x7fec780b2170 msgr2=0x7fec780aed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:31.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.003+0000 7fe06b5c8640 1 -- 172.21.15.105:0/770634128 >> [v2:172.21.15.154:6809/3920089903,v1:172.21.15.154:6811/3920089903] conn(0x7fe048074b10 msgr2=0x7fe048076f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.003+0000 7fe06b5c8640 1 --2- 172.21.15.105:0/770634128 >> [v2:172.21.15.154:6809/3920089903,v1:172.21.15.154:6811/3920089903] conn(0x7fe048074b10 0x7fe048076f30 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.004+0000 7fe06b5c8640 1 -- 172.21.15.105:0/770634128 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fe04806c970 msgr2=0x7fe04806ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.004+0000 7fe06b5c8640 1 --2- 172.21.15.105:0/770634128 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fe04806c970 0x7fe04806ee30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe06402db70 tx=0x7fe06406b7d0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.004+0000 7fe06b5c8640 1 -- 172.21.15.105:0/770634128 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe05c04f4c0 msgr2=0x7fe05c039bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.004+0000 7fe06b5c8640 1 --2- 172.21.15.105:0/770634128 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe05c04f4c0 0x7fe05c039bd0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe060009870 tx=0x7fe060004730 comp rx=0 tx=0).stop 2024-04-15T15:19:31.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.003+0000 7fec80263640 1 -- 172.21.15.105:0/2294086990 shutdown_connections 2024-04-15T15:19:31.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.004+0000 7fec80263640 1 -- 172.21.15.105:0/2294086990 wait complete. 2024-04-15T15:19:31.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.004+0000 7fe069b65640 1 -- 172.21.15.105:0/770634128 reap_dead start 2024-04-15T15:19:31.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.004+0000 7fe06b5c8640 1 -- 172.21.15.105:0/770634128 shutdown_connections 2024-04-15T15:19:31.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.004+0000 7fe06b5c8640 1 -- 172.21.15.105:0/770634128 >> 172.21.15.105:0/770634128 conn(0x7fe05c00a3f0 msgr2=0x7fe05c0e5c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:31.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.004+0000 7f6bfdffb640 1 -- 172.21.15.105:0/3051101491 <== osd.3 v2:172.21.15.105:6817/1920720186 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6c080ceb00 con 0x7f6bec074ac0 2024-04-15T15:19:31.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.004+0000 7fe06b5c8640 1 -- 172.21.15.105:0/770634128 shutdown_connections 2024-04-15T15:19:31.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.005+0000 7fe06b5c8640 1 -- 172.21.15.105:0/770634128 wait complete. 2024-04-15T15:19:31.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.005+0000 7f6bf37fe640 1 -- 172.21.15.105:0/3051101491 >> [v2:172.21.15.105:6817/1920720186,v1:172.21.15.105:6819/1920720186] conn(0x7f6bec074ac0 msgr2=0x7f6bec076ee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.005+0000 7f6bf37fe640 1 --2- 172.21.15.105:0/3051101491 >> [v2:172.21.15.105:6817/1920720186,v1:172.21.15.105:6819/1920720186] conn(0x7f6bec074ac0 0x7f6bec076ee0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.007+0000 7f6bf37fe640 1 -- 172.21.15.105:0/3051101491 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f6bec06c920 msgr2=0x7f6bec06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.007+0000 7f6bf37fe640 1 --2- 172.21.15.105:0/3051101491 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f6bec06c920 0x7f6bec06ede0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6bf8003c50 tx=0x7f6bf80079b0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.007+0000 7f6bf37fe640 1 -- 172.21.15.105:0/3051101491 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c0806fee0 msgr2=0x7f6c081448c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.007+0000 7f6bf37fe640 1 --2- 172.21.15.105:0/3051101491 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c0806fee0 0x7f6c081448c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6c0403e040 tx=0x7f6c040046c0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.007+0000 7f6c0daa6640 1 -- 172.21.15.105:0/3051101491 reap_dead start 2024-04-15T15:19:31.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.007+0000 7f6bf37fe640 1 -- 172.21.15.105:0/3051101491 shutdown_connections 2024-04-15T15:19:31.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.007+0000 7f6bf37fe640 1 -- 172.21.15.105:0/3051101491 >> 172.21.15.105:0/3051101491 conn(0x7f6c08067b70 msgr2=0x7f6c08151c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:31.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.007+0000 7f6bf37fe640 1 -- 172.21.15.105:0/3051101491 shutdown_connections 2024-04-15T15:19:31.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.007+0000 7f6bf37fe640 1 -- 172.21.15.105:0/3051101491 wait complete. 2024-04-15T15:19:31.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.008+0000 7f7bf6814640 1 -- 172.21.15.105:0/1671744519 --> [v2:172.21.15.105:6808/3798408699,v1:172.21.15.105:6809/3798408699] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7bf006f410 con 0x7f7bdc074b60 2024-04-15T15:19:31.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.008+0000 7f7becff9640 1 -- 172.21.15.105:0/1671744519 <== osd.2 v2:172.21.15.105:6808/3798408699 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7bf006f410 con 0x7f7bdc074b60 2024-04-15T15:19:31.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.009+0000 7f7bf6814640 1 -- 172.21.15.105:0/1671744519 >> [v2:172.21.15.105:6808/3798408699,v1:172.21.15.105:6809/3798408699] conn(0x7f7bdc074b60 msgr2=0x7f7bdc076f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.009+0000 7f7bf6814640 1 --2- 172.21.15.105:0/1671744519 >> [v2:172.21.15.105:6808/3798408699,v1:172.21.15.105:6809/3798408699] conn(0x7f7bdc074b60 0x7f7bdc076f80 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.010+0000 7f7bf6814640 1 -- 172.21.15.105:0/1671744519 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f7bdc06c940 msgr2=0x7f7bdc06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.010+0000 7f7bf6814640 1 --2- 172.21.15.105:0/1671744519 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f7bdc06c940 0x7f7bdc06ee00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7be00099a0 tx=0x7f7be0008040 comp rx=0 tx=0).stop 2024-04-15T15:19:31.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.010+0000 7f7bf6814640 1 -- 172.21.15.105:0/1671744519 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7bf015cca0 msgr2=0x7f7bf015d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.010+0000 7f7bf6814640 1 --2- 172.21.15.105:0/1671744519 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7bf015cca0 0x7f7bf015d120 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7be4009ec0 tx=0x7f7be400f580 comp rx=0 tx=0).stop 2024-04-15T15:19:31.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.010+0000 7f7bf4db1640 1 -- 172.21.15.105:0/1671744519 reap_dead start 2024-04-15T15:19:31.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.010+0000 7f7bf6814640 1 -- 172.21.15.105:0/1671744519 shutdown_connections 2024-04-15T15:19:31.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.010+0000 7f7bf6814640 1 -- 172.21.15.105:0/1671744519 >> 172.21.15.105:0/1671744519 conn(0x7f7bf0067e10 msgr2=0x7f7bf00b54e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:31.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.011+0000 7f7bf6814640 1 -- 172.21.15.105:0/1671744519 shutdown_connections 2024-04-15T15:19:31.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.011+0000 7f7bf6814640 1 -- 172.21.15.105:0/1671744519 wait complete. 2024-04-15T15:19:31.063 INFO:teuthology.orchestra.run.smithi105.stdout:55834574855 2024-04-15T15:19:31.063 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-15T15:19:31.064 INFO:teuthology.orchestra.run.smithi105.stdout:55834574854 2024-04-15T15:19:31.064 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-15T15:19:31.065 INFO:teuthology.orchestra.run.smithi105.stdout:55834574856 2024-04-15T15:19:31.065 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-15T15:19:31.066 INFO:teuthology.orchestra.run.smithi105.stdout:55834574855 2024-04-15T15:19:31.066 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-15T15:19:31.067 INFO:teuthology.orchestra.run.smithi105.stdout:55834574855 2024-04-15T15:19:31.067 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-15T15:19:31.068 INFO:teuthology.orchestra.run.smithi105.stdout:55834574854 2024-04-15T15:19:31.068 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-15T15:19:31.068 INFO:teuthology.orchestra.run.smithi105.stdout:55834574855 2024-04-15T15:19:31.069 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-15T15:19:31.069 INFO:teuthology.orchestra.run.smithi105.stdout:55834574856 2024-04-15T15:19:31.070 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-15T15:19:31.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.275+0000 7f2f5f59e640 1 Processor -- start 2024-04-15T15:19:31.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.275+0000 7f2f5f59e640 1 -- start start 2024-04-15T15:19:31.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.277+0000 7f2f5f59e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2f600d1b20 0x7f2f600b52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.277+0000 7f2f5f59e640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f6006d580 con 0x7f2f600d1f60 2024-04-15T15:19:31.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.277+0000 7f2f5f59e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f600cfae0 con 0x7f2f601663f0 2024-04-15T15:19:31.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.277+0000 7f2f5f59e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f2f6006d820 con 0x7f2f600d1b20 2024-04-15T15:19:31.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.277+0000 7f62ca1d1640 1 Processor -- start 2024-04-15T15:19:31.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.277+0000 7f7f8c152640 1 Processor -- start 2024-04-15T15:19:31.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.278+0000 7f7f8c152640 1 -- start start 2024-04-15T15:19:31.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.277+0000 7f2f5ed9d640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f2f600d1f60 0x7f2f600b5820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:40984/0 (socket says 172.21.15.105:40984) 2024-04-15T15:19:31.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.277+0000 7f2f5ed9d640 1 -- 172.21.15.105:0/2937597495 learned_addr learned my addr 172.21.15.105:0/2937597495 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:19:31.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.277+0000 7f2f5dd9b640 1 --2- 172.21.15.105:0/2937597495 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2f600d1b20 0x7f2f600b52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.278+0000 7f2f5d59a640 1 -- 172.21.15.105:0/2937597495 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1297730011 0 0) 0x7f2f6006d580 con 0x7f2f600d1f60 2024-04-15T15:19:31.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.278+0000 7f62ca1d1640 1 -- start start 2024-04-15T15:19:31.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.278+0000 7f2f5d59a640 1 -- 172.21.15.105:0/2937597495 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2f48003520 con 0x7f2f600d1f60 2024-04-15T15:19:31.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.278+0000 7f43ae84a640 1 Processor -- start 2024-04-15T15:19:31.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.278+0000 7f43ae84a640 1 -- start start 2024-04-15T15:19:31.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.278+0000 7f2f5d59a640 1 -- 172.21.15.105:0/2937597495 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1325287583 0 0) 0x7f2f48003520 con 0x7f2f600d1f60 2024-04-15T15:19:31.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.278+0000 7f2f5d59a640 1 -- 172.21.15.105:0/2937597495 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2f600d1b20 msgr2=0x7f2f600b52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:31.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.278+0000 7f2f5d59a640 1 --2- 172.21.15.105:0/2937597495 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2f600d1b20 0x7f2f600b52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.278+0000 7f2f5d59a640 1 -- 172.21.15.105:0/2937597495 >> v1:172.21.15.154:6789/0 conn(0x7f2f601663f0 legacy=0x7f2f6006e020 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-15T15:19:31.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.278+0000 7f2f5d59a640 1 -- 172.21.15.105:0/2937597495 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f6006db00 con 0x7f2f600d1f60 2024-04-15T15:19:31.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.278+0000 7f1f0ee2a640 1 Processor -- start 2024-04-15T15:19:31.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.278+0000 7f1f0ee2a640 1 -- start start 2024-04-15T15:19:31.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.278+0000 7f2f5d59a640 1 -- 172.21.15.105:0/2937597495 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7f2f580019f0 con 0x7f2f600d1f60 2024-04-15T15:19:31.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.279+0000 7f2f5d59a640 1 -- 172.21.15.105:0/2937597495 >> v1:172.21.15.105:6789/0 conn(0x7f2f600d1f60 legacy=0x7f2f600b5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.279+0000 7f2f5d59a640 1 --2- 172.21.15.105:0/2937597495 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2f600d1b20 0x7f2f48003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.279+0000 7f2f5d59a640 1 --2- 172.21.15.105:0/2937597495 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f480040a0 0x7f2f48004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.279+0000 7f2f5d59a640 1 --2- 172.21.15.105:0/2937597495 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f48004d90 0x7f2f480051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.279+0000 7f7f8c152640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7f8415acc0 0x7f7f840b4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.279+0000 7f62c91cf640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f62bc0df6f0 0x7f62bc053400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:40996/0 (socket says 172.21.15.105:40996) 2024-04-15T15:19:31.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.279+0000 7f2f5e59c640 1 --2- 172.21.15.105:0/2937597495 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2f600d1b20 0x7f2f48003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.279+0000 7f62c91cf640 1 -- 172.21.15.105:0/2884183583 learned_addr learned my addr 172.21.15.105:0/2884183583 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:19:31.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.279+0000 7f1f0ee2a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f0815e9d0 0x7f1f080d3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.279+0000 7f1f0ee2a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1f08163130 0x7f1f080cf820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.279+0000 7f43ae84a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f43a00085d0 0x7f43a00dfef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.279+0000 7f1a7a73d640 1 Processor -- start 2024-04-15T15:19:31.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.279+0000 7f2f5dd9b640 1 --2- 172.21.15.105:0/2937597495 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f480040a0 0x7f2f48004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.279+0000 7f1f0ee2a640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f0815edb0 con 0x7f1f080d0ef0 2024-04-15T15:19:31.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.279+0000 7f1f0ee2a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f1f080cef90 con 0x7f1f08163130 2024-04-15T15:19:31.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.279+0000 7f1f0ee2a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f1f080ba280 con 0x7f1f0815e9d0 2024-04-15T15:19:31.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.279+0000 7f62ca1d1640 1 --2- 172.21.15.105:0/2884183583 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62bc0507a0 0x7f62bc054cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.279+0000 7f2f5d59a640 1 -- 172.21.15.105:0/2937597495 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6006d820 con 0x7f2f480040a0 2024-04-15T15:19:31.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.280+0000 7f2f5d59a640 1 -- 172.21.15.105:0/2937597495 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4800bc00 con 0x7f2f48004d90 2024-04-15T15:19:31.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.280+0000 7f1f07fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1f08163130 0x7f1f080cf820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.279+0000 7f43ae84a640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f43a00e42d0 0x7f43a0006ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.279+0000 7f43ae84a640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43a0056590 con 0x7f43a00089b0 2024-04-15T15:19:31.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.279+0000 7f43ae84a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f43a00094b0 con 0x7f43a00085d0 2024-04-15T15:19:31.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.279+0000 7f43ae84a640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f43a0056870 con 0x7f43a00e42d0 2024-04-15T15:19:31.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.280+0000 7f2f5d59a640 1 -- 172.21.15.105:0/2937597495 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f2f4800be00 con 0x7f2f600d1b20 2024-04-15T15:19:31.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.280+0000 7f1f07fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1f08163130 0x7f1f080cf820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39576/0 (socket says 172.21.15.105:39576) 2024-04-15T15:19:31.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.280+0000 7f1f07fff640 1 -- 172.21.15.105:0/3158122149 learned_addr learned my addr 172.21.15.105:0/3158122149 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:31.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.280+0000 7f62ca1d1640 1 -- 172.21.15.105:0/2884183583 --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62bc053c30 con 0x7f62bc0df6f0 2024-04-15T15:19:31.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.280+0000 7f2f5ed9d640 1 --2- 172.21.15.105:0/2937597495 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f48004d90 0x7f2f480051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.280+0000 7f43a77fe640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f43a00089b0 0x7f43a00e3bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.105:36796/0 (socket says 172.21.15.105:36796) 2024-04-15T15:19:31.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.280+0000 7f43a77fe640 1 -- 172.21.15.105:0/3801197582 learned_addr learned my addr 172.21.15.105:0/3801197582 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:19:31.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.280+0000 7f43a7fff640 1 --2- 172.21.15.105:0/3801197582 >> v2:172.21.15.105:3300/0 conn(0x7f43a00085d0 0x7f43a00dfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.280+0000 7f62c99d0640 1 --2- 172.21.15.105:0/2884183583 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62bc0507a0 0x7f62bc054cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.280+0000 7f43acde7640 1 --2- 172.21.15.105:0/3801197582 >> v2:172.21.15.154:3300/0 conn(0x7f43a00e42d0 0x7f43a0006ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.280+0000 7f62bbfff640 1 -- 172.21.15.105:0/2884183583 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 382476356 0 0) 0x7f62bc053c30 con 0x7f62bc0df6f0 2024-04-15T15:19:31.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.280+0000 7f7f8c152640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f840af1b0 con 0x7f7f840b3cc0 2024-04-15T15:19:31.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.280+0000 7f7f89eee640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7f8415acc0 0x7f7f840b4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.280+0000 7f62ca1d1640 1 -- 172.21.15.105:0/2884183583 --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62bc053f10 con 0x7f62bc03ad50 2024-04-15T15:19:31.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.280+0000 7f62ca1d1640 1 -- 172.21.15.105:0/2884183583 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f62bc0541f0 con 0x7f62bc0507a0 2024-04-15T15:19:31.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.280+0000 7f7f8c152640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f840b45c0 con 0x7f7f8406bf10 2024-04-15T15:19:31.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.280+0000 7f7f8c152640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f7f8406ba80 con 0x7f7f8415acc0 2024-04-15T15:19:31.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.280+0000 7f7f8a6ef640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f7f8406bf10 0x7f7f8406c360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.105:36802/0 (socket says 172.21.15.105:36802) 2024-04-15T15:19:31.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.280+0000 7f43a6ffd640 1 -- 172.21.15.105:0/3801197582 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 288438436 0 0) 0x7f43a0056590 con 0x7f43a00089b0 2024-04-15T15:19:31.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.280+0000 7f7f8a6ef640 1 -- 172.21.15.105:0/3396482776 learned_addr learned my addr 172.21.15.105:0/3396482776 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:19:31.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.280+0000 7f62bbfff640 1 -- 172.21.15.105:0/2884183583 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f62b0003520 con 0x7f62bc0df6f0 2024-04-15T15:19:31.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.280+0000 7f7f89eee640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7f8415acc0 0x7f7f840b4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.105:44996/0 (socket says 172.21.15.105:44996) 2024-04-15T15:19:31.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.280+0000 7f43a6ffd640 1 -- 172.21.15.105:0/3801197582 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4398003520 con 0x7f43a00089b0 2024-04-15T15:19:31.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.280+0000 7f43a6ffd640 1 -- 172.21.15.105:0/3801197582 <== mon.1 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1136443287 0 0) 0x7f4398003520 con 0x7f43a00089b0 2024-04-15T15:19:31.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.280+0000 7f43a6ffd640 1 -- 172.21.15.105:0/3801197582 >> v2:172.21.15.154:3300/0 conn(0x7f43a00e42d0 msgr2=0x7f43a0006ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:31.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.280+0000 7f43a6ffd640 1 --2- 172.21.15.105:0/3801197582 >> v2:172.21.15.154:3300/0 conn(0x7f43a00e42d0 0x7f43a0006ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.281+0000 7f43a6ffd640 1 -- 172.21.15.105:0/3801197582 >> v2:172.21.15.105:3300/0 conn(0x7f43a00085d0 msgr2=0x7f43a00dfef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:31.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.281+0000 7f43a6ffd640 1 --2- 172.21.15.105:0/3801197582 >> v2:172.21.15.105:3300/0 conn(0x7f43a00085d0 0x7f43a00dfef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.281+0000 7f43a6ffd640 1 -- 172.21.15.105:0/3801197582 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43a0008e60 con 0x7f43a00089b0 2024-04-15T15:19:31.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.281+0000 7f43a6ffd640 1 -- 172.21.15.105:0/3801197582 <== mon.1 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7f439c001670 con 0x7f43a00089b0 2024-04-15T15:19:31.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.280+0000 7f1f0cbc6640 1 --2- 172.21.15.105:0/3158122149 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f0815e9d0 0x7f1f080d3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.280+0000 7f1f07fff640 1 -- 172.21.15.105:0/3158122149 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f0815e9d0 msgr2=0x7f1f080d3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:31.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.280+0000 7f1f07fff640 1 --2- 172.21.15.105:0/3158122149 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f0815e9d0 0x7f1f080d3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.280+0000 7f1f07fff640 1 -- 172.21.15.105:0/3158122149 >> v1:172.21.15.154:6789/0 conn(0x7f1f080d0ef0 legacy=0x7f1f080d1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.280+0000 7f1f07fff640 1 -- 172.21.15.105:0/3158122149 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f080ba560 con 0x7f1f08163130 2024-04-15T15:19:31.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.281+0000 7f43a6ffd640 1 -- 172.21.15.105:0/3801197582 >> v1:172.21.15.154:6789/0 conn(0x7f43a00089b0 legacy=0x7f43a00e3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.279+0000 7f1a7a73d640 1 -- start start 2024-04-15T15:19:31.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.281+0000 7f43a6ffd640 1 --2- 172.21.15.105:0/3801197582 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4398003a00 0x7f4398003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.281+0000 7f43a6ffd640 1 --2- 172.21.15.105:0/3801197582 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4398004800 0x7f4398004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.281+0000 7f1a7a73d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1a740b4ce0 0x7f1a7406b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.281+0000 7f43a6ffd640 1 --2- 172.21.15.105:0/3801197582 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f43980051c0 0x7f439800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.281+0000 7f43a6ffd640 1 -- 172.21.15.105:0/3801197582 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f43a00094b0 con 0x7f4398003a00 2024-04-15T15:19:31.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.281+0000 7f43a6ffd640 1 -- 172.21.15.105:0/3801197582 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f43a0056870 con 0x7f4398004800 2024-04-15T15:19:31.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.281+0000 7f43a77fe640 1 --2- 172.21.15.105:0/3801197582 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4398004800 0x7f4398004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.281+0000 7f43a6ffd640 1 -- 172.21.15.105:0/3801197582 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f439800c040 con 0x7f43980051c0 2024-04-15T15:19:31.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.281+0000 7f43acde7640 1 --2- 172.21.15.105:0/3801197582 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f43980051c0 0x7f439800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.281+0000 7f2f5ed9d640 1 -- 172.21.15.105:0/2937597495 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2f600d1b20 msgr2=0x7f2f48003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:31.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.281+0000 7f7f89eee640 1 -- 172.21.15.105:0/3396482776 >> v1:172.21.15.154:6789/0 conn(0x7f7f8406bf10 legacy=0x7f7f8406c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.281+0000 7f7f89eee640 1 -- 172.21.15.105:0/3396482776 >> v1:172.21.15.105:6789/0 conn(0x7f7f840b3cc0 legacy=0x7f7f840b4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:31.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.281+0000 7f7f89eee640 1 -- 172.21.15.105:0/3396482776 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f840af430 con 0x7f7f8415acc0 2024-04-15T15:19:31.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.281+0000 7f2f5ed9d640 1 --2- 172.21.15.105:0/2937597495 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2f600d1b20 0x7f2f48003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.281+0000 7f2f5ed9d640 1 -- 172.21.15.105:0/2937597495 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f480040a0 msgr2=0x7f2f48004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:31.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.281+0000 7f2f5ed9d640 1 --2- 172.21.15.105:0/2937597495 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f480040a0 0x7f2f48004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.281+0000 7f62bbfff640 1 -- 172.21.15.105:0/2884183583 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 182978624 0 0) 0x7f62b0003520 con 0x7f62bc0df6f0 2024-04-15T15:19:31.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.281+0000 7f62bbfff640 1 -- 172.21.15.105:0/2884183583 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62bc0507a0 msgr2=0x7f62bc054cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:31.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.281+0000 7f62bbfff640 1 --2- 172.21.15.105:0/2884183583 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62bc0507a0 0x7f62bc054cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.281+0000 7f62bbfff640 1 -- 172.21.15.105:0/2884183583 >> v1:172.21.15.154:6789/0 conn(0x7f62bc03ad50 legacy=0x7f62bc050090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.281+0000 7f62bbfff640 1 -- 172.21.15.105:0/2884183583 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62bc0544d0 con 0x7f62bc0df6f0 2024-04-15T15:19:31.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.281+0000 7f62bbfff640 1 -- 172.21.15.105:0/2884183583 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7f62c0001670 con 0x7f62bc0df6f0 2024-04-15T15:19:31.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.281+0000 7f62bbfff640 1 -- 172.21.15.105:0/2884183583 >> v1:172.21.15.105:6789/0 conn(0x7f62bc0df6f0 legacy=0x7f62bc053400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.281+0000 7f62bbfff640 1 --2- 172.21.15.105:0/2884183583 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62b00039e0 0x7f62b0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.281+0000 7f62bbfff640 1 --2- 172.21.15.105:0/2884183583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62b0004800 0x7f62b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.281+0000 7f62bbfff640 1 --2- 172.21.15.105:0/2884183583 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62b00051c0 0x7f62b000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.281+0000 7f62bbfff640 1 -- 172.21.15.105:0/2884183583 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f62bc0541f0 con 0x7f62b0004800 2024-04-15T15:19:31.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.281+0000 7f62bbfff640 1 -- 172.21.15.105:0/2884183583 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f62b000c0c0 con 0x7f62b00051c0 2024-04-15T15:19:31.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.281+0000 7f62bbfff640 1 -- 172.21.15.105:0/2884183583 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f62b000c2c0 con 0x7f62b00039e0 2024-04-15T15:19:31.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.281+0000 7f62c89ce640 1 --2- 172.21.15.105:0/2884183583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62b0004800 0x7f62b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.281+0000 7f2f5ed9d640 1 -- 172.21.15.105:0/2937597495 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f48003520 con 0x7f2f48004d90 2024-04-15T15:19:31.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.282+0000 7f2f5e59c640 1 --2- 172.21.15.105:0/2937597495 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2f600d1b20 0x7f2f48003a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:19:31.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.282+0000 7f43acde7640 1 -- 172.21.15.105:0/3801197582 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4398004800 msgr2=0x7f4398004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:31.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.282+0000 7f43acde7640 1 --2- 172.21.15.105:0/3801197582 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4398004800 0x7f4398004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.282+0000 7f43acde7640 1 -- 172.21.15.105:0/3801197582 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4398003a00 msgr2=0x7f4398003e20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-15T15:19:31.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.282+0000 7f43acde7640 1 --2- 172.21.15.105:0/3801197582 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4398003a00 0x7f4398003e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.282+0000 7f43acde7640 1 -- 172.21.15.105:0/3801197582 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4398003520 con 0x7f43980051c0 2024-04-15T15:19:31.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.281+0000 7f1a737fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1a740b4ce0 0x7f1a7406b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.282+0000 7f7f89eee640 1 --2- 172.21.15.105:0/3396482776 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f8415acc0 0x7f7f840b4840 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7f8406bc00 tx=0x7f7f8002beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee6181d74da49573 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:31.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.282+0000 7f1a7a73d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1a7406bf00 0x7f1a7406e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.282+0000 7f1a7a73d640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a7406d300 con 0x7f1a7415d300 2024-04-15T15:19:31.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.282+0000 7f1a7a73d640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f1a7406c6f0 con 0x7f1a740b4ce0 2024-04-15T15:19:31.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.282+0000 7f1a7a73d640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f1a7406c9d0 con 0x7f1a7406bf00 2024-04-15T15:19:31.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.282+0000 7f1a73fff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f1a7415d300 0x7f1a740b45d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.105:36816/0 (socket says 172.21.15.105:36816) 2024-04-15T15:19:31.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.282+0000 7f1a73fff640 1 -- 172.21.15.105:0/3830587571 learned_addr learned my addr 172.21.15.105:0/3830587571 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:19:31.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.282+0000 7f1a72ffd640 1 -- 172.21.15.105:0/3830587571 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 525048987 0 0) 0x7f1a7406d300 con 0x7f1a7415d300 2024-04-15T15:19:31.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.282+0000 7f1a72ffd640 1 -- 172.21.15.105:0/3830587571 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1a5c003520 con 0x7f1a7415d300 2024-04-15T15:19:31.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.282+0000 7f62c91cf640 1 --2- 172.21.15.105:0/2884183583 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62b00039e0 0x7f62b0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.282+0000 7f62c99d0640 1 --2- 172.21.15.105:0/2884183583 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62b00051c0 0x7f62b000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.282+0000 7f62c89ce640 1 -- 172.21.15.105:0/2884183583 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62b00039e0 msgr2=0x7f62b0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:31.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.282+0000 7f62c89ce640 1 --2- 172.21.15.105:0/2884183583 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62b00039e0 0x7f62b0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.282+0000 7f62c89ce640 1 -- 172.21.15.105:0/2884183583 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62b00051c0 msgr2=0x7f62b000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:31.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.282+0000 7f62c89ce640 1 --2- 172.21.15.105:0/2884183583 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62b00051c0 0x7f62b000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.282+0000 7f62c89ce640 1 -- 172.21.15.105:0/2884183583 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62b0003520 con 0x7f62b0004800 2024-04-15T15:19:31.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.282+0000 7f7f88eec640 1 -- 172.21.15.105:0/3396482776 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f80039070 con 0x7f7f8415acc0 2024-04-15T15:19:31.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.282+0000 7f7f88eec640 1 -- 172.21.15.105:0/3396482776 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f8002cc60 con 0x7f7f8415acc0 2024-04-15T15:19:31.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.282+0000 7f7f88eec640 1 -- 172.21.15.105:0/3396482776 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f80034660 con 0x7f7f8415acc0 2024-04-15T15:19:31.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.282+0000 7f1a72ffd640 1 -- 172.21.15.105:0/3830587571 <== mon.1 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2629597646 0 0) 0x7f1a5c003520 con 0x7f1a7415d300 2024-04-15T15:19:31.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.282+0000 7f1a72ffd640 1 -- 172.21.15.105:0/3830587571 >> v2:172.21.15.154:3300/0 conn(0x7f1a7406bf00 msgr2=0x7f1a7406e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:31.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.282+0000 7f1a72ffd640 1 --2- 172.21.15.105:0/3830587571 >> v2:172.21.15.154:3300/0 conn(0x7f1a7406bf00 0x7f1a7406e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.282+0000 7f1a72ffd640 1 -- 172.21.15.105:0/3830587571 >> v2:172.21.15.105:3300/0 conn(0x7f1a740b4ce0 msgr2=0x7f1a7406b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:31.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.282+0000 7f1a72ffd640 1 --2- 172.21.15.105:0/3830587571 >> v2:172.21.15.105:3300/0 conn(0x7f1a740b4ce0 0x7f1a7406b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.282+0000 7f1a72ffd640 1 -- 172.21.15.105:0/3830587571 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a7406ccb0 con 0x7f1a7415d300 2024-04-15T15:19:31.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.282+0000 7f1a72ffd640 1 -- 172.21.15.105:0/3830587571 <== mon.1 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7f1a68001690 con 0x7f1a7415d300 2024-04-15T15:19:31.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.282+0000 7f1a72ffd640 1 -- 172.21.15.105:0/3830587571 >> v1:172.21.15.154:6789/0 conn(0x7f1a7415d300 legacy=0x7f1a740b45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.283+0000 7f1a72ffd640 1 --2- 172.21.15.105:0/3830587571 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1a5c003a00 0x7f1a5c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.283+0000 7f1a72ffd640 1 --2- 172.21.15.105:0/3830587571 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a5c004800 0x7f1a5c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.283+0000 7f43acde7640 1 -- 172.21.15.105:0/3801197582 reap_dead start 2024-04-15T15:19:31.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.283+0000 7f1a72ffd640 1 --2- 172.21.15.105:0/3830587571 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1a5c005210 0x7f1a5c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.283+0000 7f1a72ffd640 1 -- 172.21.15.105:0/3830587571 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1a7406c6f0 con 0x7f1a5c004800 2024-04-15T15:19:31.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.283+0000 7f1a72ffd640 1 -- 172.21.15.105:0/3830587571 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1a7406c9d0 con 0x7f1a5c003a00 2024-04-15T15:19:31.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.283+0000 7f1a72ffd640 1 -- 172.21.15.105:0/3830587571 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f1a5c00c1d0 con 0x7f1a5c005210 2024-04-15T15:19:31.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.283+0000 7f2f5ed9d640 1 --2- 172.21.15.105:0/2937597495 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f48004d90 0x7f2f480051f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2f600b5f30 tx=0x7f2f48005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=762c4eb355165bd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:31.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.283+0000 7f1a73fff640 1 --2- 172.21.15.105:0/3830587571 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1a5c003a00 0x7f1a5c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.283+0000 7f1a78cda640 1 --2- 172.21.15.105:0/3830587571 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1a5c005210 0x7f1a5c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.283+0000 7f2f5d59a640 1 -- 172.21.15.105:0/2937597495 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f580055d0 con 0x7f2f48004d90 2024-04-15T15:19:31.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.283+0000 7f2f5d59a640 1 -- 172.21.15.105:0/2937597495 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f58031030 con 0x7f2f48004d90 2024-04-15T15:19:31.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.283+0000 7f2f5d59a640 1 -- 172.21.15.105:0/2937597495 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5802d950 con 0x7f2f48004d90 2024-04-15T15:19:31.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.283+0000 7f1f07fff640 1 --2- 172.21.15.105:0/3158122149 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f08163130 0x7f1f080cf820 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4009870 tx=0x7f1ef402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb05eece38edd249 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:31.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.283+0000 7f7f8c152640 1 -- 172.21.15.105:0/3396482776 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f8415acc0 msgr2=0x7f7f840b4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.283+0000 7f43acde7640 1 --2- 172.21.15.105:0/3801197582 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f43980051c0 0x7f439800b9c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f43a8051650 tx=0x7f4398005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=467c73618f058d95 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:31.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.283+0000 7f62c89ce640 1 --2- 172.21.15.105:0/2884183583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62b0004800 0x7f62b0004c80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f62b400e410 tx=0x7f62b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90a15149db3cbd4a server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:31.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.283+0000 7f1f077fe640 1 -- 172.21.15.105:0/3158122149 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef4035020 con 0x7f1f08163130 2024-04-15T15:19:31.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.283+0000 7f7f8c152640 1 --2- 172.21.15.105:0/3396482776 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f8415acc0 0x7f7f840b4840 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7f8406bc00 tx=0x7f7f8002beb0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.283+0000 7f1f077fe640 1 -- 172.21.15.105:0/3158122149 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ef4031640 con 0x7f1f08163130 2024-04-15T15:19:31.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.284+0000 7f43a6ffd640 1 -- 172.21.15.105:0/3801197582 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a80b5070 con 0x7f43980051c0 2024-04-15T15:19:31.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.284+0000 7f7f8c152640 1 -- 172.21.15.105:0/3396482776 shutdown_connections 2024-04-15T15:19:31.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.284+0000 7f62bbfff640 1 -- 172.21.15.105:0/2884183583 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b403a070 con 0x7f62b0004800 2024-04-15T15:19:31.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.284+0000 7f7f8c152640 1 --2- 172.21.15.105:0/3396482776 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f8415acc0 0x7f7f840b4840 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.284+0000 7f43a6ffd640 1 -- 172.21.15.105:0/3801197582 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43a80b0410 con 0x7f43980051c0 2024-04-15T15:19:31.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.284+0000 7f7f8c152640 1 -- 172.21.15.105:0/3396482776 >> 172.21.15.105:0/3396482776 conn(0x7f7f840b34b0 msgr2=0x7f7f840d0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:31.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.284+0000 7f62bbfff640 1 -- 172.21.15.105:0/2884183583 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62b4034df0 con 0x7f62b0004800 2024-04-15T15:19:31.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.284+0000 7f62bbfff640 1 -- 172.21.15.105:0/2884183583 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b400e450 con 0x7f62b0004800 2024-04-15T15:19:31.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.283+0000 7f1a73fff640 1 -- 172.21.15.105:0/3830587571 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1a5c005210 msgr2=0x7f1a5c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:31.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.283+0000 7f1a73fff640 1 --2- 172.21.15.105:0/3830587571 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1a5c005210 0x7f1a5c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.283+0000 7f1a73fff640 1 -- 172.21.15.105:0/3830587571 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a5c004800 msgr2=0x7f1a5c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:31.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.283+0000 7f1a73fff640 1 --2- 172.21.15.105:0/3830587571 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a5c004800 0x7f1a5c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.283+0000 7f1a73fff640 1 -- 172.21.15.105:0/3830587571 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a5c003520 con 0x7f1a5c003a00 2024-04-15T15:19:31.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.284+0000 7f1a78cda640 1 -- 172.21.15.105:0/3830587571 reap_dead start 2024-04-15T15:19:31.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.284+0000 7f7f8c152640 1 -- 172.21.15.105:0/3396482776 shutdown_connections 2024-04-15T15:19:31.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.284+0000 7f7f8c152640 1 -- 172.21.15.105:0/3396482776 wait complete. 2024-04-15T15:19:31.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.284+0000 7f2f5f59e640 1 -- 172.21.15.105:0/2937597495 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f48004d90 msgr2=0x7f2f480051f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.284+0000 7f2f5f59e640 1 --2- 172.21.15.105:0/2937597495 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f48004d90 0x7f2f480051f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2f600b5f30 tx=0x7f2f48005830 comp rx=0 tx=0).stop 2024-04-15T15:19:31.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.284+0000 7f2f5ed9d640 1 -- 172.21.15.105:0/2937597495 reap_dead start 2024-04-15T15:19:31.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.284+0000 7f3cf8ab1640 1 Processor -- start 2024-04-15T15:19:31.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.284+0000 7f3cf8ab1640 1 -- start start 2024-04-15T15:19:31.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.284+0000 7f1a73fff640 1 --2- 172.21.15.105:0/3830587571 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1a5c003a00 0x7f1a5c003e20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1a6800c160 tx=0x7f1a5c005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f50dcceae5f8bad server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:31.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.284+0000 7f2f5f59e640 1 -- 172.21.15.105:0/2937597495 shutdown_connections 2024-04-15T15:19:31.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.284+0000 7f2f5f59e640 1 -- 172.21.15.105:0/2937597495 >> 172.21.15.105:0/2937597495 conn(0x7f2f600b3fe0 msgr2=0x7f2f600b43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:31.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.284+0000 7f43a6ffd640 1 -- 172.21.15.105:0/3801197582 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a80502c0 con 0x7f43980051c0 2024-04-15T15:19:31.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.284+0000 7f2f5f59e640 1 -- 172.21.15.105:0/2937597495 shutdown_connections 2024-04-15T15:19:31.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.285+0000 7f2f5f59e640 1 -- 172.21.15.105:0/2937597495 wait complete. 2024-04-15T15:19:31.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.285+0000 7f43ae84a640 1 -- 172.21.15.105:0/3801197582 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f43980051c0 msgr2=0x7f439800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.285+0000 7f43ae84a640 1 --2- 172.21.15.105:0/3801197582 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f43980051c0 0x7f439800b9c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f43a8051650 tx=0x7f4398005830 comp rx=0 tx=0).stop 2024-04-15T15:19:31.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.285+0000 7f1a72ffd640 1 -- 172.21.15.105:0/3830587571 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a68007c80 con 0x7f1a5c003a00 2024-04-15T15:19:31.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.285+0000 7f1a72ffd640 1 -- 172.21.15.105:0/3830587571 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a68030070 con 0x7f1a5c003a00 2024-04-15T15:19:31.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.285+0000 7f1a72ffd640 1 -- 172.21.15.105:0/3830587571 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a68005d50 con 0x7f1a5c003a00 2024-04-15T15:19:31.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.285+0000 7f62ca1d1640 1 -- 172.21.15.105:0/2884183583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62b0004800 msgr2=0x7f62b0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.285+0000 7f62ca1d1640 1 --2- 172.21.15.105:0/2884183583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62b0004800 0x7f62b0004c80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f62b400e410 tx=0x7f62b0005830 comp rx=0 tx=0).stop 2024-04-15T15:19:31.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.285+0000 7f43ae84a640 1 -- 172.21.15.105:0/3801197582 shutdown_connections 2024-04-15T15:19:31.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.285+0000 7f43ae84a640 1 --2- 172.21.15.105:0/3801197582 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f43980051c0 0x7f439800b9c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.285+0000 7f43ae84a640 1 -- 172.21.15.105:0/3801197582 >> 172.21.15.105:0/3801197582 conn(0x7f43a001ef40 msgr2=0x7f43a0055c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:31.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.285+0000 7f62c99d0640 1 -- 172.21.15.105:0/2884183583 reap_dead start 2024-04-15T15:19:31.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.285+0000 7f62ca1d1640 1 -- 172.21.15.105:0/2884183583 shutdown_connections 2024-04-15T15:19:31.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.285+0000 7f62ca1d1640 1 -- 172.21.15.105:0/2884183583 >> 172.21.15.105:0/2884183583 conn(0x7f62bc055350 msgr2=0x7f62bc055760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:31.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.285+0000 7f62ca1d1640 1 -- 172.21.15.105:0/2884183583 shutdown_connections 2024-04-15T15:19:31.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.285+0000 7f62ca1d1640 1 -- 172.21.15.105:0/2884183583 wait complete. 2024-04-15T15:19:31.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.285+0000 7f43ae84a640 1 -- 172.21.15.105:0/3801197582 shutdown_connections 2024-04-15T15:19:31.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.285+0000 7f449b3ba640 1 Processor -- start 2024-04-15T15:19:31.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.285+0000 7f449b3ba640 1 -- start start 2024-04-15T15:19:31.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.286+0000 7f3cf8ab1640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3cf409ba50 0x7f3cf40cd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.286+0000 7f3cf8ab1640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3cf415cc60 con 0x7f3cf415c880 2024-04-15T15:19:31.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.286+0000 7f3cf8ab1640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3cf40d0bf0 con 0x7f3cf40cdcd0 2024-04-15T15:19:31.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.286+0000 7f3cf8ab1640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f3cf40d0ed0 con 0x7f3cf409ba50 2024-04-15T15:19:31.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.286+0000 7f3cf259c640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f3cf415c880 0x7f3cf40cd080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41022/0 (socket says 172.21.15.105:41022) 2024-04-15T15:19:31.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.286+0000 7f3cf259c640 1 -- 172.21.15.105:0/2149023496 learned_addr learned my addr 172.21.15.105:0/2149023496 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:19:31.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.286+0000 7f1a7a73d640 1 -- 172.21.15.105:0/3830587571 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1a5c003a00 msgr2=0x7f1a5c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.286+0000 7f1a7a73d640 1 --2- 172.21.15.105:0/3830587571 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1a5c003a00 0x7f1a5c003e20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1a6800c160 tx=0x7f1a5c005830 comp rx=0 tx=0).stop 2024-04-15T15:19:31.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.284+0000 7f1f0ee2a640 1 -- 172.21.15.105:0/3158122149 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f08163130 msgr2=0x7f1f080cf820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.284+0000 7f1f0ee2a640 1 --2- 172.21.15.105:0/3158122149 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f08163130 0x7f1f080cf820 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4009870 tx=0x7f1ef402b8f0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.286+0000 7f1f0ee2a640 1 -- 172.21.15.105:0/3158122149 shutdown_connections 2024-04-15T15:19:31.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.286+0000 7f1f0ee2a640 1 --2- 172.21.15.105:0/3158122149 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f08163130 0x7f1f080cf820 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.286+0000 7f1f0ee2a640 1 --2- 172.21.15.105:0/3158122149 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f0815e9d0 0x7f1f080d3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.286+0000 7f1f0ee2a640 1 -- 172.21.15.105:0/3158122149 >> 172.21.15.105:0/3158122149 conn(0x7f1f08067720 msgr2=0x7f1f08067b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:31.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.286+0000 7f1f0ee2a640 1 -- 172.21.15.105:0/3158122149 shutdown_connections 2024-04-15T15:19:31.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.286+0000 7f1f0ee2a640 1 -- 172.21.15.105:0/3158122149 wait complete. 2024-04-15T15:19:31.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.288+0000 7f1f0ee2a640 1 Processor -- start 2024-04-15T15:19:31.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.288+0000 7f1f0ee2a640 1 -- start start 2024-04-15T15:19:31.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.289+0000 7f1f0ee2a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f080d0ef0 0x7f1f08161b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.289+0000 7f1f0ee2a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f0815e9d0 0x7f1f08159d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.289+0000 7f1f0ee2a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f08163130 0x7f1f080c8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.288+0000 7f1a7a73d640 1 -- 172.21.15.105:0/3830587571 shutdown_connections 2024-04-15T15:19:31.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.288+0000 7f1a7a73d640 1 --2- 172.21.15.105:0/3830587571 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1a5c003a00 0x7f1a5c003e20 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.288+0000 7f1a7a73d640 1 -- 172.21.15.105:0/3830587571 >> 172.21.15.105:0/3830587571 conn(0x7f1a740b34b0 msgr2=0x7f1a740d0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:31.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.288+0000 7f1a7a73d640 1 -- 172.21.15.105:0/3830587571 shutdown_connections 2024-04-15T15:19:31.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.288+0000 7f1a7a73d640 1 -- 172.21.15.105:0/3830587571 wait complete. 2024-04-15T15:19:31.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.286+0000 7f7f8c152640 1 Processor -- start 2024-04-15T15:19:31.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.286+0000 7f7f8c152640 1 -- start start 2024-04-15T15:19:31.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.288+0000 7f7f8c152640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f8406bf10 0x7f7f8414e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.288+0000 7f7f8c152640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f840b3cc0 0x7f7f84151860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.288+0000 7f7f8c152640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f8415acc0 0x7f7f84151da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.288+0000 7f7f8c152640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7f8415b0a0 con 0x7f7f8415acc0 2024-04-15T15:19:31.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.288+0000 7f7f8c152640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7f840af9f0 con 0x7f7f8406bf10 2024-04-15T15:19:31.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.288+0000 7f7f8c152640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f7f8415b220 con 0x7f7f840b3cc0 2024-04-15T15:19:31.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.288+0000 7f7f896ed640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f840b3cc0 0x7f7f84151860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.288+0000 7f7f896ed640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f840b3cc0 0x7f7f84151860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.105:56988/0 (socket says 172.21.15.105:56988) 2024-04-15T15:19:31.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.288+0000 7f7f896ed640 1 -- 172.21.15.105:0/1526616838 learned_addr learned my addr 172.21.15.105:0/1526616838 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:31.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.288+0000 7f7f8a6ef640 1 --2- 172.21.15.105:0/1526616838 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f8415acc0 0x7f7f84151da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.288+0000 7f7f89eee640 1 --2- 172.21.15.105:0/1526616838 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f8406bf10 0x7f7f8414e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.289+0000 7f7f8a6ef640 1 -- 172.21.15.105:0/1526616838 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f840b3cc0 msgr2=0x7f7f84151860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.289+0000 7f7f8a6ef640 1 --2- 172.21.15.105:0/1526616838 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f840b3cc0 0x7f7f84151860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.289+0000 7f7f8a6ef640 1 -- 172.21.15.105:0/1526616838 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f8406bf10 msgr2=0x7f7f8414e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.289+0000 7f7f8a6ef640 1 --2- 172.21.15.105:0/1526616838 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f8406bf10 0x7f7f8414e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.289+0000 7f7f8a6ef640 1 -- 172.21.15.105:0/1526616838 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f8414fd30 con 0x7f7f8415acc0 2024-04-15T15:19:31.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.289+0000 7f7f8a6ef640 1 --2- 172.21.15.105:0/1526616838 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f8415acc0 0x7f7f84151da0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c00c390 tx=0x7f7f7c00c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:31.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.289+0000 7f7f7a7fc640 1 -- 172.21.15.105:0/1526616838 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c017070 con 0x7f7f8415acc0 2024-04-15T15:19:31.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.289+0000 7f7f7a7fc640 1 -- 172.21.15.105:0/1526616838 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f7c0064e0 con 0x7f7f8415acc0 2024-04-15T15:19:31.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.289+0000 7f7f8c152640 1 -- 172.21.15.105:0/1526616838 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f8414ed70 con 0x7f7f8415acc0 2024-04-15T15:19:31.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.288+0000 7f62ca1d1640 1 Processor -- start 2024-04-15T15:19:31.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.288+0000 7f62ca1d1640 1 -- start start 2024-04-15T15:19:31.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.289+0000 7f62ca1d1640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62b0004800 0x7f62bc034db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.289+0000 7f62ca1d1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62bc0080f0 0x7f62bc0352f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.285+0000 7f43ae84a640 1 -- 172.21.15.105:0/3801197582 wait complete. 2024-04-15T15:19:31.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.287+0000 7f43ae84a640 1 Processor -- start 2024-04-15T15:19:31.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.287+0000 7f43ae84a640 1 -- start start 2024-04-15T15:19:31.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.289+0000 7f43ae84a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f43980051c0 0x7f43a0053c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.289+0000 7f43ae84a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43a00524e0 0x7f43a0052960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.289+0000 7f43ae84a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43a00e0120 0x7f43a00e05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.289+0000 7f43a77fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43a00524e0 0x7f43a0052960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.289+0000 7f43a77fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43a00524e0 0x7f43a0052960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.105:45056/0 (socket says 172.21.15.105:45056) 2024-04-15T15:19:31.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.289+0000 7f43a77fe640 1 -- 172.21.15.105:0/205335355 learned_addr learned my addr 172.21.15.105:0/205335355 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:31.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.289+0000 7f43ae84a640 1 -- 172.21.15.105:0/205335355 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f43a00e4910 con 0x7f43a00e0120 2024-04-15T15:19:31.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.289+0000 7f43ae84a640 1 -- 172.21.15.105:0/205335355 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f43a0056b50 con 0x7f43a00524e0 2024-04-15T15:19:31.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.289+0000 7f43ae84a640 1 -- 172.21.15.105:0/205335355 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f43a00e3bc0 con 0x7f43980051c0 2024-04-15T15:19:31.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.286+0000 7f2f5f59e640 1 Processor -- start 2024-04-15T15:19:31.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.286+0000 7f2f5f59e640 1 -- start start 2024-04-15T15:19:31.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.288+0000 7f2f5f59e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f48004d90 0x7f2f600b0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.288+0000 7f2f5f59e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2f600b2720 0x7f2f6014d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.288+0000 7f2f5f59e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f6009aa40 0x7f2f6009aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.288+0000 7f2f5e59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f48004d90 0x7f2f600b0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.288+0000 7f2f5f59e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2f600b52e0 con 0x7f2f6009aa40 2024-04-15T15:19:31.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.288+0000 7f2f5e59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f48004d90 0x7f2f600b0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.105:45042/0 (socket says 172.21.15.105:45042) 2024-04-15T15:19:31.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.288+0000 7f2f5e59c640 1 -- 172.21.15.105:0/3482216990 learned_addr learned my addr 172.21.15.105:0/3482216990 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:31.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.288+0000 7f2f5dd9b640 1 --2- 172.21.15.105:0/3482216990 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2f600b2720 0x7f2f6014d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.288+0000 7f2f5f59e640 1 -- 172.21.15.105:0/3482216990 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2f600cfdc0 con 0x7f2f48004d90 2024-04-15T15:19:31.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.288+0000 7f2f5f59e640 1 -- 172.21.15.105:0/3482216990 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f2f600b5460 con 0x7f2f600b2720 2024-04-15T15:19:31.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.288+0000 7f2f5ed9d640 1 --2- 172.21.15.105:0/3482216990 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f6009aa40 0x7f2f6009aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.288+0000 7f2f5e59c640 1 -- 172.21.15.105:0/3482216990 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2f600b2720 msgr2=0x7f2f6014d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.288+0000 7f2f5e59c640 1 --2- 172.21.15.105:0/3482216990 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2f600b2720 0x7f2f6014d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.288+0000 7f2f5e59c640 1 -- 172.21.15.105:0/3482216990 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f6009aa40 msgr2=0x7f2f6009aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.288+0000 7f2f5e59c640 1 --2- 172.21.15.105:0/3482216990 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f6009aa40 0x7f2f6009aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.288+0000 7f2f5e59c640 1 -- 172.21.15.105:0/3482216990 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f600b32a0 con 0x7f2f48004d90 2024-04-15T15:19:31.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.288+0000 7f2f5dd9b640 1 --2- 172.21.15.105:0/3482216990 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2f600b2720 0x7f2f6014d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:19:31.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.289+0000 7f2f5ed9d640 1 --2- 172.21.15.105:0/3482216990 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f6009aa40 0x7f2f6009aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:31.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.289+0000 7f2f5e59c640 1 --2- 172.21.15.105:0/3482216990 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f48004d90 0x7f2f600b0140 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2f5400e890 tx=0x7f2f5400ed50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:31.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.289+0000 7f2f4effd640 1 -- 172.21.15.105:0/3482216990 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f54017070 con 0x7f2f48004d90 2024-04-15T15:19:31.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.289+0000 7f2f4effd640 1 -- 172.21.15.105:0/3482216990 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f5400c940 con 0x7f2f48004d90 2024-04-15T15:19:31.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.289+0000 7f2f4effd640 1 -- 172.21.15.105:0/3482216990 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5400cd30 con 0x7f2f48004d90 2024-04-15T15:19:31.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.290+0000 7f2f5f59e640 1 -- 172.21.15.105:0/3482216990 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f6015c400 con 0x7f2f48004d90 2024-04-15T15:19:31.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.287+0000 7f449b3ba640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f449415cb00 0x7f44940cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.287+0000 7f449b3ba640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f44940ce020 0x7f44940b0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.287+0000 7f449b3ba640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44940af4b0 con 0x7f44940d0120 2024-04-15T15:19:31.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.287+0000 7f4498955640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f449415cb00 0x7f44940cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.287+0000 7f4498955640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f449415cb00 0x7f44940cdae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.105:45028/0 (socket says 172.21.15.105:45028) 2024-04-15T15:19:31.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.287+0000 7f4498955640 1 -- 172.21.15.105:0/3214840278 learned_addr learned my addr 172.21.15.105:0/3214840278 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:31.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.287+0000 7f449b3ba640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f44940af790 con 0x7f44940ce020 2024-04-15T15:19:31.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.287+0000 7f449b3ba640 1 -- 172.21.15.105:0/3214840278 --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f44940afa70 con 0x7f449415cb00 2024-04-15T15:19:31.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.287+0000 7f4499957640 1 --2- 172.21.15.105:0/3214840278 >> v2:172.21.15.105:3300/0 conn(0x7f44940ce020 0x7f44940b0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.289+0000 7f448bfff640 1 -- 172.21.15.105:0/3214840278 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 192660613 0 0) 0x7f44940af4b0 con 0x7f44940d0120 2024-04-15T15:19:31.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.289+0000 7f448bfff640 1 -- 172.21.15.105:0/3214840278 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4480003520 con 0x7f44940d0120 2024-04-15T15:19:31.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.286+0000 7f3cf1d9b640 1 --2- 172.21.15.105:0/2149023496 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3cf409ba50 0x7f3cf40cd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.286+0000 7f3cf159a640 1 -- 172.21.15.105:0/2149023496 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 902816473 0 0) 0x7f3cf415cc60 con 0x7f3cf415c880 2024-04-15T15:19:31.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.286+0000 7f3cf159a640 1 -- 172.21.15.105:0/2149023496 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3cdc003520 con 0x7f3cf415c880 2024-04-15T15:19:31.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.287+0000 7f3cf159a640 1 -- 172.21.15.105:0/2149023496 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1827123309 0 0) 0x7f3cdc003520 con 0x7f3cf415c880 2024-04-15T15:19:31.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.287+0000 7f3cf159a640 1 -- 172.21.15.105:0/2149023496 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3cf409ba50 msgr2=0x7f3cf40cd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:31.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.287+0000 7f3cf159a640 1 --2- 172.21.15.105:0/2149023496 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3cf409ba50 0x7f3cf40cd790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.287+0000 7f3cf159a640 1 -- 172.21.15.105:0/2149023496 >> v1:172.21.15.154:6789/0 conn(0x7f3cf40cdcd0 legacy=0x7f3cf40707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:31.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.287+0000 7f3cf159a640 1 -- 172.21.15.105:0/2149023496 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cf40d11b0 con 0x7f3cf415c880 2024-04-15T15:19:31.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.287+0000 7f3cf159a640 1 -- 172.21.15.105:0/2149023496 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7f3cec0019f0 con 0x7f3cf415c880 2024-04-15T15:19:31.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.287+0000 7f3cf159a640 1 -- 172.21.15.105:0/2149023496 >> v1:172.21.15.105:6789/0 conn(0x7f3cf415c880 legacy=0x7f3cf40cd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.287+0000 7f3cf159a640 1 --2- 172.21.15.105:0/2149023496 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3cdc0039e0 0x7f3cdc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.287+0000 7f3cf159a640 1 --2- 172.21.15.105:0/2149023496 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3cdc004800 0x7f3cdc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.287+0000 7f3cf159a640 1 --2- 172.21.15.105:0/2149023496 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3cf409ba50 0x7f3cdc005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.287+0000 7f3cf159a640 1 -- 172.21.15.105:0/2149023496 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf40d0ed0 con 0x7f3cdc004800 2024-04-15T15:19:31.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.287+0000 7f3cf159a640 1 -- 172.21.15.105:0/2149023496 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3cdc00bb50 con 0x7f3cdc0039e0 2024-04-15T15:19:31.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.287+0000 7f3cf159a640 1 -- 172.21.15.105:0/2149023496 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f3cdc00bd20 con 0x7f3cf409ba50 2024-04-15T15:19:31.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.287+0000 7f3cf259c640 1 --2- 172.21.15.105:0/2149023496 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3cdc0039e0 0x7f3cdc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.287+0000 7f3cf2d9d640 1 --2- 172.21.15.105:0/2149023496 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3cf409ba50 0x7f3cdc005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.287+0000 7f3cf1d9b640 1 --2- 172.21.15.105:0/2149023496 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3cdc004800 0x7f3cdc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.287+0000 7f3cf259c640 1 -- 172.21.15.105:0/2149023496 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3cf409ba50 msgr2=0x7f3cdc005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:31.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.287+0000 7f3cf259c640 1 --2- 172.21.15.105:0/2149023496 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3cf409ba50 0x7f3cdc005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.287+0000 7f3cf259c640 1 -- 172.21.15.105:0/2149023496 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3cdc004800 msgr2=0x7f3cdc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:31.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.287+0000 7f3cf259c640 1 --2- 172.21.15.105:0/2149023496 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3cdc004800 0x7f3cdc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.287+0000 7f3cf259c640 1 -- 172.21.15.105:0/2149023496 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cdc003520 con 0x7f3cdc0039e0 2024-04-15T15:19:31.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.289+0000 7f3cf259c640 1 --2- 172.21.15.105:0/2149023496 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3cdc0039e0 0x7f3cdc003de0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3cec00a7a0 tx=0x7f3cdc005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1cb57d91abb8614e server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:31.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.289+0000 7f3cf159a640 1 -- 172.21.15.105:0/2149023496 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cec00a3f0 con 0x7f3cdc0039e0 2024-04-15T15:19:31.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.289+0000 7f3cf159a640 1 -- 172.21.15.105:0/2149023496 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cec034650 con 0x7f3cdc0039e0 2024-04-15T15:19:31.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.289+0000 7f3cf159a640 1 -- 172.21.15.105:0/2149023496 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cec034a80 con 0x7f3cdc0039e0 2024-04-15T15:19:31.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.290+0000 7f62c91cf640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62b0004800 0x7f62bc034db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.290+0000 7f62c91cf640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62b0004800 0x7f62bc034db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.105:57016/0 (socket says 172.21.15.105:57016) 2024-04-15T15:19:31.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.290+0000 7f62c91cf640 1 -- 172.21.15.105:0/1983986801 learned_addr learned my addr 172.21.15.105:0/1983986801 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:31.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.290+0000 7f62ca1d1640 1 --2- 172.21.15.105:0/1983986801 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62bc035830 0x7f62bc177b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.290+0000 7f2f5f59e640 1 -- 172.21.15.105:0/3482216990 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f600b2fc0 con 0x7f2f48004d90 2024-04-15T15:19:31.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.290+0000 7f62ca1d1640 1 -- 172.21.15.105:0/1983986801 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f62bc054e50 con 0x7f62bc0080f0 2024-04-15T15:19:31.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.290+0000 7f62ca1d1640 1 -- 172.21.15.105:0/1983986801 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f62bc054cd0 con 0x7f62bc035830 2024-04-15T15:19:31.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.290+0000 7f62c99d0640 1 --2- 172.21.15.105:0/1983986801 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62bc035830 0x7f62bc177b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.290+0000 7f62ca1d1640 1 -- 172.21.15.105:0/1983986801 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f62bc054fd0 con 0x7f62b0004800 2024-04-15T15:19:31.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.289+0000 7f4498955640 1 -- 172.21.15.105:0/3214840278 >> v2:172.21.15.105:3300/0 conn(0x7f44940ce020 msgr2=0x7f44940b0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:31.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.289+0000 7f4498955640 1 --2- 172.21.15.105:0/3214840278 >> v2:172.21.15.105:3300/0 conn(0x7f44940ce020 0x7f44940b0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.289+0000 7f4498955640 1 -- 172.21.15.105:0/3214840278 >> v1:172.21.15.154:6789/0 conn(0x7f44940d0120 legacy=0x7f44940d0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.289+0000 7f4498955640 1 -- 172.21.15.105:0/3214840278 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44940afd50 con 0x7f449415cb00 2024-04-15T15:19:31.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.289+0000 7f4499957640 1 --2- 172.21.15.105:0/3214840278 >> v2:172.21.15.105:3300/0 conn(0x7f44940ce020 0x7f44940b0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:19:31.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.290+0000 7f1f0d3c7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f08163130 0x7f1f080c8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.290+0000 7f1f0d3c7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f08163130 0x7f1f080c8ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.105:45058/0 (socket says 172.21.15.105:45058) 2024-04-15T15:19:31.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.290+0000 7f1f0d3c7640 1 -- 172.21.15.105:0/3563963370 learned_addr learned my addr 172.21.15.105:0/3563963370 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:31.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.290+0000 7f1f0ee2a640 1 -- 172.21.15.105:0/3563963370 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1f080d45d0 con 0x7f1f080d0ef0 2024-04-15T15:19:31.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.290+0000 7f43a7fff640 1 --2- 172.21.15.105:0/205335355 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f43980051c0 0x7f43a0053c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.290+0000 7f1f0ee2a640 1 -- 172.21.15.105:0/3563963370 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1f080d4450 con 0x7f1f08163130 2024-04-15T15:19:31.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.290+0000 7f1f0ee2a640 1 -- 172.21.15.105:0/3563963370 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f1f080cfd60 con 0x7f1f0815e9d0 2024-04-15T15:19:31.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.290+0000 7f1a7a73d640 1 Processor -- start 2024-04-15T15:19:31.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.290+0000 7f1a7a73d640 1 -- start start 2024-04-15T15:19:31.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.290+0000 7f3cf8ab1640 1 -- 172.21.15.105:0/2149023496 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3cdc0039e0 msgr2=0x7f3cdc003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.290+0000 7f3cf8ab1640 1 --2- 172.21.15.105:0/2149023496 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3cdc0039e0 0x7f3cdc003de0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3cec00a7a0 tx=0x7f3cdc005830 comp rx=0 tx=0).stop 2024-04-15T15:19:31.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.290+0000 7f3cf2d9d640 1 -- 172.21.15.105:0/2149023496 reap_dead start 2024-04-15T15:19:31.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.290+0000 7f3cf8ab1640 1 -- 172.21.15.105:0/2149023496 shutdown_connections 2024-04-15T15:19:31.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.290+0000 7f1f07fff640 1 --2- 172.21.15.105:0/3563963370 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f0815e9d0 0x7f1f08159d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.290+0000 7f3cf8ab1640 1 -- 172.21.15.105:0/2149023496 >> 172.21.15.105:0/2149023496 conn(0x7f3cf40d2570 msgr2=0x7f3cf40d2980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:31.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.290+0000 7f43acde7640 1 --2- 172.21.15.105:0/205335355 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43a00e0120 0x7f43a00e05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.290+0000 7f1f0cbc6640 1 --2- 172.21.15.105:0/3563963370 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f080d0ef0 0x7f1f08161b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.291+0000 7f43a77fe640 1 -- 172.21.15.105:0/205335355 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f43980051c0 msgr2=0x7f43a0053c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.291+0000 7f43a77fe640 1 --2- 172.21.15.105:0/205335355 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f43980051c0 0x7f43a0053c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.291+0000 7f4498955640 1 --2- 172.21.15.105:0/3214840278 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f449415cb00 0x7f44940cdae0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f449406fe20 tx=0x7f448c02b9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45bed5e4d09fd4b server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:31.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.291+0000 7f43a77fe640 1 -- 172.21.15.105:0/205335355 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43a00e0120 msgr2=0x7f43a00e05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.291+0000 7f43a77fe640 1 --2- 172.21.15.105:0/205335355 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43a00e0120 0x7f43a00e05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.291+0000 7f43a77fe640 1 -- 172.21.15.105:0/205335355 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43a0053050 con 0x7f43a00524e0 2024-04-15T15:19:31.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.291+0000 7f1f0cbc6640 1 -- 172.21.15.105:0/3563963370 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f0815e9d0 msgr2=0x7f1f08159d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.291+0000 7f1f0cbc6640 1 --2- 172.21.15.105:0/3563963370 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f0815e9d0 0x7f1f08159d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.291+0000 7f1f0cbc6640 1 -- 172.21.15.105:0/3563963370 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f08163130 msgr2=0x7f1f080c8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.291+0000 7f1f0cbc6640 1 --2- 172.21.15.105:0/3563963370 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f08163130 0x7f1f080c8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.291+0000 7f1f0cbc6640 1 -- 172.21.15.105:0/3563963370 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f080c9530 con 0x7f1f080d0ef0 2024-04-15T15:19:31.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.291+0000 7f43acde7640 1 --2- 172.21.15.105:0/205335355 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43a00e0120 0x7f43a00e05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:19:31.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.291+0000 7f1a7a73d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a5c003a00 0x7f1a740aebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.291+0000 7f2f5f59e640 1 -- 172.21.15.105:0/3482216990 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f60069150 con 0x7f2f48004d90 2024-04-15T15:19:31.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.291+0000 7f1f0cbc6640 1 --2- 172.21.15.105:0/3563963370 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f080d0ef0 0x7f1f08161b80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1efc00d5f0 tx=0x7f1efc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:31.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.291+0000 7f7f7a7fc640 1 -- 172.21.15.105:0/1526616838 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c0049e0 con 0x7f7f8415acc0 2024-04-15T15:19:31.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.291+0000 7f7f8c152640 1 -- 172.21.15.105:0/1526616838 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f841502b0 con 0x7f7f8415acc0 2024-04-15T15:19:31.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.292+0000 7f1a7a73d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1a740aff70 0x7f1a74204940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.292+0000 7f1a7a73d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1a74204e80 0x7f1a742013c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.291+0000 7f62c91cf640 1 -- 172.21.15.105:0/1983986801 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62bc035830 msgr2=0x7f62bc177b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.291+0000 7f62c91cf640 1 --2- 172.21.15.105:0/1983986801 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62bc035830 0x7f62bc177b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.291+0000 7f62c91cf640 1 -- 172.21.15.105:0/1983986801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62bc0080f0 msgr2=0x7f62bc0352f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.291+0000 7f62c91cf640 1 --2- 172.21.15.105:0/1983986801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62bc0080f0 0x7f62bc0352f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.291+0000 7f62c91cf640 1 -- 172.21.15.105:0/1983986801 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62bc037a30 con 0x7f62b0004800 2024-04-15T15:19:31.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.291+0000 7f62c91cf640 1 --2- 172.21.15.105:0/1983986801 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62b0004800 0x7f62bc034db0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f62c000e730 tx=0x7f62c000ebf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:31.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.292+0000 7f1a7a73d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1a7406c370 con 0x7f1a5c003a00 2024-04-15T15:19:31.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.292+0000 7f1a7a73d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1a7415d6e0 con 0x7f1a74204e80 2024-04-15T15:19:31.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.292+0000 7f1a737fe640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1a740aff70 0x7f1a74204940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.292+0000 7f1a737fe640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1a740aff70 0x7f1a74204940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.105:57040/0 (socket says 172.21.15.105:57040) 2024-04-15T15:19:31.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.292+0000 7f1a737fe640 1 -- 172.21.15.105:0/3681819607 learned_addr learned my addr 172.21.15.105:0/3681819607 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:31.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.292+0000 7f1a7a73d640 1 -- 172.21.15.105:0/3681819607 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f1a7406b9c0 con 0x7f1a740aff70 2024-04-15T15:19:31.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.292+0000 7f43a77fe640 1 --2- 172.21.15.105:0/205335355 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43a00524e0 0x7f43a0052960 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f439c007310 tx=0x7f439c0077d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:31.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.292+0000 7f1a78cda640 1 --2- 172.21.15.105:0/3681819607 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1a74204e80 0x7f1a742013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.292+0000 7f1a73fff640 1 --2- 172.21.15.105:0/3681819607 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a5c003a00 0x7f1a740aebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.292+0000 7f3cf8ab1640 1 -- 172.21.15.105:0/2149023496 shutdown_connections 2024-04-15T15:19:31.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.292+0000 7f1f057fa640 1 -- 172.21.15.105:0/3563963370 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1efc018020 con 0x7f1f080d0ef0 2024-04-15T15:19:31.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.292+0000 7f43a4ff9640 1 -- 172.21.15.105:0/205335355 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439c01a070 con 0x7f43a00524e0 2024-04-15T15:19:31.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.291+0000 7f448bfff640 1 -- 172.21.15.105:0/3214840278 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f448c034070 con 0x7f449415cb00 2024-04-15T15:19:31.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.292+0000 7f1f057fa640 1 -- 172.21.15.105:0/3563963370 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1efc014620 con 0x7f1f080d0ef0 2024-04-15T15:19:31.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.292+0000 7f1f057fa640 1 -- 172.21.15.105:0/3563963370 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1efc0149d0 con 0x7f1f080d0ef0 2024-04-15T15:19:31.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.292+0000 7f2f4effd640 1 -- 172.21.15.105:0/3482216990 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2f5401a070 con 0x7f2f48004d90 2024-04-15T15:19:31.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.292+0000 7f7f7a7fc640 1 -- 172.21.15.105:0/1526616838 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7f7c01a070 con 0x7f7f8415acc0 2024-04-15T15:19:31.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.292+0000 7f1a737fe640 1 -- 172.21.15.105:0/3681819607 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1a74204e80 msgr2=0x7f1a742013c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.292+0000 7f1a737fe640 1 --2- 172.21.15.105:0/3681819607 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1a74204e80 0x7f1a742013c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.292+0000 7f1a737fe640 1 -- 172.21.15.105:0/3681819607 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a5c003a00 msgr2=0x7f1a740aebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.292+0000 7f1a737fe640 1 --2- 172.21.15.105:0/3681819607 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a5c003a00 0x7f1a740aebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.292+0000 7f1a737fe640 1 -- 172.21.15.105:0/3681819607 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a7415b840 con 0x7f1a740aff70 2024-04-15T15:19:31.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.292+0000 7f1f0ee2a640 1 -- 172.21.15.105:0/3563963370 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f080c9810 con 0x7f1f080d0ef0 2024-04-15T15:19:31.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.292+0000 7f1f0ee2a640 1 -- 172.21.15.105:0/3563963370 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f080c9dd0 con 0x7f1f080d0ef0 2024-04-15T15:19:31.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.293+0000 7f1a737fe640 1 --2- 172.21.15.105:0/3681819607 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1a740aff70 0x7f1a74204940 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1a6400d800 tx=0x7f1a6400dcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:31.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.293+0000 7f1a70ff9640 1 -- 172.21.15.105:0/3681819607 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a6400b6c0 con 0x7f1a740aff70 2024-04-15T15:19:31.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.293+0000 7f1a70ff9640 1 -- 172.21.15.105:0/3681819607 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a64019070 con 0x7f1a740aff70 2024-04-15T15:19:31.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.292+0000 7f62b9ffb640 1 -- 172.21.15.105:0/1983986801 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c000c6c0 con 0x7f62b0004800 2024-04-15T15:19:31.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.292+0000 7f62b9ffb640 1 -- 172.21.15.105:0/1983986801 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62c0018070 con 0x7f62b0004800 2024-04-15T15:19:31.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.293+0000 7f62b9ffb640 1 -- 172.21.15.105:0/1983986801 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c000caa0 con 0x7f62b0004800 2024-04-15T15:19:31.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.292+0000 7f43a4ff9640 1 -- 172.21.15.105:0/205335355 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f439c00f960 con 0x7f43a00524e0 2024-04-15T15:19:31.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.292+0000 7f43ae84a640 1 -- 172.21.15.105:0/205335355 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43a00d37f0 con 0x7f43a00524e0 2024-04-15T15:19:31.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.292+0000 7f43ae84a640 1 -- 172.21.15.105:0/205335355 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43a00d3db0 con 0x7f43a00524e0 2024-04-15T15:19:31.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.293+0000 7f43a4ff9640 1 -- 172.21.15.105:0/205335355 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439c01a070 con 0x7f43a00524e0 2024-04-15T15:19:31.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.291+0000 7f448bfff640 1 -- 172.21.15.105:0/3214840278 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f448c030890 con 0x7f449415cb00 2024-04-15T15:19:31.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.291+0000 7f448bfff640 1 -- 172.21.15.105:0/3214840278 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f448c030ca0 con 0x7f449415cb00 2024-04-15T15:19:31.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.292+0000 7f449b3ba640 1 -- 172.21.15.105:0/3214840278 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f449415cb00 msgr2=0x7f44940cdae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.292+0000 7f449b3ba640 1 --2- 172.21.15.105:0/3214840278 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f449415cb00 0x7f44940cdae0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f449406fe20 tx=0x7f448c02b9d0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.294+0000 7f7f8c152640 1 -- 172.21.15.105:0/1526616838 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f840b27a0 con 0x7f7f8415acc0 2024-04-15T15:19:31.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.294+0000 7f7f7a7fc640 1 --2- 172.21.15.105:0/1526616838 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f7f6806c970 0x7f7f6806ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.293+0000 7f62ca1d1640 1 -- 172.21.15.105:0/1983986801 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62bc037090 con 0x7f62b0004800 2024-04-15T15:19:31.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.294+0000 7f62ca1d1640 1 -- 172.21.15.105:0/1983986801 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f62bc037650 con 0x7f62b0004800 2024-04-15T15:19:31.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.294+0000 7f1a70ff9640 1 -- 172.21.15.105:0/3681819607 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a6400baa0 con 0x7f1a740aff70 2024-04-15T15:19:31.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.294+0000 7f1a7a73d640 1 -- 172.21.15.105:0/3681819607 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a7415bb20 con 0x7f1a740aff70 2024-04-15T15:19:31.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.294+0000 7f7f7a7fc640 1 -- 172.21.15.105:0/1526616838 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7f7c08cbd0 con 0x7f7f8415acc0 2024-04-15T15:19:31.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.294+0000 7f1a7a73d640 1 -- 172.21.15.105:0/3681819607 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a7415c0e0 con 0x7f1a740aff70 2024-04-15T15:19:31.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.292+0000 7f3cf8ab1640 1 -- 172.21.15.105:0/2149023496 wait complete. 2024-04-15T15:19:31.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.294+0000 7f3cf8ab1640 1 Processor -- start 2024-04-15T15:19:31.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.294+0000 7f43ae84a640 1 -- 172.21.15.105:0/205335355 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43a00d3fb0 con 0x7f43a00524e0 2024-04-15T15:19:31.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.294+0000 7f7f89eee640 1 --2- 172.21.15.105:0/1526616838 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f7f6806c970 0x7f7f6806ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.294+0000 7f449b3ba640 1 -- 172.21.15.105:0/3214840278 shutdown_connections 2024-04-15T15:19:31.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.294+0000 7f449b3ba640 1 --2- 172.21.15.105:0/3214840278 >> v2:172.21.15.105:3300/0 conn(0x7f44940ce020 0x7f44940b0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.294+0000 7f449b3ba640 1 --2- 172.21.15.105:0/3214840278 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f449415cb00 0x7f44940cdae0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.294+0000 7f449b3ba640 1 -- 172.21.15.105:0/3214840278 >> 172.21.15.105:0/3214840278 conn(0x7f44940b4040 msgr2=0x7f44940b4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:31.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.293+0000 7f2f4effd640 1 --2- 172.21.15.105:0/3482216990 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f2f4006c8f0 0x7f2f4006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.294+0000 7f1f0ee2a640 1 -- 172.21.15.105:0/3563963370 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f08069150 con 0x7f1f080d0ef0 2024-04-15T15:19:31.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.293+0000 7f2f4effd640 1 -- 172.21.15.105:0/3482216990 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2f5408bab0 con 0x7f2f48004d90 2024-04-15T15:19:31.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.294+0000 7f2f5dd9b640 1 --2- 172.21.15.105:0/3482216990 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f2f4006c8f0 0x7f2f4006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.295+0000 7f7f89eee640 1 --2- 172.21.15.105:0/1526616838 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f7f6806c970 0x7f7f6806ee30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7f80036000 tx=0x7f7f80030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:31.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.295+0000 7f2f5dd9b640 1 --2- 172.21.15.105:0/3482216990 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f2f4006c8f0 0x7f2f4006edb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2f60157380 tx=0x7f2f50008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:31.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.295+0000 7f449b3ba640 1 -- 172.21.15.105:0/3214840278 shutdown_connections 2024-04-15T15:19:31.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.294+0000 7f3cf8ab1640 1 -- start start 2024-04-15T15:19:31.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.295+0000 7f62ca1d1640 1 -- 172.21.15.105:0/1983986801 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62bc0057e0 con 0x7f62b0004800 2024-04-15T15:19:31.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.295+0000 7f1a7a73d640 1 -- 172.21.15.105:0/3681819607 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a74069150 con 0x7f1a740aff70 2024-04-15T15:19:31.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.295+0000 7f449b3ba640 1 -- 172.21.15.105:0/3214840278 wait complete. 2024-04-15T15:19:31.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.295+0000 7f3cf8ab1640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3cdc0039e0 0x7f3cf414e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.296+0000 7f3cf8ab1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3cf40c9640 0x7f3cf40c6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.296+0000 7f3cf8ab1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3cf40c65c0 0x7f3cf40c6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.296+0000 7f3cf1d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3cf40c9640 0x7f3cf40c6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.296+0000 7f3cf259c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3cdc0039e0 0x7f3cf414e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.296+0000 7f3cf259c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3cdc0039e0 0x7f3cf414e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.105:57052/0 (socket says 172.21.15.105:57052) 2024-04-15T15:19:31.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.296+0000 7f3cf1d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3cf40c9640 0x7f3cf40c6080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39692/0 (socket says 172.21.15.105:39692) 2024-04-15T15:19:31.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.296+0000 7f3cf259c640 1 -- 172.21.15.105:0/1692470865 learned_addr learned my addr 172.21.15.105:0/1692470865 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:31.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.296+0000 7f3cf8ab1640 1 -- 172.21.15.105:0/1692470865 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf40ce2e0 con 0x7f3cf40c9640 2024-04-15T15:19:31.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.296+0000 7f3cf8ab1640 1 -- 172.21.15.105:0/1692470865 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf40d1490 con 0x7f3cf40c65c0 2024-04-15T15:19:31.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.296+0000 7f3cf8ab1640 1 -- 172.21.15.105:0/1692470865 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f3cf40ce460 con 0x7f3cdc0039e0 2024-04-15T15:19:31.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.296+0000 7f3cf2d9d640 1 --2- 172.21.15.105:0/1692470865 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3cf40c65c0 0x7f3cf40c6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.296+0000 7f1f057fa640 1 -- 172.21.15.105:0/3563963370 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1efc01b020 con 0x7f1f080d0ef0 2024-04-15T15:19:31.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.296+0000 7f1a70ff9640 1 -- 172.21.15.105:0/3681819607 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1a6400bc40 con 0x7f1a740aff70 2024-04-15T15:19:31.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.296+0000 7f62b9ffb640 1 -- 172.21.15.105:0/1983986801 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f62c000cc40 con 0x7f62b0004800 2024-04-15T15:19:31.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.296+0000 7f3cf2d9d640 1 -- 172.21.15.105:0/1692470865 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3cdc0039e0 msgr2=0x7f3cf414e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.296+0000 7f3cf2d9d640 1 --2- 172.21.15.105:0/1692470865 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3cdc0039e0 0x7f3cf414e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.296+0000 7f3cf2d9d640 1 -- 172.21.15.105:0/1692470865 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3cf40c9640 msgr2=0x7f3cf40c6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.296+0000 7f3cf2d9d640 1 --2- 172.21.15.105:0/1692470865 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3cf40c9640 0x7f3cf40c6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.296+0000 7f3cf2d9d640 1 -- 172.21.15.105:0/1692470865 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cf40c6f80 con 0x7f3cf40c65c0 2024-04-15T15:19:31.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.297+0000 7f3cf1d9b640 1 --2- 172.21.15.105:0/1692470865 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3cf40c9640 0x7f3cf40c6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:19:31.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.297+0000 7f449b3ba640 1 Processor -- start 2024-04-15T15:19:31.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.297+0000 7f1f057fa640 1 --2- 172.21.15.105:0/3563963370 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f1eec06c940 0x7f1eec06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.297+0000 7f62b9ffb640 1 --2- 172.21.15.105:0/1983986801 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f62a806c9f0 0x7f62a806eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.297+0000 7f449b3ba640 1 -- start start 2024-04-15T15:19:31.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.297+0000 7f1a70ff9640 1 --2- 172.21.15.105:0/3681819607 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f1a6006c940 0x7f1a6006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.297+0000 7f1a70ff9640 1 -- 172.21.15.105:0/3681819607 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1a6408bf60 con 0x7f1a740aff70 2024-04-15T15:19:31.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.297+0000 7f1f057fa640 1 -- 172.21.15.105:0/3563963370 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1efc08c320 con 0x7f1f080d0ef0 2024-04-15T15:19:31.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.297+0000 7f62b9ffb640 1 -- 172.21.15.105:0/1983986801 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f62c008b8f0 con 0x7f62b0004800 2024-04-15T15:19:31.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.297+0000 7f62c89ce640 1 --2- 172.21.15.105:0/1983986801 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f62a806c9f0 0x7f62a806eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.297+0000 7f3cf2d9d640 1 --2- 172.21.15.105:0/1692470865 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3cf40c65c0 0x7f3cf40c6a40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4073350 tx=0x7f3ce800e7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:31.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.297+0000 7f1f07fff640 1 --2- 172.21.15.105:0/3563963370 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f1eec06c940 0x7f1eec06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.297+0000 7f1a73fff640 1 --2- 172.21.15.105:0/3681819607 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f1a6006c940 0x7f1a6006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.298+0000 7f449b3ba640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44940ce020 0x7f449414cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.298+0000 7f449b3ba640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44940d0120 0x7f449415cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.298+0000 7f62c89ce640 1 --2- 172.21.15.105:0/1983986801 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f62a806c9f0 0x7f62a806eeb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f62bc03b1c0 tx=0x7f62b4036000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:31.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.298+0000 7f449b3ba640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f449415cb00 0x7f449415d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.298+0000 7f449b3ba640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f44940b0930 con 0x7f44940ce020 2024-04-15T15:19:31.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.298+0000 7f449b3ba640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f44940b07b0 con 0x7f44940d0120 2024-04-15T15:19:31.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.298+0000 7f449b3ba640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f44940b0ab0 con 0x7f449415cb00 2024-04-15T15:19:31.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.298+0000 7f1a73fff640 1 --2- 172.21.15.105:0/3681819607 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f1a6006c940 0x7f1a6006ee00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1a68034000 tx=0x7f1a68007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:31.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.298+0000 7f4499156640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44940ce020 0x7f449414cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.298+0000 7f4498955640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44940d0120 0x7f449415cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.298+0000 7f4499156640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44940ce020 0x7f449414cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39706/0 (socket says 172.21.15.105:39706) 2024-04-15T15:19:31.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.298+0000 7f4499156640 1 -- 172.21.15.105:0/3714460289 learned_addr learned my addr 172.21.15.105:0/3714460289 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:31.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.298+0000 7f1f07fff640 1 --2- 172.21.15.105:0/3563963370 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f1eec06c940 0x7f1eec06ee00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4036000 tx=0x7f1ef403c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:31.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.298+0000 7f4499957640 1 --2- 172.21.15.105:0/3714460289 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f449415cb00 0x7f449415d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.298+0000 7f4499156640 1 -- 172.21.15.105:0/3714460289 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f449415cb00 msgr2=0x7f449415d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.298+0000 7f4499156640 1 --2- 172.21.15.105:0/3714460289 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f449415cb00 0x7f449415d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.298+0000 7f4499156640 1 -- 172.21.15.105:0/3714460289 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44940d0120 msgr2=0x7f449415cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.298+0000 7f4499156640 1 --2- 172.21.15.105:0/3714460289 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44940d0120 0x7f449415cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.298+0000 7f4499156640 1 -- 172.21.15.105:0/3714460289 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4494151430 con 0x7f44940ce020 2024-04-15T15:19:31.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.298+0000 7f4498955640 1 --2- 172.21.15.105:0/3714460289 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44940d0120 0x7f449415cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:19:31.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.298+0000 7f4499957640 1 --2- 172.21.15.105:0/3714460289 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f449415cb00 0x7f449415d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:19:31.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.298+0000 7f4499156640 1 --2- 172.21.15.105:0/3714460289 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44940ce020 0x7f449414cd40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f449000c870 tx=0x7f449000cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:31.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.301+0000 7f2f4effd640 1 -- 172.21.15.105:0/3482216990 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2f54059880 con 0x7f2f48004d90 2024-04-15T15:19:31.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.301+0000 7f3ce2ffd640 1 -- 172.21.15.105:0/1692470865 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce8019070 con 0x7f3cf40c65c0 2024-04-15T15:19:31.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.301+0000 7f7f7a7fc640 1 -- 172.21.15.105:0/1526616838 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7f840b27a0 con 0x7f7f8415acc0 2024-04-15T15:19:31.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.301+0000 7f3ce2ffd640 1 -- 172.21.15.105:0/1692470865 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ce80052e0 con 0x7f3cf40c65c0 2024-04-15T15:19:31.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.301+0000 7f3cf8ab1640 1 -- 172.21.15.105:0/1692470865 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cf40c9b90 con 0x7f3cf40c65c0 2024-04-15T15:19:31.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.301+0000 7f3ce2ffd640 1 -- 172.21.15.105:0/1692470865 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce8005490 con 0x7f3cf40c65c0 2024-04-15T15:19:31.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.301+0000 7f3cf8ab1640 1 -- 172.21.15.105:0/1692470865 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cf40ca150 con 0x7f3cf40c65c0 2024-04-15T15:19:31.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.302+0000 7f3cf8ab1640 1 -- 172.21.15.105:0/1692470865 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cf409be30 con 0x7f3cf40c65c0 2024-04-15T15:19:31.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.302+0000 7f43a4ff9640 1 -- 172.21.15.105:0/205335355 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f439c00fb10 con 0x7f43a00524e0 2024-04-15T15:19:31.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.303+0000 7f43a4ff9640 1 --2- 172.21.15.105:0/205335355 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f438c06ccf0 0x7f438c06f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.303+0000 7f43a4ff9640 1 -- 172.21.15.105:0/205335355 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f439c010040 con 0x7f43a00524e0 2024-04-15T15:19:31.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.303+0000 7f43a7fff640 1 --2- 172.21.15.105:0/205335355 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f438c06ccf0 0x7f438c06f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.304+0000 7f43a7fff640 1 --2- 172.21.15.105:0/205335355 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f438c06ccf0 0x7f438c06f1b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4394006fd0 tx=0x7f4394008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:31.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.307+0000 7f62b9ffb640 1 -- 172.21.15.105:0/1983986801 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f62c0059d90 con 0x7f62b0004800 2024-04-15T15:19:31.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.310+0000 7f4489ffb640 1 -- 172.21.15.105:0/3714460289 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4490003b70 con 0x7f44940ce020 2024-04-15T15:19:31.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.310+0000 7f1f057fa640 1 -- 172.21.15.105:0/3563963370 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1efc022020 con 0x7f1f080d0ef0 2024-04-15T15:19:31.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.310+0000 7f4489ffb640 1 -- 172.21.15.105:0/3714460289 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4490003d10 con 0x7f44940ce020 2024-04-15T15:19:31.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.310+0000 7f449b3ba640 1 -- 172.21.15.105:0/3714460289 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4494151710 con 0x7f44940ce020 2024-04-15T15:19:31.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.310+0000 7f449b3ba640 1 -- 172.21.15.105:0/3714460289 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44940b53c0 con 0x7f44940ce020 2024-04-15T15:19:31.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.311+0000 7f4489ffb640 1 -- 172.21.15.105:0/3714460289 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4490003b70 con 0x7f44940ce020 2024-04-15T15:19:31.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.312+0000 7f43a4ff9640 1 -- 172.21.15.105:0/205335355 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f43a00d3fb0 con 0x7f43a00524e0 2024-04-15T15:19:31.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.312+0000 7f449b3ba640 1 -- 172.21.15.105:0/3714460289 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4494068a40 con 0x7f44940ce020 2024-04-15T15:19:31.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.313+0000 7f4489ffb640 1 -- 172.21.15.105:0/3714460289 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f449000a850 con 0x7f44940ce020 2024-04-15T15:19:31.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.314+0000 7f4489ffb640 1 --2- 172.21.15.105:0/3714460289 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f447806caf0 0x7f447806efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.314+0000 7f4489ffb640 1 -- 172.21.15.105:0/3714460289 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f449008b650 con 0x7f44940ce020 2024-04-15T15:19:31.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.313+0000 7f3ce2ffd640 1 -- 172.21.15.105:0/1692470865 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3ce801c070 con 0x7f3cf40c65c0 2024-04-15T15:19:31.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.314+0000 7f4498955640 1 --2- 172.21.15.105:0/3714460289 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f447806caf0 0x7f447806efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.314+0000 7f3ce2ffd640 1 --2- 172.21.15.105:0/1692470865 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f3cd406c8f0 0x7f3cd406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.314+0000 7f3ce2ffd640 1 -- 172.21.15.105:0/1692470865 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3ce808ad70 con 0x7f3cf40c65c0 2024-04-15T15:19:31.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.314+0000 7f3cf259c640 1 --2- 172.21.15.105:0/1692470865 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f3cd406c8f0 0x7f3cd406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.315+0000 7f3cf259c640 1 --2- 172.21.15.105:0/1692470865 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f3cd406c8f0 0x7f3cd406edb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3cec00a8c0 tx=0x7f3cec02f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:31.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.315+0000 7f4498955640 1 --2- 172.21.15.105:0/3714460289 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f447806caf0 0x7f447806efb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f448c036000 tx=0x7f448c03b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:31.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.319+0000 7f3ce2ffd640 1 -- 172.21.15.105:0/1692470865 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3cf409be30 con 0x7f3cf40c65c0 2024-04-15T15:19:31.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.318+0000 7f1a70ff9640 1 -- 172.21.15.105:0/3681819607 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1a6405a400 con 0x7f1a740aff70 2024-04-15T15:19:31.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.320+0000 7f4489ffb640 1 -- 172.21.15.105:0/3714460289 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f44900593a0 con 0x7f44940ce020 2024-04-15T15:19:31.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.719+0000 7f7f8c152640 1 -- 172.21.15.105:0/1526616838 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f7f841547e0 con 0x7f7f8415acc0 2024-04-15T15:19:31.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.720+0000 7f7f7a7fc640 1 -- 172.21.15.105:0/1526616838 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7f7c058970 con 0x7f7f8415acc0 2024-04-15T15:19:31.722 INFO:teuthology.orchestra.run.smithi105.stdout:55834574855 2024-04-15T15:19:31.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.721+0000 7f62ca1d1640 1 -- 172.21.15.105:0/1983986801 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f62bc0503d0 con 0x7f62b0004800 2024-04-15T15:19:31.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.721+0000 7f62b9ffb640 1 -- 172.21.15.105:0/1983986801 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f62c0059690 con 0x7f62b0004800 2024-04-15T15:19:31.723 INFO:teuthology.orchestra.run.smithi105.stdout:55834574854 2024-04-15T15:19:31.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.723+0000 7f2f5f59e640 1 -- 172.21.15.105:0/3482216990 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f2f600b6050 con 0x7f2f48004d90 2024-04-15T15:19:31.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.723+0000 7f7f8c152640 1 -- 172.21.15.105:0/1526616838 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f7f6806c970 msgr2=0x7f7f6806ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.723+0000 7f7f8c152640 1 --2- 172.21.15.105:0/1526616838 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f7f6806c970 0x7f7f6806ee30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7f80036000 tx=0x7f7f80030000 comp rx=0 tx=0).stop 2024-04-15T15:19:31.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.723+0000 7f2f4effd640 1 -- 172.21.15.105:0/3482216990 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2f5405db80 con 0x7f2f48004d90 2024-04-15T15:19:31.726 INFO:teuthology.orchestra.run.smithi105.stdout:55834574853 2024-04-15T15:19:31.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.724+0000 7f7f8c152640 1 -- 172.21.15.105:0/1526616838 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f8415acc0 msgr2=0x7f7f84151da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.724+0000 7f7f8c152640 1 --2- 172.21.15.105:0/1526616838 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f8415acc0 0x7f7f84151da0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c00c390 tx=0x7f7f7c00c790 comp rx=0 tx=0).stop 2024-04-15T15:19:31.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.724+0000 7f7f8c152640 1 -- 172.21.15.105:0/1526616838 shutdown_connections 2024-04-15T15:19:31.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.724+0000 7f7f8c152640 1 --2- 172.21.15.105:0/1526616838 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f840b3cc0 0x7f7f84151860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.724+0000 7f7f8c152640 1 --2- 172.21.15.105:0/1526616838 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f7f6806c970 0x7f7f6806ee30 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.724+0000 7f7f8c152640 1 --2- 172.21.15.105:0/1526616838 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f8415acc0 0x7f7f84151da0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.724+0000 7f7f8c152640 1 --2- 172.21.15.105:0/1526616838 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f8406bf10 0x7f7f8414e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.725+0000 7f7f8c152640 1 -- 172.21.15.105:0/1526616838 >> 172.21.15.105:0/1526616838 conn(0x7f7f840b34b0 msgr2=0x7f7f8406f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:31.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.725+0000 7f7f8c152640 1 -- 172.21.15.105:0/1526616838 shutdown_connections 2024-04-15T15:19:31.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.725+0000 7f7f8c152640 1 -- 172.21.15.105:0/1526616838 wait complete. 2024-04-15T15:19:31.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.725+0000 7f62ca1d1640 1 -- 172.21.15.105:0/1983986801 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f62a806c9f0 msgr2=0x7f62a806eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.725+0000 7f62ca1d1640 1 --2- 172.21.15.105:0/1983986801 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f62a806c9f0 0x7f62a806eeb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f62bc03b1c0 tx=0x7f62b4036000 comp rx=0 tx=0).stop 2024-04-15T15:19:31.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.725+0000 7f43ae84a640 1 -- 172.21.15.105:0/205335355 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f43a00057e0 con 0x7f43a00524e0 2024-04-15T15:19:31.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.725+0000 7f62ca1d1640 1 -- 172.21.15.105:0/1983986801 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62b0004800 msgr2=0x7f62bc034db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.725+0000 7f62ca1d1640 1 --2- 172.21.15.105:0/1983986801 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62b0004800 0x7f62bc034db0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f62c000e730 tx=0x7f62c000ebf0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.725+0000 7f62ca1d1640 1 -- 172.21.15.105:0/1983986801 shutdown_connections 2024-04-15T15:19:31.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.725+0000 7f62ca1d1640 1 --2- 172.21.15.105:0/1983986801 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f62a806c9f0 0x7f62a806eeb0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.725+0000 7f62ca1d1640 1 --2- 172.21.15.105:0/1983986801 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62bc035830 0x7f62bc177b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.725+0000 7f62ca1d1640 1 --2- 172.21.15.105:0/1983986801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62bc0080f0 0x7f62bc0352f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.725+0000 7f62ca1d1640 1 --2- 172.21.15.105:0/1983986801 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62b0004800 0x7f62bc034db0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.725+0000 7f62ca1d1640 1 -- 172.21.15.105:0/1983986801 >> 172.21.15.105:0/1983986801 conn(0x7f62bc055350 msgr2=0x7f62bc0dd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:31.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.726+0000 7f62ca1d1640 1 -- 172.21.15.105:0/1983986801 shutdown_connections 2024-04-15T15:19:31.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.726+0000 7f62ca1d1640 1 -- 172.21.15.105:0/1983986801 wait complete. 2024-04-15T15:19:31.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.726+0000 7f43a4ff9640 1 -- 172.21.15.105:0/205335355 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f439c023020 con 0x7f43a00524e0 2024-04-15T15:19:31.731 INFO:teuthology.orchestra.run.smithi105.stdout:55834574854 2024-04-15T15:19:31.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.727+0000 7f2f5f59e640 1 -- 172.21.15.105:0/3482216990 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f2f4006c8f0 msgr2=0x7f2f4006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.727+0000 7f2f5f59e640 1 --2- 172.21.15.105:0/3482216990 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f2f4006c8f0 0x7f2f4006edb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2f60157380 tx=0x7f2f50008040 comp rx=0 tx=0).stop 2024-04-15T15:19:31.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.727+0000 7f2f5f59e640 1 -- 172.21.15.105:0/3482216990 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f48004d90 msgr2=0x7f2f600b0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.727+0000 7f2f5f59e640 1 --2- 172.21.15.105:0/3482216990 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f48004d90 0x7f2f600b0140 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2f5400e890 tx=0x7f2f5400ed50 comp rx=0 tx=0).stop 2024-04-15T15:19:31.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.727+0000 7f2f5f59e640 1 -- 172.21.15.105:0/3482216990 shutdown_connections 2024-04-15T15:19:31.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.727+0000 7f2f5f59e640 1 --2- 172.21.15.105:0/3482216990 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2f600b2720 0x7f2f6014d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.727+0000 7f2f5f59e640 1 --2- 172.21.15.105:0/3482216990 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f2f4006c8f0 0x7f2f4006edb0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.727+0000 7f2f5f59e640 1 --2- 172.21.15.105:0/3482216990 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f6009aa40 0x7f2f6009aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.727+0000 7f2f5f59e640 1 --2- 172.21.15.105:0/3482216990 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f48004d90 0x7f2f600b0140 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.728+0000 7f2f5f59e640 1 -- 172.21.15.105:0/3482216990 >> 172.21.15.105:0/3482216990 conn(0x7f2f600b3fe0 msgr2=0x7f2f6006ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:31.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.728+0000 7f2f5f59e640 1 -- 172.21.15.105:0/3482216990 shutdown_connections 2024-04-15T15:19:31.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.728+0000 7f2f5f59e640 1 -- 172.21.15.105:0/3482216990 wait complete. 2024-04-15T15:19:31.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.729+0000 7f43ae84a640 1 -- 172.21.15.105:0/205335355 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f438c06ccf0 msgr2=0x7f438c06f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.729+0000 7f43ae84a640 1 --2- 172.21.15.105:0/205335355 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f438c06ccf0 0x7f438c06f1b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4394006fd0 tx=0x7f4394008040 comp rx=0 tx=0).stop 2024-04-15T15:19:31.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.730+0000 7f43ae84a640 1 -- 172.21.15.105:0/205335355 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43a00524e0 msgr2=0x7f43a0052960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.730+0000 7f43ae84a640 1 --2- 172.21.15.105:0/205335355 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43a00524e0 0x7f43a0052960 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f439c007310 tx=0x7f439c0077d0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.730+0000 7f43ae84a640 1 -- 172.21.15.105:0/205335355 shutdown_connections 2024-04-15T15:19:31.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.730+0000 7f43ae84a640 1 --2- 172.21.15.105:0/205335355 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f438c06ccf0 0x7f438c06f1b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.730+0000 7f43ae84a640 1 --2- 172.21.15.105:0/205335355 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43a00e0120 0x7f43a00e05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.730+0000 7f43ae84a640 1 --2- 172.21.15.105:0/205335355 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43a00524e0 0x7f43a0052960 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.730+0000 7f43ae84a640 1 --2- 172.21.15.105:0/205335355 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f43980051c0 0x7f43a0053c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.730+0000 7f43ae84a640 1 -- 172.21.15.105:0/205335355 >> 172.21.15.105:0/205335355 conn(0x7f43a001ef40 msgr2=0x7f43a0055c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:31.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.730+0000 7f43ae84a640 1 -- 172.21.15.105:0/205335355 shutdown_connections 2024-04-15T15:19:31.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.730+0000 7f43ae84a640 1 -- 172.21.15.105:0/205335355 wait complete. 2024-04-15T15:19:31.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.731+0000 7f1f0ee2a640 1 -- 172.21.15.105:0/3563963370 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f1f080d2540 con 0x7f1f080d0ef0 2024-04-15T15:19:31.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.732+0000 7f1f057fa640 1 -- 172.21.15.105:0/3563963370 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1efc05a830 con 0x7f1f080d0ef0 2024-04-15T15:19:31.736 INFO:teuthology.orchestra.run.smithi105.stdout:55834574854 2024-04-15T15:19:31.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.736+0000 7f3cf8ab1640 1 -- 172.21.15.105:0/1692470865 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f3cf4075720 con 0x7f3cf40c65c0 2024-04-15T15:19:31.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.737+0000 7f1f0ee2a640 1 -- 172.21.15.105:0/3563963370 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f1eec06c940 msgr2=0x7f1eec06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.737+0000 7f1f0ee2a640 1 --2- 172.21.15.105:0/3563963370 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f1eec06c940 0x7f1eec06ee00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4036000 tx=0x7f1ef403c040 comp rx=0 tx=0).stop 2024-04-15T15:19:31.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.737+0000 7f1f0ee2a640 1 -- 172.21.15.105:0/3563963370 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f080d0ef0 msgr2=0x7f1f08161b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.737+0000 7f1f0ee2a640 1 --2- 172.21.15.105:0/3563963370 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f080d0ef0 0x7f1f08161b80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1efc00d5f0 tx=0x7f1efc00dab0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.737+0000 7f3ce2ffd640 1 -- 172.21.15.105:0/1692470865 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3cf4075720 con 0x7f3cf40c65c0 2024-04-15T15:19:31.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.738+0000 7f1f0ee2a640 1 -- 172.21.15.105:0/3563963370 shutdown_connections 2024-04-15T15:19:31.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.738+0000 7f1f0ee2a640 1 --2- 172.21.15.105:0/3563963370 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f0815e9d0 0x7f1f08159d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.738+0000 7f1f0ee2a640 1 --2- 172.21.15.105:0/3563963370 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f1eec06c940 0x7f1eec06ee00 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.738+0000 7f1f0ee2a640 1 --2- 172.21.15.105:0/3563963370 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f08163130 0x7f1f080c8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.738+0000 7f1f0ee2a640 1 --2- 172.21.15.105:0/3563963370 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f080d0ef0 0x7f1f08161b80 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.738+0000 7f1f0ee2a640 1 -- 172.21.15.105:0/3563963370 >> 172.21.15.105:0/3563963370 conn(0x7f1f08067720 msgr2=0x7f1f0815d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:31.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.738+0000 7f1f0ee2a640 1 -- 172.21.15.105:0/3563963370 shutdown_connections 2024-04-15T15:19:31.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.738+0000 7f1f0ee2a640 1 -- 172.21.15.105:0/3563963370 wait complete. 2024-04-15T15:19:31.743 INFO:teuthology.orchestra.run.smithi105.stdout:55834574855 2024-04-15T15:19:31.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.742+0000 7f3cf8ab1640 1 -- 172.21.15.105:0/1692470865 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f3cd406c8f0 msgr2=0x7f3cd406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.742+0000 7f3cf8ab1640 1 --2- 172.21.15.105:0/1692470865 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f3cd406c8f0 0x7f3cd406edb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3cec00a8c0 tx=0x7f3cec02f000 comp rx=0 tx=0).stop 2024-04-15T15:19:31.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.742+0000 7f3cf8ab1640 1 -- 172.21.15.105:0/1692470865 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3cf40c65c0 msgr2=0x7f3cf40c6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.742+0000 7f3cf8ab1640 1 --2- 172.21.15.105:0/1692470865 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3cf40c65c0 0x7f3cf40c6a40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4073350 tx=0x7f3ce800e7e0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.742+0000 7f3cf8ab1640 1 -- 172.21.15.105:0/1692470865 shutdown_connections 2024-04-15T15:19:31.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.742+0000 7f3cf8ab1640 1 --2- 172.21.15.105:0/1692470865 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f3cd406c8f0 0x7f3cd406edb0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.742+0000 7f3cf8ab1640 1 --2- 172.21.15.105:0/1692470865 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3cf40c65c0 0x7f3cf40c6a40 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.742+0000 7f3cf8ab1640 1 --2- 172.21.15.105:0/1692470865 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3cf40c9640 0x7f3cf40c6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.742+0000 7f3cf8ab1640 1 --2- 172.21.15.105:0/1692470865 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3cdc0039e0 0x7f3cf414e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.742+0000 7f3cf8ab1640 1 -- 172.21.15.105:0/1692470865 >> 172.21.15.105:0/1692470865 conn(0x7f3cf40d2570 msgr2=0x7f3cf4071960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:31.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.742+0000 7f1a7a73d640 1 -- 172.21.15.105:0/3681819607 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f1a74158fa0 con 0x7f1a740aff70 2024-04-15T15:19:31.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.743+0000 7f3cf8ab1640 1 -- 172.21.15.105:0/1692470865 shutdown_connections 2024-04-15T15:19:31.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.743+0000 7f3cf8ab1640 1 -- 172.21.15.105:0/1692470865 wait complete. 2024-04-15T15:19:31.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.743+0000 7f1a70ff9640 1 -- 172.21.15.105:0/3681819607 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1a64059d00 con 0x7f1a740aff70 2024-04-15T15:19:31.748 INFO:teuthology.orchestra.run.smithi105.stdout:55834574853 2024-04-15T15:19:31.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.746+0000 7f449b3ba640 1 -- 172.21.15.105:0/3714460289 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f44940d0840 con 0x7f44940ce020 2024-04-15T15:19:31.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.747+0000 7f1a7a73d640 1 -- 172.21.15.105:0/3681819607 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f1a6006c940 msgr2=0x7f1a6006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.747+0000 7f1a7a73d640 1 --2- 172.21.15.105:0/3681819607 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f1a6006c940 0x7f1a6006ee00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1a68034000 tx=0x7f1a68007000 comp rx=0 tx=0).stop 2024-04-15T15:19:31.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.747+0000 7f1a7a73d640 1 -- 172.21.15.105:0/3681819607 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1a740aff70 msgr2=0x7f1a74204940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.747+0000 7f1a7a73d640 1 --2- 172.21.15.105:0/3681819607 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1a740aff70 0x7f1a74204940 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1a6400d800 tx=0x7f1a6400dcc0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.747+0000 7f1a7a73d640 1 -- 172.21.15.105:0/3681819607 shutdown_connections 2024-04-15T15:19:31.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.747+0000 7f1a7a73d640 1 --2- 172.21.15.105:0/3681819607 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1a740aff70 0x7f1a74204940 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.747+0000 7f1a7a73d640 1 --2- 172.21.15.105:0/3681819607 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f1a6006c940 0x7f1a6006ee00 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.747+0000 7f1a7a73d640 1 --2- 172.21.15.105:0/3681819607 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1a74204e80 0x7f1a742013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.747+0000 7f1a7a73d640 1 --2- 172.21.15.105:0/3681819607 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a5c003a00 0x7f1a740aebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.750 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.5 2024-04-15T15:19:31.750 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.6 2024-04-15T15:19:31.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.747+0000 7f1a7a73d640 1 -- 172.21.15.105:0/3681819607 >> 172.21.15.105:0/3681819607 conn(0x7f1a740b34b0 msgr2=0x7f1a74074a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:31.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.747+0000 7f1a7a73d640 1 -- 172.21.15.105:0/3681819607 shutdown_connections 2024-04-15T15:19:31.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.747+0000 7f1a7a73d640 1 -- 172.21.15.105:0/3681819607 wait complete. 2024-04-15T15:19:31.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.747+0000 7f4489ffb640 1 -- 172.21.15.105:0/3714460289 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f449005d6a0 con 0x7f44940ce020 2024-04-15T15:19:31.752 INFO:teuthology.orchestra.run.smithi105.stdout:55834574854 2024-04-15T15:19:31.753 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.7 2024-04-15T15:19:31.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.751+0000 7f449b3ba640 1 -- 172.21.15.105:0/3714460289 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f447806caf0 msgr2=0x7f447806efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.751+0000 7f449b3ba640 1 --2- 172.21.15.105:0/3714460289 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f447806caf0 0x7f447806efb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f448c036000 tx=0x7f448c03b040 comp rx=0 tx=0).stop 2024-04-15T15:19:31.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.751+0000 7f449b3ba640 1 -- 172.21.15.105:0/3714460289 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44940ce020 msgr2=0x7f449414cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.751+0000 7f449b3ba640 1 --2- 172.21.15.105:0/3714460289 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44940ce020 0x7f449414cd40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f449000c870 tx=0x7f449000cd30 comp rx=0 tx=0).stop 2024-04-15T15:19:31.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.751+0000 7f449b3ba640 1 -- 172.21.15.105:0/3714460289 shutdown_connections 2024-04-15T15:19:31.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.751+0000 7f449b3ba640 1 --2- 172.21.15.105:0/3714460289 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f449415cb00 0x7f449415d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.751+0000 7f449b3ba640 1 --2- 172.21.15.105:0/3714460289 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f447806caf0 0x7f447806efb0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.751+0000 7f449b3ba640 1 --2- 172.21.15.105:0/3714460289 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44940d0120 0x7f449415cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.751+0000 7f449b3ba640 1 --2- 172.21.15.105:0/3714460289 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44940ce020 0x7f449414cd40 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.751+0000 7f449b3ba640 1 -- 172.21.15.105:0/3714460289 >> 172.21.15.105:0/3714460289 conn(0x7f44940b4040 msgr2=0x7f44940ce5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:31.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.751+0000 7f449b3ba640 1 -- 172.21.15.105:0/3714460289 shutdown_connections 2024-04-15T15:19:31.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:31.751+0000 7f449b3ba640 1 -- 172.21.15.105:0/3714460289 wait complete. 2024-04-15T15:19:31.755 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.1 2024-04-15T15:19:31.759 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.0 2024-04-15T15:19:31.764 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.4 2024-04-15T15:19:31.764 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.2 2024-04-15T15:19:31.768 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.3 2024-04-15T15:19:32.752 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-15T15:19:32.752 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-15T15:19:32.753 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-15T15:19:32.756 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-15T15:19:32.760 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-15T15:19:32.764 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-15T15:19:32.766 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-15T15:19:32.769 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-15T15:19:32.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.916+0000 7fefa76ba640 1 Processor -- start 2024-04-15T15:19:32.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.916+0000 7fefa76ba640 1 -- start start 2024-04-15T15:19:32.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.916+0000 7f6b89a2a640 1 Processor -- start 2024-04-15T15:19:32.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.916+0000 7f6b89a2a640 1 -- start start 2024-04-15T15:19:32.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.918+0000 7f6b89a2a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6b7c104970 0x7f6b7c104e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:32.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.918+0000 7f6b89a2a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6b7c053680 0x7f6b7c053b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:32.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.918+0000 7f6b89a2a640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b7c037b50 con 0x7f6b7c100cf0 2024-04-15T15:19:32.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.918+0000 7f6b89a2a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f6b7c03aae0 con 0x7f6b7c104970 2024-04-15T15:19:32.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.918+0000 7f6b89a2a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f6b7c007340 con 0x7f6b7c053680 2024-04-15T15:19:32.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.918+0000 7f6b827fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6b7c104970 0x7f6b7c104e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:32.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.918+0000 7f6b827fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6b7c104970 0x7f6b7c104e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39716/0 (socket says 172.21.15.105:39716) 2024-04-15T15:19:32.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.918+0000 7f6b827fc640 1 -- 172.21.15.105:0/1742955502 learned_addr learned my addr 172.21.15.105:0/1742955502 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:32.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.918+0000 7f6b837fe640 1 --2- 172.21.15.105:0/1742955502 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6b7c053680 0x7f6b7c053b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:32.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.919+0000 7f6b827fc640 1 -- 172.21.15.105:0/1742955502 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6b7c053680 msgr2=0x7f6b7c053b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:32.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.919+0000 7f6b827fc640 1 --2- 172.21.15.105:0/1742955502 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6b7c053680 0x7f6b7c053b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:32.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.919+0000 7f6b827fc640 1 -- 172.21.15.105:0/1742955502 >> v1:172.21.15.154:6789/0 conn(0x7f6b7c100cf0 legacy=0x7f6b7c03a3d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-15T15:19:32.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.919+0000 7f6b827fc640 1 -- 172.21.15.105:0/1742955502 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b7c007620 con 0x7f6b7c104970 2024-04-15T15:19:32.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.920+0000 7f6b827fc640 1 --2- 172.21.15.105:0/1742955502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b7c104970 0x7f6b7c104e00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6b84069a20 tx=0x7f6b84096ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f86b626ec28fb0a server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:32.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.920+0000 7f6b81ffb640 1 -- 172.21.15.105:0/1742955502 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8409d070 con 0x7f6b7c104970 2024-04-15T15:19:32.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.920+0000 7f6b81ffb640 1 -- 172.21.15.105:0/1742955502 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b84097840 con 0x7f6b7c104970 2024-04-15T15:19:32.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.921+0000 7f6b89a2a640 1 -- 172.21.15.105:0/1742955502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b7c104970 msgr2=0x7f6b7c104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:32.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.921+0000 7f6b89a2a640 1 --2- 172.21.15.105:0/1742955502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b7c104970 0x7f6b7c104e00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6b84069a20 tx=0x7f6b84096ba0 comp rx=0 tx=0).stop 2024-04-15T15:19:32.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.918+0000 7fefa76ba640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fefa016b4d0 0x7fefa00ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:32.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.918+0000 7fefa76ba640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fefa016b8b0 0x7fefa006de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:32.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.918+0000 7fefa76ba640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefa006e740 con 0x7fefa00cf3f0 2024-04-15T15:19:32.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.918+0000 7fefa76ba640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fefa006f070 con 0x7fefa016b4d0 2024-04-15T15:19:32.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.918+0000 7fefa76ba640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fefa006ea20 con 0x7fefa016b8b0 2024-04-15T15:19:32.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.919+0000 7fefa4c55640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fefa016b8b0 0x7fefa006de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:32.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.919+0000 7fefa4c55640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fefa016b8b0 0x7fefa006de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.105:57060/0 (socket says 172.21.15.105:57060) 2024-04-15T15:19:32.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.919+0000 7fefa4c55640 1 -- 172.21.15.105:0/999186460 learned_addr learned my addr 172.21.15.105:0/999186460 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:32.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.919+0000 7fefa4c55640 1 -- 172.21.15.105:0/999186460 >> v2:172.21.15.154:3300/0 conn(0x7fefa016b4d0 msgr2=0x7fefa00ceeb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-15T15:19:32.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.919+0000 7fefa4c55640 1 --2- 172.21.15.105:0/999186460 >> v2:172.21.15.154:3300/0 conn(0x7fefa016b4d0 0x7fefa00ceeb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:32.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.919+0000 7fefa4c55640 1 -- 172.21.15.105:0/999186460 >> v1:172.21.15.154:6789/0 conn(0x7fefa00cf3f0 legacy=0x7fefa0072280 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-15T15:19:32.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.919+0000 7fefa4c55640 1 -- 172.21.15.105:0/999186460 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefa006ed00 con 0x7fefa016b8b0 2024-04-15T15:19:32.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.922+0000 7f6b89a2a640 1 -- 172.21.15.105:0/1742955502 shutdown_connections 2024-04-15T15:19:32.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.922+0000 7f6b89a2a640 1 --2- 172.21.15.105:0/1742955502 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6b7c053680 0x7f6b7c053b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:32.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.922+0000 7f6b89a2a640 1 --2- 172.21.15.105:0/1742955502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b7c104970 0x7f6b7c104e00 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:32.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.922+0000 7f6b89a2a640 1 -- 172.21.15.105:0/1742955502 >> 172.21.15.105:0/1742955502 conn(0x7f6b7c009fe0 msgr2=0x7f6b7c00a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:32.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.921+0000 7fefa4c55640 1 --2- 172.21.15.105:0/999186460 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fefa016b8b0 0x7fefa006de80 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fef98014a00 tx=0x7fef98034aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2480301625a9d119 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:32.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.921+0000 7fef97fff640 1 -- 172.21.15.105:0/999186460 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9803c070 con 0x7fefa016b8b0 2024-04-15T15:19:32.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.921+0000 7fef97fff640 1 -- 172.21.15.105:0/999186460 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef980114e0 con 0x7fefa016b8b0 2024-04-15T15:19:32.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.922+0000 7fefa76ba640 1 -- 172.21.15.105:0/999186460 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fefa016b8b0 msgr2=0x7fefa006de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:32.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.922+0000 7fefa76ba640 1 --2- 172.21.15.105:0/999186460 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fefa016b8b0 0x7fefa006de80 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fef98014a00 tx=0x7fef98034aa0 comp rx=0 tx=0).stop 2024-04-15T15:19:32.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.922+0000 7fefa76ba640 1 -- 172.21.15.105:0/999186460 shutdown_connections 2024-04-15T15:19:32.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.922+0000 7fefa76ba640 1 --2- 172.21.15.105:0/999186460 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fefa016b8b0 0x7fefa006de80 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:32.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.922+0000 7fefa76ba640 1 --2- 172.21.15.105:0/999186460 >> v2:172.21.15.154:3300/0 conn(0x7fefa016b4d0 0x7fefa00ceeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:32.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.922+0000 7fefa76ba640 1 -- 172.21.15.105:0/999186460 >> 172.21.15.105:0/999186460 conn(0x7fefa006bde0 msgr2=0x7fefa006c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:32.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.923+0000 7fefa76ba640 1 -- 172.21.15.105:0/999186460 shutdown_connections 2024-04-15T15:19:32.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.923+0000 7f6b89a2a640 1 -- 172.21.15.105:0/1742955502 shutdown_connections 2024-04-15T15:19:32.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.923+0000 7f6b89a2a640 1 -- 172.21.15.105:0/1742955502 wait complete. 2024-04-15T15:19:32.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.925+0000 7f6b89a2a640 1 Processor -- start 2024-04-15T15:19:32.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.925+0000 7f6b89a2a640 1 -- start start 2024-04-15T15:19:32.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.926+0000 7f6b89a2a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b7c053680 0x7f6b7c0f4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:32.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.926+0000 7f6b89a2a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6b7c100cf0 0x7f6b7c04cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:32.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.926+0000 7f6b89a2a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b7c104970 0x7f6b7c04d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:32.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.926+0000 7f6b89a2a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7c1010d0 con 0x7f6b7c053680 2024-04-15T15:19:32.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.926+0000 7f6b89a2a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7c053160 con 0x7f6b7c104970 2024-04-15T15:19:32.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.926+0000 7f6b89a2a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f6b7c101250 con 0x7f6b7c100cf0 2024-04-15T15:19:32.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.927+0000 7f6b82ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b7c053680 0x7f6b7c0f4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:32.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.927+0000 7f6b82ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b7c053680 0x7f6b7c0f4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39728/0 (socket says 172.21.15.105:39728) 2024-04-15T15:19:32.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.927+0000 7f6b82ffd640 1 -- 172.21.15.105:0/25538698 learned_addr learned my addr 172.21.15.105:0/25538698 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:32.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.927+0000 7f6b827fc640 1 --2- 172.21.15.105:0/25538698 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6b7c100cf0 0x7f6b7c04cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:32.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.923+0000 7fefa76ba640 1 -- 172.21.15.105:0/999186460 wait complete. 2024-04-15T15:19:32.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.925+0000 7fefa76ba640 1 Processor -- start 2024-04-15T15:19:32.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.925+0000 7fefa76ba640 1 -- start start 2024-04-15T15:19:32.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.926+0000 7fefa76ba640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fefa00cf3f0 0x7fefa008ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:32.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.926+0000 7fefa76ba640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fefa00a94a0 0x7fefa00c7460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:32.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.927+0000 7f6b82ffd640 1 -- 172.21.15.105:0/25538698 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6b7c100cf0 msgr2=0x7f6b7c04cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:32.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.927+0000 7f6b82ffd640 1 --2- 172.21.15.105:0/25538698 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6b7c100cf0 0x7f6b7c04cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:32.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.927+0000 7f6b82ffd640 1 -- 172.21.15.105:0/25538698 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b7c104970 msgr2=0x7f6b7c04d0d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-15T15:19:32.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.927+0000 7f6b82ffd640 1 --2- 172.21.15.105:0/25538698 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b7c104970 0x7f6b7c04d0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:32.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.927+0000 7f6b82ffd640 1 -- 172.21.15.105:0/25538698 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b7c04d810 con 0x7f6b7c053680 2024-04-15T15:19:32.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.928+0000 7f6b82ffd640 1 --2- 172.21.15.105:0/25538698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b7c053680 0x7f6b7c0f4e50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6b7400b470 tx=0x7f6b7400b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:32.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.926+0000 7fefa76ba640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefa00c79a0 0x7fefa00c7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:32.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.926+0000 7fefa76ba640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fefa006f500 con 0x7fefa00c79a0 2024-04-15T15:19:32.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.926+0000 7fefa76ba640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fefa0072be0 con 0x7fefa00a94a0 2024-04-15T15:19:32.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.926+0000 7fefa76ba640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fefa006f680 con 0x7fefa00cf3f0 2024-04-15T15:19:32.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.926+0000 7fefa4c55640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fefa00a94a0 0x7fefa00c7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:32.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.927+0000 7fefa4c55640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fefa00a94a0 0x7fefa00c7460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.105:45104/0 (socket says 172.21.15.105:45104) 2024-04-15T15:19:32.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.927+0000 7fefa4c55640 1 -- 172.21.15.105:0/3141210596 learned_addr learned my addr 172.21.15.105:0/3141210596 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:32.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.928+0000 7f6b73fff640 1 -- 172.21.15.105:0/25538698 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b74015020 con 0x7f6b7c053680 2024-04-15T15:19:32.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.927+0000 7fefa5c57640 1 --2- 172.21.15.105:0/3141210596 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefa00c79a0 0x7fefa00c7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:32.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.927+0000 7fefa5c57640 1 -- 172.21.15.105:0/3141210596 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fefa00cf3f0 msgr2=0x7fefa008ab30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-15T15:19:32.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.927+0000 7fefa5c57640 1 --2- 172.21.15.105:0/3141210596 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fefa00cf3f0 0x7fefa008ab30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:32.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.927+0000 7fefa5c57640 1 -- 172.21.15.105:0/3141210596 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fefa00a94a0 msgr2=0x7fefa00c7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:32.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.927+0000 7fefa5c57640 1 --2- 172.21.15.105:0/3141210596 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fefa00a94a0 0x7fefa00c7460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:32.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.927+0000 7fefa5c57640 1 -- 172.21.15.105:0/3141210596 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefa00a9960 con 0x7fefa00c79a0 2024-04-15T15:19:32.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.928+0000 7fefa5c57640 1 --2- 172.21.15.105:0/3141210596 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefa00c79a0 0x7fefa00c7e30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fef9c00b410 tx=0x7fef9c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:32.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.928+0000 7fef95ffb640 1 -- 172.21.15.105:0/3141210596 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9c015020 con 0x7fefa00c79a0 2024-04-15T15:19:32.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.928+0000 7f6b89a2a640 1 -- 172.21.15.105:0/25538698 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b7c04daf0 con 0x7f6b7c053680 2024-04-15T15:19:32.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.929+0000 7fefa76ba640 1 -- 172.21.15.105:0/3141210596 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefa01592c0 con 0x7fefa00c79a0 2024-04-15T15:19:32.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.929+0000 7fefa76ba640 1 -- 172.21.15.105:0/3141210596 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefa01684e0 con 0x7fefa00c79a0 2024-04-15T15:19:32.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.930+0000 7fef95ffb640 1 -- 172.21.15.105:0/3141210596 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef9c011850 con 0x7fefa00c79a0 2024-04-15T15:19:32.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.930+0000 7fef95ffb640 1 -- 172.21.15.105:0/3141210596 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9c011c40 con 0x7fefa00c79a0 2024-04-15T15:19:32.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.928+0000 7f6b89a2a640 1 -- 172.21.15.105:0/25538698 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b7c04e0b0 con 0x7f6b7c053680 2024-04-15T15:19:32.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.933+0000 7fefa76ba640 1 -- 172.21.15.105:0/3141210596 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefa0068a40 con 0x7fefa00c79a0 2024-04-15T15:19:32.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.930+0000 7f6b73fff640 1 -- 172.21.15.105:0/25538698 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b74011890 con 0x7f6b7c053680 2024-04-15T15:19:32.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.930+0000 7f6b73fff640 1 -- 172.21.15.105:0/25538698 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b74011ca0 con 0x7f6b7c053680 2024-04-15T15:19:32.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.933+0000 7f6b89a2a640 1 -- 172.21.15.105:0/25538698 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b60005290 con 0x7f6b7c053680 2024-04-15T15:19:32.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.936+0000 7f6b73fff640 1 -- 172.21.15.105:0/25538698 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6b74018020 con 0x7f6b7c053680 2024-04-15T15:19:32.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.937+0000 7f6b73fff640 1 --2- 172.21.15.105:0/25538698 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f6b6806c9c0 0x7f6b6806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:32.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.937+0000 7f6b827fc640 1 --2- 172.21.15.105:0/25538698 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f6b6806c9c0 0x7f6b6806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:32.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.938+0000 7f6b827fc640 1 --2- 172.21.15.105:0/25538698 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f6b6806c9c0 0x7f6b6806ee80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c037d20 tx=0x7f6b8409d000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:32.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.938+0000 7f6b73fff640 1 -- 172.21.15.105:0/25538698 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6b7408bc70 con 0x7f6b7c053680 2024-04-15T15:19:32.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.934+0000 7fef95ffb640 1 -- 172.21.15.105:0/3141210596 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fef9c018020 con 0x7fefa00c79a0 2024-04-15T15:19:32.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.935+0000 7fef95ffb640 1 --2- 172.21.15.105:0/3141210596 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fef8406c9f0 0x7fef8406eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:32.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.935+0000 7fefa5456640 1 --2- 172.21.15.105:0/3141210596 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fef8406c9f0 0x7fef8406eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:32.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.935+0000 7fef95ffb640 1 -- 172.21.15.105:0/3141210596 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fef9c08b8f0 con 0x7fefa00c79a0 2024-04-15T15:19:32.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.936+0000 7fefa5456640 1 --2- 172.21.15.105:0/3141210596 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fef8406c9f0 0x7fef8406eeb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fefa0071840 tx=0x7fef90006bd0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:32.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.943+0000 7fef95ffb640 1 -- 172.21.15.105:0/3141210596 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fef9c01f020 con 0x7fefa00c79a0 2024-04-15T15:19:32.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.952+0000 7f6b73fff640 1 -- 172.21.15.105:0/25538698 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6b7401f020 con 0x7f6b7c053680 2024-04-15T15:19:32.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.973+0000 7f5a570f7640 1 Processor -- start 2024-04-15T15:19:32.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.973+0000 7f5a570f7640 1 -- start start 2024-04-15T15:19:32.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.973+0000 7ff0b4552640 1 Processor -- start 2024-04-15T15:19:32.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.973+0000 7ff0b4552640 1 -- start start 2024-04-15T15:19:32.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.974+0000 7f4ef93a1640 1 Processor -- start 2024-04-15T15:19:32.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.974+0000 7f4ef93a1640 1 -- start start 2024-04-15T15:19:32.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.975+0000 7f5a570f7640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5a480e12b0 0x7f5a480534f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:32.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.975+0000 7f5a570f7640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5a480e4d30 0x7f5a48053a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:32.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.975+0000 7f5a570f7640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a48054fe0 con 0x7f5a48054b50 2024-04-15T15:19:32.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.975+0000 7f5a570f7640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f5a480552c0 con 0x7f5a480e12b0 2024-04-15T15:19:32.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.975+0000 7f5a570f7640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f5a480555a0 con 0x7f5a480e4d30 2024-04-15T15:19:32.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.975+0000 7ff0b4552640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0ac1637a0 0x7ff0ac165b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:32.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.975+0000 7f5a568f6640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5a480e4d30 0x7f5a48053a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:32.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.975+0000 7f5a560f5640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5a480e12b0 0x7f5a480534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:32.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.975+0000 7ff0b22ee640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7ff0ac0b4b20 0x7ff0ac06e350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41032/0 (socket says 172.21.15.105:41032) 2024-04-15T15:19:32.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.975+0000 7ff0b22ee640 1 -- 172.21.15.105:0/4140810948 learned_addr learned my addr 172.21.15.105:0/4140810948 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:19:32.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.975+0000 7ff0b4552640 1 -- 172.21.15.105:0/4140810948 --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0ac15bde0 con 0x7ff0ac0b4b20 2024-04-15T15:19:32.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.976+0000 7ff0b4552640 1 -- 172.21.15.105:0/4140810948 --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0ac06f170 con 0x7ff0ac15b8b0 2024-04-15T15:19:32.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.976+0000 7ff0b4552640 1 -- 172.21.15.105:0/4140810948 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7ff0ac15e550 con 0x7ff0ac1637a0 2024-04-15T15:19:32.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.975+0000 7f5a568f6640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5a480e4d30 0x7f5a48053a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.105:45118/0 (socket says 172.21.15.105:45118) 2024-04-15T15:19:32.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.975+0000 7f5a560f5640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5a480e12b0 0x7f5a480534f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39740/0 (socket says 172.21.15.105:39740) 2024-04-15T15:19:32.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.975+0000 7f5a568f6640 1 -- 172.21.15.105:0/1076243679 learned_addr learned my addr 172.21.15.105:0/1076243679 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:32.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.976+0000 7f4ef93a1640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4eec054b50 0x7f4eec051af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:32.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.976+0000 7f5a560f5640 1 -- 172.21.15.105:0/1076243679 >> v2:172.21.15.154:3300/0 conn(0x7f5a480e4d30 msgr2=0x7f5a48053a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:32.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.976+0000 7f5a560f5640 1 --2- 172.21.15.105:0/1076243679 >> v2:172.21.15.154:3300/0 conn(0x7f5a480e4d30 0x7f5a48053a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:32.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.976+0000 7f5a560f5640 1 -- 172.21.15.105:0/1076243679 >> v1:172.21.15.154:6789/0 conn(0x7f5a48054b50 legacy=0x7f5a480e1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:32.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.976+0000 7f4ef3fff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f4eec0e12b0 0x7f4eec0534f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.105:36864/0 (socket says 172.21.15.105:36864) 2024-04-15T15:19:32.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.976+0000 7f5a560f5640 1 -- 172.21.15.105:0/1076243679 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a48055880 con 0x7f5a480e12b0 2024-04-15T15:19:32.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.976+0000 7f4ef3fff640 1 -- 172.21.15.105:0/1885503676 learned_addr learned my addr 172.21.15.105:0/1885503676 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:19:32.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.976+0000 7f5a550f3640 1 -- 172.21.15.105:0/1076243679 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 315441031 0 0) 0x7f5a48054fe0 con 0x7f5a48054b50 2024-04-15T15:19:32.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.976+0000 7ff0b2aef640 1 --2- 172.21.15.105:0/4140810948 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0ac1637a0 0x7ff0ac165b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:32.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.976+0000 7f4ef37fe640 1 --2- 172.21.15.105:0/1885503676 >> v2:172.21.15.105:3300/0 conn(0x7f4eec054b50 0x7f4eec051af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:32.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.976+0000 7f4ef93a1640 1 -- 172.21.15.105:0/1885503676 --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4eec054fe0 con 0x7f4eec053c00 2024-04-15T15:19:32.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.976+0000 7f4ef93a1640 1 -- 172.21.15.105:0/1885503676 --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4eec0552c0 con 0x7f4eec0e12b0 2024-04-15T15:19:32.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.976+0000 7f4ef93a1640 1 -- 172.21.15.105:0/1885503676 --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f4eec0e26a0 con 0x7f4eec054b50 2024-04-15T15:19:32.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.976+0000 7ff0b2aef640 1 -- 172.21.15.105:0/4140810948 >> v1:172.21.15.154:6789/0 conn(0x7ff0ac15b8b0 legacy=0x7ff0ac06ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:32.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.976+0000 7f4ef2ffd640 1 -- 172.21.15.105:0/1885503676 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3223604938 0 0) 0x7f4eec0552c0 con 0x7f4eec0e12b0 2024-04-15T15:19:32.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.976+0000 7f4ef2ffd640 1 -- 172.21.15.105:0/1885503676 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4ee4003520 con 0x7f4eec0e12b0 2024-04-15T15:19:32.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.976+0000 7ff0b12ec640 1 -- 172.21.15.105:0/4140810948 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3506270703 0 0) 0x7ff0ac15bde0 con 0x7ff0ac0b4b20 2024-04-15T15:19:32.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.976+0000 7ff0b2aef640 1 -- 172.21.15.105:0/4140810948 >> v1:172.21.15.105:6789/0 conn(0x7ff0ac0b4b20 legacy=0x7ff0ac06e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:32.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.976+0000 7ff0b2aef640 1 -- 172.21.15.105:0/4140810948 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0ac070650 con 0x7ff0ac1637a0 2024-04-15T15:19:32.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.976+0000 7fbc8d874640 1 Processor -- start 2024-04-15T15:19:32.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.976+0000 7fbc8d874640 1 -- start start 2024-04-15T15:19:32.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.977+0000 7f5a560f5640 1 --2- 172.21.15.105:0/1076243679 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a480e12b0 0x7f5a480534f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5a4800bfb0 tx=0x7f5a4002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f84f7f0374fd525a server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:32.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.977+0000 7f4ef2ffd640 1 -- 172.21.15.105:0/1885503676 <== mon.1 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 381014856 0 0) 0x7f4ee4003520 con 0x7f4eec0e12b0 2024-04-15T15:19:32.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.977+0000 7f4ef2ffd640 1 -- 172.21.15.105:0/1885503676 >> v2:172.21.15.105:3300/0 conn(0x7f4eec054b50 msgr2=0x7f4eec051af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:32.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.977+0000 7f4ef2ffd640 1 --2- 172.21.15.105:0/1885503676 >> v2:172.21.15.105:3300/0 conn(0x7f4eec054b50 0x7f4eec051af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:32.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.977+0000 7f4ef2ffd640 1 -- 172.21.15.105:0/1885503676 >> v1:172.21.15.105:6789/0 conn(0x7f4eec053c00 legacy=0x7f4eec052030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:32.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.977+0000 7f4ef2ffd640 1 -- 172.21.15.105:0/1885503676 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eec0555a0 con 0x7f4eec0e12b0 2024-04-15T15:19:32.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.977+0000 7f4ef2ffd640 1 -- 172.21.15.105:0/1885503676 <== mon.1 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7f4ef4075dc0 con 0x7f4eec0e12b0 2024-04-15T15:19:32.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.977+0000 7f4ef2ffd640 1 -- 172.21.15.105:0/1885503676 >> v1:172.21.15.154:6789/0 conn(0x7f4eec0e12b0 legacy=0x7f4eec0534f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:32.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.977+0000 7f4ef2ffd640 1 --2- 172.21.15.105:0/1885503676 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ee40039b0 0x7f4ee4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:32.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.977+0000 7f4ef2ffd640 1 --2- 172.21.15.105:0/1885503676 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4ee4004800 0x7f4ee4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:32.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.977+0000 7f4ef2ffd640 1 --2- 172.21.15.105:0/1885503676 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ee40051c0 0x7f4ee400fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:32.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.977+0000 7f4ef2ffd640 1 -- 172.21.15.105:0/1885503676 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4eec0e26a0 con 0x7f4ee4004800 2024-04-15T15:19:32.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.977+0000 7f4ef2ffd640 1 -- 172.21.15.105:0/1885503676 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee4010550 con 0x7f4ee40039b0 2024-04-15T15:19:32.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.977+0000 7f4ef2ffd640 1 -- 172.21.15.105:0/1885503676 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f4ee4010750 con 0x7f4ee40051c0 2024-04-15T15:19:32.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.977+0000 7f4ef37fe640 1 --2- 172.21.15.105:0/1885503676 >> v2:172.21.15.105:3300/0 conn(0x7f4eec054b50 0x7f4eec051af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:32.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.977+0000 7f4ef3fff640 1 --2- 172.21.15.105:0/1885503676 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ee40039b0 0x7f4ee4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:32.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.978+0000 7fbc8d874640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbc8815c7b0 0x7fbc880d09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:32.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.978+0000 7fbc8d874640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbc880b2cb0 0x7fbc880cd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:32.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.977+0000 7f4ef8ba0640 1 --2- 172.21.15.105:0/1885503676 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ee40051c0 0x7f4ee400fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:32.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.978+0000 7fbc8d874640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc880ccf40 con 0x7fbc880d0f30 2024-04-15T15:19:32.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.978+0000 7fbc8d874640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fbc880b32d0 con 0x7fbc8815c7b0 2024-04-15T15:19:32.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.978+0000 7fbc8d874640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fbc880b35b0 con 0x7fbc880b2cb0 2024-04-15T15:19:32.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.978+0000 7fbc867fc640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fbc880d0f30 0x7fbc880b25a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.105:36872/0 (socket says 172.21.15.105:36872) 2024-04-15T15:19:32.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.978+0000 7fbc867fc640 1 -- 172.21.15.105:0/3262441713 learned_addr learned my addr 172.21.15.105:0/3262441713 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:19:32.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.978+0000 7f4ef37fe640 1 --2- 172.21.15.105:0/1885503676 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4ee4004800 0x7f4ee4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:32.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.978+0000 7fbc86ffd640 1 --2- 172.21.15.105:0/3262441713 >> v2:172.21.15.105:3300/0 conn(0x7fbc8815c7b0 0x7fbc880d09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:32.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.978+0000 7f5a550f3640 1 -- 172.21.15.105:0/1076243679 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a40035020 con 0x7f5a480e12b0 2024-04-15T15:19:32.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.978+0000 7fbc877fe640 1 --2- 172.21.15.105:0/3262441713 >> v2:172.21.15.154:3300/0 conn(0x7fbc880b2cb0 0x7fbc880cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:32.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.978+0000 7f5a550f3640 1 -- 172.21.15.105:0/1076243679 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a40031830 con 0x7f5a480e12b0 2024-04-15T15:19:32.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.978+0000 7fbc85ffb640 1 -- 172.21.15.105:0/3262441713 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1130961570 0 0) 0x7fbc880ccf40 con 0x7fbc880d0f30 2024-04-15T15:19:32.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.978+0000 7fbc85ffb640 1 -- 172.21.15.105:0/3262441713 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbc68003520 con 0x7fbc880d0f30 2024-04-15T15:19:32.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.978+0000 7ff0b2aef640 1 --2- 172.21.15.105:0/4140810948 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0ac1637a0 0x7ff0ac165b90 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4009dc0 tx=0x7ff0a402be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55d2961a91b051ca server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:32.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.978+0000 7fbc85ffb640 1 -- 172.21.15.105:0/3262441713 <== mon.1 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 398139556 0 0) 0x7fbc68003520 con 0x7fbc880d0f30 2024-04-15T15:19:32.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.979+0000 7fbc85ffb640 1 -- 172.21.15.105:0/3262441713 >> v2:172.21.15.154:3300/0 conn(0x7fbc880b2cb0 msgr2=0x7fbc880cd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:32.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.979+0000 7fbc85ffb640 1 --2- 172.21.15.105:0/3262441713 >> v2:172.21.15.154:3300/0 conn(0x7fbc880b2cb0 0x7fbc880cd420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:32.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.979+0000 7fbc85ffb640 1 -- 172.21.15.105:0/3262441713 >> v2:172.21.15.105:3300/0 conn(0x7fbc8815c7b0 msgr2=0x7fbc880d09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:32.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.979+0000 7fbc85ffb640 1 --2- 172.21.15.105:0/3262441713 >> v2:172.21.15.105:3300/0 conn(0x7fbc8815c7b0 0x7fbc880d09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:32.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.979+0000 7fbc85ffb640 1 -- 172.21.15.105:0/3262441713 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc880b3890 con 0x7fbc880d0f30 2024-04-15T15:19:32.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.979+0000 7fbc85ffb640 1 -- 172.21.15.105:0/3262441713 <== mon.1 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7fbc700019f0 con 0x7fbc880d0f30 2024-04-15T15:19:32.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.979+0000 7fbc85ffb640 1 -- 172.21.15.105:0/3262441713 >> v1:172.21.15.154:6789/0 conn(0x7fbc880d0f30 legacy=0x7fbc880b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:32.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.979+0000 7fbc85ffb640 1 --2- 172.21.15.105:0/3262441713 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbc680039b0 0x7fbc68003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:32.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.979+0000 7fbc85ffb640 1 --2- 172.21.15.105:0/3262441713 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc68004800 0x7fbc68004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:32.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.979+0000 7fbc85ffb640 1 --2- 172.21.15.105:0/3262441713 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc680051c0 0x7fbc6800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:32.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.979+0000 7fbc85ffb640 1 -- 172.21.15.105:0/3262441713 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbc880b32d0 con 0x7fbc680051c0 2024-04-15T15:19:32.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.979+0000 7fbc85ffb640 1 -- 172.21.15.105:0/3262441713 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbc880b35b0 con 0x7fbc68004800 2024-04-15T15:19:32.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.979+0000 7fbc85ffb640 1 -- 172.21.15.105:0/3262441713 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fbc6800bf50 con 0x7fbc680039b0 2024-04-15T15:19:32.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.978+0000 7ff0b12ec640 1 -- 172.21.15.105:0/4140810948 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a4039070 con 0x7ff0ac1637a0 2024-04-15T15:19:32.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.978+0000 7ff0b12ec640 1 -- 172.21.15.105:0/4140810948 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0a402cc40 con 0x7ff0ac1637a0 2024-04-15T15:19:32.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.978+0000 7ff0b12ec640 1 -- 172.21.15.105:0/4140810948 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a4034440 con 0x7ff0ac1637a0 2024-04-15T15:19:32.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.979+0000 7f5a550f3640 1 -- 172.21.15.105:0/1076243679 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a40031c00 con 0x7f5a480e12b0 2024-04-15T15:19:32.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.979+0000 7fbc86ffd640 1 --2- 172.21.15.105:0/3262441713 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbc680039b0 0x7fbc68003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:32.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.979+0000 7fbc867fc640 1 --2- 172.21.15.105:0/3262441713 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc68004800 0x7fbc68004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:32.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.979+0000 7fbc877fe640 1 --2- 172.21.15.105:0/3262441713 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc680051c0 0x7fbc6800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:32.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.979+0000 7f5a570f7640 1 -- 172.21.15.105:0/1076243679 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a480e12b0 msgr2=0x7f5a480534f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:32.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.979+0000 7f5a570f7640 1 --2- 172.21.15.105:0/1076243679 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a480e12b0 0x7f5a480534f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5a4800bfb0 tx=0x7f5a4002b8d0 comp rx=0 tx=0).stop 2024-04-15T15:19:32.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.978+0000 7f4ef8ba0640 1 -- 172.21.15.105:0/1885503676 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ee40039b0 msgr2=0x7f4ee4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:32.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.978+0000 7f4ef8ba0640 1 --2- 172.21.15.105:0/1885503676 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ee40039b0 0x7f4ee4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:32.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.978+0000 7f4ef8ba0640 1 -- 172.21.15.105:0/1885503676 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4ee4004800 msgr2=0x7f4ee4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:32.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.978+0000 7f4ef8ba0640 1 --2- 172.21.15.105:0/1885503676 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4ee4004800 0x7f4ee4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:32.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.978+0000 7f4ef8ba0640 1 -- 172.21.15.105:0/1885503676 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ee4003520 con 0x7f4ee40051c0 2024-04-15T15:19:32.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.979+0000 7f4ef8ba0640 1 -- 172.21.15.105:0/1885503676 reap_dead start 2024-04-15T15:19:32.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.979+0000 7f2931074640 1 Processor -- start 2024-04-15T15:19:32.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.979+0000 7f94f9192640 1 Processor -- start 2024-04-15T15:19:32.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.979+0000 7fbc867fc640 1 -- 172.21.15.105:0/3262441713 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbc680039b0 msgr2=0x7fbc68003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:32.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.979+0000 7fbc867fc640 1 --2- 172.21.15.105:0/3262441713 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbc680039b0 0x7fbc68003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:32.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.979+0000 7fbc867fc640 1 -- 172.21.15.105:0/3262441713 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc680051c0 msgr2=0x7fbc6800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:32.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.979+0000 7fbc867fc640 1 --2- 172.21.15.105:0/3262441713 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc680051c0 0x7fbc6800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:32.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.979+0000 7fbc867fc640 1 -- 172.21.15.105:0/3262441713 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc880b3890 con 0x7fbc68004800 2024-04-15T15:19:32.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.979+0000 7f4ef8ba0640 1 --2- 172.21.15.105:0/1885503676 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ee40051c0 0x7f4ee400fdd0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4eec020740 tx=0x7f4ee4005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5bf7273c0ebdd46 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:32.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.980+0000 7fbc877fe640 1 -- 172.21.15.105:0/3262441713 reap_dead start 2024-04-15T15:19:32.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.980+0000 7f4ef2ffd640 1 -- 172.21.15.105:0/1885503676 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee8001370 con 0x7f4ee40051c0 2024-04-15T15:19:32.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.980+0000 7f4ef2ffd640 1 -- 172.21.15.105:0/1885503676 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ee80052f0 con 0x7f4ee40051c0 2024-04-15T15:19:32.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.980+0000 7f4ef2ffd640 1 -- 172.21.15.105:0/1885503676 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee8003830 con 0x7f4ee40051c0 2024-04-15T15:19:32.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.980+0000 7f2931074640 1 -- start start 2024-04-15T15:19:32.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.980+0000 7ff0b4552640 1 -- 172.21.15.105:0/4140810948 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0ac1637a0 msgr2=0x7ff0ac165b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:32.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.980+0000 7ff0b4552640 1 --2- 172.21.15.105:0/4140810948 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0ac1637a0 0x7ff0ac165b90 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4009dc0 tx=0x7ff0a402be90 comp rx=0 tx=0).stop 2024-04-15T15:19:32.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.980+0000 7f5a570f7640 1 -- 172.21.15.105:0/1076243679 shutdown_connections 2024-04-15T15:19:32.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.980+0000 7f5a570f7640 1 --2- 172.21.15.105:0/1076243679 >> v2:172.21.15.154:3300/0 conn(0x7f5a480e4d30 0x7f5a48053a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:32.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.980+0000 7f5a570f7640 1 --2- 172.21.15.105:0/1076243679 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a480e12b0 0x7f5a480534f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:32.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.980+0000 7f5a570f7640 1 -- 172.21.15.105:0/1076243679 >> 172.21.15.105:0/1076243679 conn(0x7f5a48020040 msgr2=0x7f5a48056d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:32.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.980+0000 7f5a570f7640 1 -- 172.21.15.105:0/1076243679 shutdown_connections 2024-04-15T15:19:32.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.980+0000 7ff0b4552640 1 -- 172.21.15.105:0/4140810948 shutdown_connections 2024-04-15T15:19:32.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.980+0000 7ff0b4552640 1 --2- 172.21.15.105:0/4140810948 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0ac1637a0 0x7ff0ac165b90 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:32.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.980+0000 7ff0b4552640 1 -- 172.21.15.105:0/4140810948 >> 172.21.15.105:0/4140810948 conn(0x7ff0ac06bde0 msgr2=0x7ff0ac06c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:32.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.980+0000 7ff0b4552640 1 -- 172.21.15.105:0/4140810948 shutdown_connections 2024-04-15T15:19:32.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.981+0000 7f4ef93a1640 1 -- 172.21.15.105:0/1885503676 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ee40051c0 msgr2=0x7f4ee400fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:32.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.981+0000 7f4ef93a1640 1 --2- 172.21.15.105:0/1885503676 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ee40051c0 0x7f4ee400fdd0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4eec020740 tx=0x7f4ee4005830 comp rx=0 tx=0).stop 2024-04-15T15:19:32.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.980+0000 7f94f9192640 1 -- start start 2024-04-15T15:19:32.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.980+0000 7ff0b4552640 1 -- 172.21.15.105:0/4140810948 wait complete. 2024-04-15T15:19:32.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.981+0000 7f2931074640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f292c161e40 0x7f292c0d31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:33.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.981+0000 7f2931074640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f292c0cfcf0 0x7f292c0d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:33.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.981+0000 7f2931074640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f292c0d0150 0x7f292c0d1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:33.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.981+0000 7f2931074640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f292c0d21a0 con 0x7f292c0d0150 2024-04-15T15:19:33.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.981+0000 7f2931074640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f292c0d36f0 con 0x7f292c0cfcf0 2024-04-15T15:19:33.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.981+0000 7f2931074640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f292c0d23f0 con 0x7f292c161e40 2024-04-15T15:19:33.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.981+0000 7f292b59e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f292c0d0150 0x7f292c0d1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:33.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.981+0000 7f292b59e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f292c0d0150 0x7f292c0d1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39780/0 (socket says 172.21.15.105:39780) 2024-04-15T15:19:33.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.981+0000 7f292b59e640 1 -- 172.21.15.105:0/3706004146 learned_addr learned my addr 172.21.15.105:0/3706004146 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:33.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.981+0000 7f292a59c640 1 --2- 172.21.15.105:0/3706004146 >> v2:172.21.15.154:3300/0 conn(0x7f292c0cfcf0 0x7f292c0d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:33.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.981+0000 7f292b59e640 1 -- 172.21.15.105:0/3706004146 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f292c161e40 msgr2=0x7f292c0d31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:33.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.981+0000 7f292b59e640 1 --2- 172.21.15.105:0/3706004146 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f292c161e40 0x7f292c0d31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.981+0000 7f292b59e640 1 -- 172.21.15.105:0/3706004146 >> v2:172.21.15.154:3300/0 conn(0x7f292c0cfcf0 msgr2=0x7f292c0d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:33.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.981+0000 7f292b59e640 1 --2- 172.21.15.105:0/3706004146 >> v2:172.21.15.154:3300/0 conn(0x7f292c0cfcf0 0x7f292c0d0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.981+0000 7f292b59e640 1 -- 172.21.15.105:0/3706004146 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f292c0d26d0 con 0x7f292c0d0150 2024-04-15T15:19:33.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.981+0000 7fbc867fc640 1 --2- 172.21.15.105:0/3262441713 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc68004800 0x7fbc68004c80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbc7000a8e0 tx=0x7fbc68005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab8c667aef094db6 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:33.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.981+0000 7fbc85ffb640 1 -- 172.21.15.105:0/3262441713 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc7000a4e0 con 0x7fbc68004800 2024-04-15T15:19:33.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.981+0000 7f94f9192640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f94ec0df220 0x7f94ec038080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:33.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.981+0000 7fbc85ffb640 1 -- 172.21.15.105:0/3262441713 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc70034650 con 0x7fbc68004800 2024-04-15T15:19:33.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.981+0000 7fbc85ffb640 1 -- 172.21.15.105:0/3262441713 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc70034a80 con 0x7fbc68004800 2024-04-15T15:19:33.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.982+0000 7f4ef93a1640 1 -- 172.21.15.105:0/1885503676 shutdown_connections 2024-04-15T15:19:33.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.982+0000 7f4ef93a1640 1 --2- 172.21.15.105:0/1885503676 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ee40051c0 0x7f4ee400fdd0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.982+0000 7f4ef93a1640 1 -- 172.21.15.105:0/1885503676 >> 172.21.15.105:0/1885503676 conn(0x7f4eec020040 msgr2=0x7f4eec056d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:33.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.982+0000 7f94f9192640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f94ec053c70 0x7f94ec04fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:33.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.982+0000 7f4ef93a1640 1 -- 172.21.15.105:0/1885503676 shutdown_connections 2024-04-15T15:19:33.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.982+0000 7f94f9192640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94ec0544c0 con 0x7f94ec033ef0 2024-04-15T15:19:33.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.982+0000 7f94f9192640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f94ec0541e0 con 0x7f94ec053c70 2024-04-15T15:19:33.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.982+0000 7f94f9192640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f94ec005f30 con 0x7f94ec0df220 2024-04-15T15:19:33.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.982+0000 7f94f3fff640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f94ec0df220 0x7f94ec038080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:33.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.982+0000 7f94f3fff640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f94ec0df220 0x7f94ec038080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.105:57132/0 (socket says 172.21.15.105:57132) 2024-04-15T15:19:33.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.982+0000 7f94f3fff640 1 -- 172.21.15.105:0/223086940 learned_addr learned my addr 172.21.15.105:0/223086940 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:33.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.982+0000 7fbc8d874640 1 -- 172.21.15.105:0/3262441713 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc68004800 msgr2=0x7fbc68004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:33.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.982+0000 7fbc8d874640 1 --2- 172.21.15.105:0/3262441713 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc68004800 0x7fbc68004c80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbc7000a8e0 tx=0x7fbc68005830 comp rx=0 tx=0).stop 2024-04-15T15:19:33.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.982+0000 7fbc8d874640 1 -- 172.21.15.105:0/3262441713 shutdown_connections 2024-04-15T15:19:33.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.982+0000 7fbc8d874640 1 --2- 172.21.15.105:0/3262441713 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc68004800 0x7fbc68004c80 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.982+0000 7fbc8d874640 1 -- 172.21.15.105:0/3262441713 >> 172.21.15.105:0/3262441713 conn(0x7fbc880d25a0 msgr2=0x7fbc880d29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:33.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.980+0000 7f5a570f7640 1 -- 172.21.15.105:0/1076243679 wait complete. 2024-04-15T15:19:33.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.982+0000 7f5a570f7640 1 Processor -- start 2024-04-15T15:19:33.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.982+0000 7fbc8d874640 1 -- 172.21.15.105:0/3262441713 shutdown_connections 2024-04-15T15:19:33.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.982+0000 7fbc8d874640 1 -- 172.21.15.105:0/3262441713 wait complete. 2024-04-15T15:19:33.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.982+0000 7f5a570f7640 1 -- start start 2024-04-15T15:19:33.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.983+0000 7ff0b4552640 1 Processor -- start 2024-04-15T15:19:33.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.983+0000 7f292b59e640 1 --2- 172.21.15.105:0/3706004146 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f292c0d0150 0x7f292c0d1450 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f292c08cf10 tx=0x7f292403ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cff19c3abecc7e32 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:33.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.983+0000 7f2929d9b640 1 -- 172.21.15.105:0/3706004146 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f292403da00 con 0x7f292c0d0150 2024-04-15T15:19:33.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.983+0000 7f2929d9b640 1 -- 172.21.15.105:0/3706004146 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f292403dba0 con 0x7f292c0d0150 2024-04-15T15:19:33.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.983+0000 7f2929d9b640 1 -- 172.21.15.105:0/3706004146 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29240453f0 con 0x7f292c0d0150 2024-04-15T15:19:33.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.982+0000 7f4ef93a1640 1 -- 172.21.15.105:0/1885503676 wait complete. 2024-04-15T15:19:33.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.984+0000 7f5a570f7640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5a480e12b0 0x7f5a480d2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:33.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.984+0000 7f4ef93a1640 1 Processor -- start 2024-04-15T15:19:33.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.983+0000 7f94f2ffd640 1 -- 172.21.15.105:0/223086940 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4003370043 0 0) 0x7f94ec0544c0 con 0x7f94ec033ef0 2024-04-15T15:19:33.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.984+0000 7f5a570f7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a480e4d30 0x7f5a480d2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:33.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.983+0000 7f94f2ffd640 1 -- 172.21.15.105:0/223086940 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f94e4003520 con 0x7f94ec033ef0 2024-04-15T15:19:33.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.984+0000 7f94f3fff640 1 -- 172.21.15.105:0/223086940 >> v2:172.21.15.105:3300/0 conn(0x7f94ec053c70 msgr2=0x7f94ec04fa90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-15T15:19:33.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.984+0000 7f94f3fff640 1 --2- 172.21.15.105:0/223086940 >> v2:172.21.15.105:3300/0 conn(0x7f94ec053c70 0x7f94ec04fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.984+0000 7f94f3fff640 1 -- 172.21.15.105:0/223086940 >> v1:172.21.15.154:6789/0 conn(0x7f94ec033ef0 legacy=0x7f94ec006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:33.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.984+0000 7f94f3fff640 1 -- 172.21.15.105:0/223086940 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94ec006210 con 0x7f94ec0df220 2024-04-15T15:19:33.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.984+0000 7f2931074640 1 -- 172.21.15.105:0/3706004146 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f292c0d0150 msgr2=0x7f292c0d1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:33.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.984+0000 7f5a558f4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a480e4d30 0x7f5a480d2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:33.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.984+0000 7f5a558f4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a480e4d30 0x7f5a480d2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39782/0 (socket says 172.21.15.105:39782) 2024-04-15T15:19:33.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.984+0000 7f5a560f5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5a480e12b0 0x7f5a480d2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:33.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.984+0000 7f2931074640 1 --2- 172.21.15.105:0/3706004146 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f292c0d0150 0x7f292c0d1450 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f292c08cf10 tx=0x7f292403ce60 comp rx=0 tx=0).stop 2024-04-15T15:19:33.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.984+0000 7f2931074640 1 -- 172.21.15.105:0/3706004146 shutdown_connections 2024-04-15T15:19:33.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.984+0000 7f2931074640 1 --2- 172.21.15.105:0/3706004146 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f292c0d0150 0x7f292c0d1450 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.984+0000 7f2931074640 1 --2- 172.21.15.105:0/3706004146 >> v2:172.21.15.154:3300/0 conn(0x7f292c0cfcf0 0x7f292c0d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.984+0000 7f2931074640 1 --2- 172.21.15.105:0/3706004146 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f292c161e40 0x7f292c0d31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.984+0000 7f2931074640 1 -- 172.21.15.105:0/3706004146 >> 172.21.15.105:0/3706004146 conn(0x7f292c08bfd0 msgr2=0x7f292c0d45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:33.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.984+0000 7fbc8d874640 1 Processor -- start 2024-04-15T15:19:33.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.984+0000 7f5a560f5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5a480e12b0 0x7f5a480d2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.105:57140/0 (socket says 172.21.15.105:57140) 2024-04-15T15:19:33.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.984+0000 7fbc8d874640 1 -- start start 2024-04-15T15:19:33.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.984+0000 7f5a558f4640 1 -- 172.21.15.105:0/1962303513 learned_addr learned my addr 172.21.15.105:0/1962303513 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:33.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.984+0000 7f5a570f7640 1 --2- 172.21.15.105:0/1962303513 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a480d32b0 0x7f5a48008000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:33.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.983+0000 7ff0b4552640 1 -- start start 2024-04-15T15:19:33.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.984+0000 7ff0b4552640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0ac0b4b20 0x7ff0ac14ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:33.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.984+0000 7f5a570f7640 1 -- 172.21.15.105:0/1962303513 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5a48057820 con 0x7f5a480e4d30 2024-04-15T15:19:33.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.984+0000 7f5a570f7640 1 -- 172.21.15.105:0/1962303513 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5a480576a0 con 0x7f5a480d32b0 2024-04-15T15:19:33.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.985+0000 7f5a568f6640 1 --2- 172.21.15.105:0/1962303513 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a480d32b0 0x7f5a48008000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:33.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.984+0000 7ff0b4552640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0ac15b8b0 0x7ff0ac1406d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:33.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.984+0000 7ff0b4552640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0ac1637a0 0x7ff0ac14fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:33.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.984+0000 7ff0b4552640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ac15ead0 con 0x7ff0ac0b4b20 2024-04-15T15:19:33.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.984+0000 7ff0b4552640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ac15e7f0 con 0x7ff0ac15b8b0 2024-04-15T15:19:33.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.984+0000 7ff0b4552640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7ff0ac15f2d0 con 0x7ff0ac1637a0 2024-04-15T15:19:33.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.984+0000 7ff0b1aed640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0ac15b8b0 0x7ff0ac1406d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:33.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.984+0000 7ff0b2aef640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0ac1637a0 0x7ff0ac14fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:33.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.984+0000 7ff0b1aed640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0ac15b8b0 0x7ff0ac1406d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.105:45152/0 (socket says 172.21.15.105:45152) 2024-04-15T15:19:33.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.984+0000 7ff0b1aed640 1 -- 172.21.15.105:0/382641105 learned_addr learned my addr 172.21.15.105:0/382641105 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:33.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.985+0000 7ff0b22ee640 1 --2- 172.21.15.105:0/382641105 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0ac0b4b20 0x7ff0ac14ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:33.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.985+0000 7f5a570f7640 1 -- 172.21.15.105:0/1962303513 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f5a480e1da0 con 0x7f5a480e12b0 2024-04-15T15:19:33.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.985+0000 7ff0b1aed640 1 -- 172.21.15.105:0/382641105 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0ac1637a0 msgr2=0x7ff0ac14fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:33.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.985+0000 7f2931074640 1 -- 172.21.15.105:0/3706004146 shutdown_connections 2024-04-15T15:19:33.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.985+0000 7ff0b1aed640 1 --2- 172.21.15.105:0/382641105 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0ac1637a0 0x7ff0ac14fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.985+0000 7ff0b1aed640 1 -- 172.21.15.105:0/382641105 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0ac0b4b20 msgr2=0x7ff0ac14ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:33.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.985+0000 7ff0b1aed640 1 --2- 172.21.15.105:0/382641105 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0ac0b4b20 0x7ff0ac14ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.985+0000 7ff0b1aed640 1 -- 172.21.15.105:0/382641105 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0ac14c640 con 0x7ff0ac15b8b0 2024-04-15T15:19:33.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.985+0000 7ff0b22ee640 1 --2- 172.21.15.105:0/382641105 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0ac0b4b20 0x7ff0ac14ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:19:33.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.986+0000 7fbc8d874640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc68004800 0x7fbc880b7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:33.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.985+0000 7f4ef93a1640 1 -- start start 2024-04-15T15:19:33.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.985+0000 7f94f3fff640 1 --2- 172.21.15.105:0/223086940 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f94ec0df220 0x7f94ec038080 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f94e800de50 tx=0x7f94e802ff00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93e66d62aa976939 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:33.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.985+0000 7f94f2ffd640 1 -- 172.21.15.105:0/223086940 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e8030aa0 con 0x7f94ec0df220 2024-04-15T15:19:33.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.985+0000 7f94f2ffd640 1 -- 172.21.15.105:0/223086940 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94e8030c40 con 0x7f94ec0df220 2024-04-15T15:19:33.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.985+0000 7f94f2ffd640 1 -- 172.21.15.105:0/223086940 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e80383f0 con 0x7f94ec0df220 2024-04-15T15:19:33.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.986+0000 7f4ef93a1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ee40051c0 0x7f4eec0de840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:33.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.985+0000 7f5a558f4640 1 -- 172.21.15.105:0/1962303513 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5a480e12b0 msgr2=0x7f5a480d2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:33.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.985+0000 7f5a558f4640 1 --2- 172.21.15.105:0/1962303513 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5a480e12b0 0x7f5a480d2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.985+0000 7f5a558f4640 1 -- 172.21.15.105:0/1962303513 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a480d32b0 msgr2=0x7f5a48008000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:33.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.985+0000 7f5a558f4640 1 --2- 172.21.15.105:0/1962303513 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a480d32b0 0x7f5a48008000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.985+0000 7f5a558f4640 1 -- 172.21.15.105:0/1962303513 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a480e46d0 con 0x7f5a480e4d30 2024-04-15T15:19:33.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.986+0000 7f5a558f4640 1 --2- 172.21.15.105:0/1962303513 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a480e4d30 0x7f5a480d2d70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c00c780 tx=0x7f5a4c00cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:33.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.986+0000 7f5a568f6640 1 --2- 172.21.15.105:0/1962303513 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a480d32b0 0x7f5a48008000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:33.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.986+0000 7f5a560f5640 1 --2- 172.21.15.105:0/1962303513 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5a480e12b0 0x7f5a480d2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:33.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.986+0000 7fbc86ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc68004800 0x7fbc880b7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:33.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.986+0000 7fbc86ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc68004800 0x7fbc880b7e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39800/0 (socket says 172.21.15.105:39800) 2024-04-15T15:19:33.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.986+0000 7fbc86ffd640 1 -- 172.21.15.105:0/392281873 learned_addr learned my addr 172.21.15.105:0/392281873 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:33.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.986+0000 7f5a46ffd640 1 -- 172.21.15.105:0/1962303513 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c005070 con 0x7f5a480e4d30 2024-04-15T15:19:33.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.986+0000 7fbc8d874640 1 --2- 172.21.15.105:0/392281873 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc88159c10 0x7fbc88151a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:33.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.986+0000 7fbc8d874640 1 --2- 172.21.15.105:0/392281873 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbc8815f650 0x7fbc881603a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:33.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.986+0000 7f4ef93a1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4eec0d2b10 0x7f4eec0df4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:33.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.986+0000 7fbc8d874640 1 -- 172.21.15.105:0/392281873 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8815cd10 con 0x7fbc68004800 2024-04-15T15:19:33.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.986+0000 7fbc867fc640 1 --2- 172.21.15.105:0/392281873 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc88159c10 0x7fbc88151a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:33.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.986+0000 7f5a570f7640 1 -- 172.21.15.105:0/1962303513 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a4804fdf0 con 0x7f5a480e4d30 2024-04-15T15:19:33.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.986+0000 7f5a570f7640 1 -- 172.21.15.105:0/1962303513 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a480503b0 con 0x7f5a480e4d30 2024-04-15T15:19:33.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.986+0000 7f5a46ffd640 1 -- 172.21.15.105:0/1962303513 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a4c005210 con 0x7f5a480e4d30 2024-04-15T15:19:33.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.986+0000 7fbc8d874640 1 -- 172.21.15.105:0/392281873 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8815cb90 con 0x7fbc88159c10 2024-04-15T15:19:33.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.986+0000 7fbc877fe640 1 --2- 172.21.15.105:0/392281873 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbc8815f650 0x7fbc881603a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:33.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.986+0000 7f5a46ffd640 1 -- 172.21.15.105:0/1962303513 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c00a640 con 0x7f5a480e4d30 2024-04-15T15:19:33.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.986+0000 7fbc8d874640 1 -- 172.21.15.105:0/392281873 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fbc880d09f0 con 0x7fbc8815f650 2024-04-15T15:19:33.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.986+0000 7f94f9192640 1 -- 172.21.15.105:0/223086940 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f94ec0df220 msgr2=0x7f94ec038080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:33.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.986+0000 7f94f9192640 1 --2- 172.21.15.105:0/223086940 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f94ec0df220 0x7f94ec038080 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f94e800de50 tx=0x7f94e802ff00 comp rx=0 tx=0).stop 2024-04-15T15:19:33.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.985+0000 7f2931074640 1 -- 172.21.15.105:0/3706004146 wait complete. 2024-04-15T15:19:33.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.987+0000 7f4ef3fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ee40051c0 0x7f4eec0de840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:33.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.987+0000 7f4ef3fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ee40051c0 0x7f4eec0de840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.105:45184/0 (socket says 172.21.15.105:45184) 2024-04-15T15:19:33.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.987+0000 7f4ef37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4eec0d2b10 0x7f4eec0df4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:33.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.987+0000 7f4ef37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4eec0d2b10 0x7f4eec0df4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39802/0 (socket says 172.21.15.105:39802) 2024-04-15T15:19:33.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.987+0000 7f4ef3fff640 1 -- 172.21.15.105:0/102749193 learned_addr learned my addr 172.21.15.105:0/102749193 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:33.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.987+0000 7f4ef93a1640 1 --2- 172.21.15.105:0/102749193 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4eec052b40 0x7f4eec052ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:33.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.987+0000 7f4ef93a1640 1 -- 172.21.15.105:0/102749193 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4eec057820 con 0x7f4eec0d2b10 2024-04-15T15:19:33.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.987+0000 7f4ef93a1640 1 -- 172.21.15.105:0/102749193 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4eec0576a0 con 0x7f4ee40051c0 2024-04-15T15:19:33.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.987+0000 7f4ef93a1640 1 -- 172.21.15.105:0/102749193 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f4eec0e4d30 con 0x7f4eec052b40 2024-04-15T15:19:33.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.987+0000 7f4ef8ba0640 1 --2- 172.21.15.105:0/102749193 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4eec052b40 0x7f4eec052ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:33.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.988+0000 7f94f9192640 1 -- 172.21.15.105:0/223086940 shutdown_connections 2024-04-15T15:19:33.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.987+0000 7ff0b1aed640 1 --2- 172.21.15.105:0/382641105 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0ac15b8b0 0x7ff0ac1406d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff09c00c400 tx=0x7ff09c00c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:33.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.987+0000 7ff0a2ffd640 1 -- 172.21.15.105:0/382641105 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09c017070 con 0x7ff0ac15b8b0 2024-04-15T15:19:33.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.987+0000 7ff0a2ffd640 1 -- 172.21.15.105:0/382641105 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff09c0052d0 con 0x7ff0ac15b8b0 2024-04-15T15:19:33.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.987+0000 7ff0a2ffd640 1 -- 172.21.15.105:0/382641105 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09c003830 con 0x7ff0ac15b8b0 2024-04-15T15:19:33.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.988+0000 7f94f9192640 1 --2- 172.21.15.105:0/223086940 >> v2:172.21.15.105:3300/0 conn(0x7f94ec053c70 0x7f94ec04fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.987+0000 7f2931074640 1 Processor -- start 2024-04-15T15:19:33.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.987+0000 7f2931074640 1 -- start start 2024-04-15T15:19:33.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.988+0000 7f94f9192640 1 --2- 172.21.15.105:0/223086940 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f94ec0df220 0x7f94ec038080 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.988+0000 7f94f9192640 1 -- 172.21.15.105:0/223086940 >> 172.21.15.105:0/223086940 conn(0x7f94ec009fb0 msgr2=0x7f94ec00a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:33.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.988+0000 7fbc867fc640 1 -- 172.21.15.105:0/392281873 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbc8815f650 msgr2=0x7fbc881603a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:33.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.988+0000 7f4ef3fff640 1 -- 172.21.15.105:0/102749193 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4eec052b40 msgr2=0x7f4eec052ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:33.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.988+0000 7f4ef3fff640 1 --2- 172.21.15.105:0/102749193 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4eec052b40 0x7f4eec052ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.988+0000 7f4ef3fff640 1 -- 172.21.15.105:0/102749193 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4eec0d2b10 msgr2=0x7f4eec0df4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:33.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.988+0000 7f4ef3fff640 1 --2- 172.21.15.105:0/102749193 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4eec0d2b10 0x7f4eec0df4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.988+0000 7fbc867fc640 1 --2- 172.21.15.105:0/392281873 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbc8815f650 0x7fbc881603a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.988+0000 7fbc867fc640 1 -- 172.21.15.105:0/392281873 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc68004800 msgr2=0x7fbc880b7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:33.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.988+0000 7fbc867fc640 1 --2- 172.21.15.105:0/392281873 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc68004800 0x7fbc880b7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.988+0000 7ff0b4552640 1 -- 172.21.15.105:0/382641105 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0ac14c920 con 0x7ff0ac15b8b0 2024-04-15T15:19:33.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.988+0000 7ff0b4552640 1 -- 172.21.15.105:0/382641105 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0ac14cee0 con 0x7ff0ac15b8b0 2024-04-15T15:19:33.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.988+0000 7f5a570f7640 1 -- 172.21.15.105:0/1962303513 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a38005290 con 0x7f5a480e4d30 2024-04-15T15:19:33.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.988+0000 7fbc867fc640 1 -- 172.21.15.105:0/392281873 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc88151f70 con 0x7fbc88159c10 2024-04-15T15:19:33.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.988+0000 7fbc877fe640 1 --2- 172.21.15.105:0/392281873 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbc8815f650 0x7fbc881603a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:19:33.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.988+0000 7f94f9192640 1 -- 172.21.15.105:0/223086940 shutdown_connections 2024-04-15T15:19:33.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.988+0000 7f94f9192640 1 -- 172.21.15.105:0/223086940 wait complete. 2024-04-15T15:19:33.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.988+0000 7f2931074640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f292c0cfcf0 0x7f292c152a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:33.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.989+0000 7f2931074640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f292c0d0150 0x7f292c0b7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:33.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.989+0000 7f2931074640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f292c161e40 0x7f292c0b9280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:33.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.988+0000 7f4ef3fff640 1 -- 172.21.15.105:0/102749193 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eec0df9e0 con 0x7f4ee40051c0 2024-04-15T15:19:33.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.989+0000 7f2931074640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f292c162220 con 0x7f292c0cfcf0 2024-04-15T15:19:33.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.989+0000 7f2931074640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f292c0d2c90 con 0x7f292c161e40 2024-04-15T15:19:33.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.989+0000 7f2931074640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f292c1623a0 con 0x7f292c0d0150 2024-04-15T15:19:33.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.989+0000 7f4ef37fe640 1 --2- 172.21.15.105:0/102749193 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4eec0d2b10 0x7f4eec0df4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:19:33.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.989+0000 7f4ef3fff640 1 --2- 172.21.15.105:0/102749193 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ee40051c0 0x7f4eec0de840 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4ef40654e0 tx=0x7f4ef40724b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:33.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.989+0000 7fbc867fc640 1 --2- 172.21.15.105:0/392281873 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc88159c10 0x7fbc88151a30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbc70031000 tx=0x7fbc70001360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:33.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.989+0000 7fbc7ffff640 1 -- 172.21.15.105:0/392281873 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc70041020 con 0x7fbc88159c10 2024-04-15T15:19:33.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.989+0000 7fbc7ffff640 1 -- 172.21.15.105:0/392281873 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc700019c0 con 0x7fbc88159c10 2024-04-15T15:19:33.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.989+0000 7fbc8d874640 1 -- 172.21.15.105:0/392281873 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc8815e5b0 con 0x7fbc88159c10 2024-04-15T15:19:33.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.989+0000 7f292b59e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f292c161e40 0x7f292c0b9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:33.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.989+0000 7f292b59e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f292c161e40 0x7f292c0b9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.105:45198/0 (socket says 172.21.15.105:45198) 2024-04-15T15:19:33.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.989+0000 7f292b59e640 1 -- 172.21.15.105:0/3232439012 learned_addr learned my addr 172.21.15.105:0/3232439012 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:33.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.989+0000 7f4ef0ff9640 1 -- 172.21.15.105:0/102749193 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef4099070 con 0x7f4ee40051c0 2024-04-15T15:19:33.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.989+0000 7f4ef0ff9640 1 -- 172.21.15.105:0/102749193 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ef4072c40 con 0x7f4ee40051c0 2024-04-15T15:19:33.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.989+0000 7f4ef93a1640 1 -- 172.21.15.105:0/102749193 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4eec04c320 con 0x7f4ee40051c0 2024-04-15T15:19:33.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.989+0000 7ff0b4552640 1 -- 172.21.15.105:0/382641105 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0ac068a40 con 0x7ff0ac15b8b0 2024-04-15T15:19:33.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.989+0000 7fbc8d874640 1 -- 172.21.15.105:0/392281873 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc8815eb70 con 0x7fbc88159c10 2024-04-15T15:19:33.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.991+0000 7fbc7ffff640 1 -- 172.21.15.105:0/392281873 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc70041020 con 0x7fbc88159c10 2024-04-15T15:19:33.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.991+0000 7fbc8d874640 1 -- 172.21.15.105:0/392281873 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc60005290 con 0x7fbc88159c10 2024-04-15T15:19:33.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.991+0000 7f5a46ffd640 1 -- 172.21.15.105:0/1962303513 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5a4c00a7e0 con 0x7f5a480e4d30 2024-04-15T15:19:33.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.991+0000 7ff0a2ffd640 1 -- 172.21.15.105:0/382641105 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff09c01a070 con 0x7ff0ac15b8b0 2024-04-15T15:19:33.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.990+0000 7f4ef93a1640 1 -- 172.21.15.105:0/102749193 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4eec04c8e0 con 0x7f4ee40051c0 2024-04-15T15:19:33.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.990+0000 7f4ef0ff9640 1 -- 172.21.15.105:0/102749193 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef40943f0 con 0x7f4ee40051c0 2024-04-15T15:19:33.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.991+0000 7f94f9192640 1 Processor -- start 2024-04-15T15:19:33.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.992+0000 7f94f9192640 1 -- start start 2024-04-15T15:19:33.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.992+0000 7f5a46ffd640 1 --2- 172.21.15.105:0/1962303513 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f5a3406c920 0x7f5a3406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:33.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.992+0000 7f5a560f5640 1 --2- 172.21.15.105:0/1962303513 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f5a3406c920 0x7f5a3406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:33.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.992+0000 7f5a46ffd640 1 -- 172.21.15.105:0/1962303513 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5a4c08c220 con 0x7f5a480e4d30 2024-04-15T15:19:33.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.992+0000 7ff0a2ffd640 1 --2- 172.21.15.105:0/382641105 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff09006c9c0 0x7ff09006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:33.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.992+0000 7ff0a2ffd640 1 -- 172.21.15.105:0/382641105 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff09c08af90 con 0x7ff0ac15b8b0 2024-04-15T15:19:33.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.992+0000 7f292b59e640 1 -- 172.21.15.105:0/3232439012 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f292c0d0150 msgr2=0x7f292c0b7a30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-15T15:19:33.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.992+0000 7f292b59e640 1 --2- 172.21.15.105:0/3232439012 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f292c0d0150 0x7f292c0b7a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.992+0000 7f292b59e640 1 -- 172.21.15.105:0/3232439012 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f292c0cfcf0 msgr2=0x7f292c152a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:33.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.992+0000 7f292b59e640 1 --2- 172.21.15.105:0/3232439012 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f292c0cfcf0 0x7f292c152a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.992+0000 7f292b59e640 1 -- 172.21.15.105:0/3232439012 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f292c0b9bb0 con 0x7f292c161e40 2024-04-15T15:19:33.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.992+0000 7f292b59e640 1 --2- 172.21.15.105:0/3232439012 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f292c161e40 0x7f292c0b9280 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2924009f40 tx=0x7f2924004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:33.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.992+0000 7ff0b22ee640 1 --2- 172.21.15.105:0/382641105 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff09006c9c0 0x7ff09006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:33.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.993+0000 7f4ef0ff9640 1 -- 172.21.15.105:0/102749193 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4ef409c070 con 0x7f4ee40051c0 2024-04-15T15:19:33.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.993+0000 7ff0b22ee640 1 --2- 172.21.15.105:0/382641105 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff09006c9c0 0x7ff09006ee80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff0a8000f60 tx=0x7ff0a8007c70 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:33.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.993+0000 7f94f9192640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f94ec033ef0 0x7f94ec04aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:33.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.993+0000 7f94f3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f94ec033ef0 0x7f94ec04aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:33.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.993+0000 7f94f3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f94ec033ef0 0x7f94ec04aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39828/0 (socket says 172.21.15.105:39828) 2024-04-15T15:19:33.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.993+0000 7f94f3fff640 1 -- 172.21.15.105:0/855079544 learned_addr learned my addr 172.21.15.105:0/855079544 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:33.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.993+0000 7f94f9192640 1 --2- 172.21.15.105:0/855079544 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f94ec053c70 0x7f94ec04e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:33.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.993+0000 7f4ef0ff9640 1 --2- 172.21.15.105:0/102749193 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f4ed806c940 0x7f4ed806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:33.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.993+0000 7f94f9192640 1 --2- 172.21.15.105:0/855079544 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f94ec0df220 0x7f94ec00a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:33.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.993+0000 7f94f9192640 1 -- 172.21.15.105:0/855079544 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f94ec037a90 con 0x7f94ec033ef0 2024-04-15T15:19:33.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.993+0000 7f94f9192640 1 -- 172.21.15.105:0/855079544 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f94ec037910 con 0x7f94ec0df220 2024-04-15T15:19:33.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.993+0000 7f94f9192640 1 -- 172.21.15.105:0/855079544 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f94ec0df600 con 0x7f94ec053c70 2024-04-15T15:19:33.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.993+0000 7f4ef93a1640 1 -- 172.21.15.105:0/102749193 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4eec0057e0 con 0x7f4ee40051c0 2024-04-15T15:19:33.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.994+0000 7f4ef37fe640 1 --2- 172.21.15.105:0/102749193 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f4ed806c940 0x7f4ed806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:33.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.994+0000 7f94f8991640 1 --2- 172.21.15.105:0/855079544 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f94ec0df220 0x7f94ec00a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:33.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.994+0000 7f94f37fe640 1 --2- 172.21.15.105:0/855079544 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f94ec053c70 0x7f94ec04e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:33.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.994+0000 7f5a560f5640 1 --2- 172.21.15.105:0/1962303513 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f5a3406c920 0x7f5a3406ede0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5a40036000 tx=0x7f5a40008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:33.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.994+0000 7f94f3fff640 1 -- 172.21.15.105:0/855079544 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f94ec053c70 msgr2=0x7f94ec04e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:33.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.994+0000 7f94f3fff640 1 --2- 172.21.15.105:0/855079544 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f94ec053c70 0x7f94ec04e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.994+0000 7f94f3fff640 1 -- 172.21.15.105:0/855079544 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f94ec0df220 msgr2=0x7f94ec00a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:33.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.994+0000 7f94f3fff640 1 --2- 172.21.15.105:0/855079544 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f94ec0df220 0x7f94ec00a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.994+0000 7f94f3fff640 1 -- 172.21.15.105:0/855079544 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94ec04af70 con 0x7f94ec033ef0 2024-04-15T15:19:33.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.994+0000 7f94f3fff640 1 --2- 172.21.15.105:0/855079544 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f94ec033ef0 0x7f94ec04aa30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f94e800de50 tx=0x7f94e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:33.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.994+0000 7f4ef37fe640 1 --2- 172.21.15.105:0/102749193 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f4ed806c940 0x7f4ed806ee00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4eec007df0 tx=0x7f4ee0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:33.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.995+0000 7ff0a2ffd640 1 -- 172.21.15.105:0/382641105 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff09c058d50 con 0x7ff0ac15b8b0 2024-04-15T15:19:33.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.995+0000 7f4ef0ff9640 1 -- 172.21.15.105:0/102749193 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4ef410b060 con 0x7f4ee40051c0 2024-04-15T15:19:33.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.995+0000 7f29137fe640 1 -- 172.21.15.105:0/3232439012 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2924054020 con 0x7f292c161e40 2024-04-15T15:19:33.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.996+0000 7f2931074640 1 -- 172.21.15.105:0/3232439012 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f292c0b9e90 con 0x7f292c161e40 2024-04-15T15:19:33.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.996+0000 7f2931074640 1 -- 172.21.15.105:0/3232439012 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f292c0cc300 con 0x7f292c161e40 2024-04-15T15:19:33.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.996+0000 7fbc7ffff640 1 -- 172.21.15.105:0/392281873 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbc70044020 con 0x7fbc88159c10 2024-04-15T15:19:33.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.996+0000 7fbc7ffff640 1 --2- 172.21.15.105:0/392281873 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fbc6c06ccf0 0x7fbc6c06f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:33.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.997+0000 7fbc7ffff640 1 -- 172.21.15.105:0/392281873 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbc70039030 con 0x7fbc88159c10 2024-04-15T15:19:33.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.997+0000 7fbc86ffd640 1 --2- 172.21.15.105:0/392281873 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fbc6c06ccf0 0x7fbc6c06f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:33.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.997+0000 7f2931074640 1 -- 172.21.15.105:0/3232439012 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f292c068a40 con 0x7f292c161e40 2024-04-15T15:19:33.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.997+0000 7fbc86ffd640 1 --2- 172.21.15.105:0/392281873 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fbc6c06ccf0 0x7fbc6c06f1b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbc8806cee0 tx=0x7fbc80008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:33.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.998+0000 7f94f0ff9640 1 -- 172.21.15.105:0/855079544 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e8047020 con 0x7f94ec033ef0 2024-04-15T15:19:33.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.998+0000 7f94f0ff9640 1 -- 172.21.15.105:0/855079544 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94e8042450 con 0x7f94ec033ef0 2024-04-15T15:19:33.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.998+0000 7f94f9192640 1 -- 172.21.15.105:0/855079544 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94ec174b00 con 0x7f94ec033ef0 2024-04-15T15:19:33.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.998+0000 7f94f0ff9640 1 -- 172.21.15.105:0/855079544 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e800d510 con 0x7f94ec033ef0 2024-04-15T15:19:33.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.998+0000 7f94f9192640 1 -- 172.21.15.105:0/855079544 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94ec1750c0 con 0x7f94ec033ef0 2024-04-15T15:19:33.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.998+0000 7f5a46ffd640 1 -- 172.21.15.105:0/1962303513 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5a4c05a6c0 con 0x7f5a480e4d30 2024-04-15T15:19:33.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.999+0000 7f29137fe640 1 -- 172.21.15.105:0/3232439012 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f292403da10 con 0x7f292c161e40 2024-04-15T15:19:33.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.000+0000 7f29137fe640 1 -- 172.21.15.105:0/3232439012 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2924009510 con 0x7f292c161e40 2024-04-15T15:19:33.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:32.999+0000 7f94f9192640 1 -- 172.21.15.105:0/855079544 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94ec005390 con 0x7f94ec033ef0 2024-04-15T15:19:33.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.000+0000 7f94f0ff9640 1 -- 172.21.15.105:0/855079544 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f94e804a020 con 0x7f94ec033ef0 2024-04-15T15:19:33.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.001+0000 7f4ef0ff9640 1 -- 172.21.15.105:0/102749193 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4ef4091070 con 0x7f4ee40051c0 2024-04-15T15:19:33.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.001+0000 7f94f0ff9640 1 --2- 172.21.15.105:0/855079544 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f94d806c8f0 0x7f94d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:33.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.001+0000 7f94f0ff9640 1 -- 172.21.15.105:0/855079544 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f94e80b22a0 con 0x7f94ec033ef0 2024-04-15T15:19:33.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.001+0000 7f94f37fe640 1 --2- 172.21.15.105:0/855079544 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f94d806c8f0 0x7f94d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:33.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.003+0000 7f94f37fe640 1 --2- 172.21.15.105:0/855079544 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f94d806c8f0 0x7f94d806edb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f94ec0e1b40 tx=0x7f94e00090e0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:33.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.005+0000 7fbc7ffff640 1 -- 172.21.15.105:0/392281873 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbc7004b030 con 0x7fbc88159c10 2024-04-15T15:19:33.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.006+0000 7f29137fe640 1 -- 172.21.15.105:0/3232439012 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2924057020 con 0x7f292c161e40 2024-04-15T15:19:33.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.006+0000 7f29137fe640 1 --2- 172.21.15.105:0/3232439012 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f291806c8f0 0x7f291806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:33.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.006+0000 7f29137fe640 1 -- 172.21.15.105:0/3232439012 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f29240bfa30 con 0x7f292c161e40 2024-04-15T15:19:33.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.007+0000 7f292ad9d640 1 --2- 172.21.15.105:0/3232439012 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f291806c8f0 0x7f291806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:33.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.007+0000 7f292ad9d640 1 --2- 172.21.15.105:0/3232439012 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f291806c8f0 0x7f291806edb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2920009670 tx=0x7f2920006bf0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:33.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.009+0000 7f29137fe640 1 -- 172.21.15.105:0/3232439012 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f292405e030 con 0x7f292c161e40 2024-04-15T15:19:33.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.010+0000 7f94f0ff9640 1 -- 172.21.15.105:0/855079544 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f94e8051030 con 0x7f94ec033ef0 2024-04-15T15:19:33.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.346+0000 7fefa76ba640 1 -- 172.21.15.105:0/3141210596 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fefa00ccae0 con 0x7fefa00c79a0 2024-04-15T15:19:33.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.347+0000 7fef95ffb640 1 -- 172.21.15.105:0/3141210596 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fef9c0172c0 con 0x7fefa00c79a0 2024-04-15T15:19:33.349 INFO:teuthology.orchestra.run.smithi105.stdout:55834574856 2024-04-15T15:19:33.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.350+0000 7fef8b7fe640 1 -- 172.21.15.105:0/3141210596 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fef8406c9f0 msgr2=0x7fef8406eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:33.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.350+0000 7fef8b7fe640 1 --2- 172.21.15.105:0/3141210596 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fef8406c9f0 0x7fef8406eeb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fefa0071840 tx=0x7fef90006bd0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.350+0000 7fef8b7fe640 1 -- 172.21.15.105:0/3141210596 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefa00c79a0 msgr2=0x7fefa00c7e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:33.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.350+0000 7fef8b7fe640 1 --2- 172.21.15.105:0/3141210596 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefa00c79a0 0x7fefa00c7e30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fef9c00b410 tx=0x7fef9c00b8d0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.351+0000 7fef8b7fe640 1 -- 172.21.15.105:0/3141210596 shutdown_connections 2024-04-15T15:19:33.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.351+0000 7fef8b7fe640 1 --2- 172.21.15.105:0/3141210596 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fef8406c9f0 0x7fef8406eeb0 secure :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fefa0071840 tx=0x7fef90006bd0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.351+0000 7fef8b7fe640 1 --2- 172.21.15.105:0/3141210596 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefa00c79a0 0x7fefa00c7e30 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.351+0000 7fef8b7fe640 1 --2- 172.21.15.105:0/3141210596 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fefa00a94a0 0x7fefa00c7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.351+0000 7fef8b7fe640 1 --2- 172.21.15.105:0/3141210596 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fefa00cf3f0 0x7fefa008ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.351+0000 7fef8b7fe640 1 -- 172.21.15.105:0/3141210596 >> 172.21.15.105:0/3141210596 conn(0x7fefa006bde0 msgr2=0x7fefa0072250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:33.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.352+0000 7fef8b7fe640 1 -- 172.21.15.105:0/3141210596 shutdown_connections 2024-04-15T15:19:33.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.352+0000 7fef8b7fe640 1 -- 172.21.15.105:0/3141210596 wait complete. 2024-04-15T15:19:33.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.353+0000 7f6b89a2a640 1 -- 172.21.15.105:0/25538698 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f6b60005580 con 0x7f6b7c053680 2024-04-15T15:19:33.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.354+0000 7f6b73fff640 1 -- 172.21.15.105:0/25538698 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6b7400d040 con 0x7f6b7c053680 2024-04-15T15:19:33.355 INFO:teuthology.orchestra.run.smithi105.stdout:55834574855 2024-04-15T15:19:33.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.357+0000 7f6b71ffb640 1 -- 172.21.15.105:0/25538698 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f6b6806c9c0 msgr2=0x7f6b6806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:33.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.357+0000 7f6b71ffb640 1 --2- 172.21.15.105:0/25538698 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f6b6806c9c0 0x7f6b6806ee80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c037d20 tx=0x7f6b8409d000 comp rx=0 tx=0).stop 2024-04-15T15:19:33.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.357+0000 7f6b71ffb640 1 -- 172.21.15.105:0/25538698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b7c053680 msgr2=0x7f6b7c0f4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:33.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.357+0000 7f6b71ffb640 1 --2- 172.21.15.105:0/25538698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b7c053680 0x7f6b7c0f4e50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6b7400b470 tx=0x7f6b7400b930 comp rx=0 tx=0).stop 2024-04-15T15:19:33.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.358+0000 7f6b71ffb640 1 -- 172.21.15.105:0/25538698 shutdown_connections 2024-04-15T15:19:33.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.358+0000 7f6b71ffb640 1 --2- 172.21.15.105:0/25538698 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6b7c100cf0 0x7f6b7c04cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.358+0000 7f6b71ffb640 1 --2- 172.21.15.105:0/25538698 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f6b6806c9c0 0x7f6b6806ee80 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.358+0000 7f6b71ffb640 1 --2- 172.21.15.105:0/25538698 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b7c104970 0x7f6b7c04d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.358+0000 7f6b71ffb640 1 --2- 172.21.15.105:0/25538698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b7c053680 0x7f6b7c0f4e50 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.358+0000 7f6b71ffb640 1 -- 172.21.15.105:0/25538698 >> 172.21.15.105:0/25538698 conn(0x7f6b7c009fe0 msgr2=0x7f6b7c0fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:33.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.358+0000 7f6b71ffb640 1 -- 172.21.15.105:0/25538698 shutdown_connections 2024-04-15T15:19:33.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.359+0000 7f6b71ffb640 1 -- 172.21.15.105:0/25538698 wait complete. 2024-04-15T15:19:33.371 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.6 2024-04-15T15:19:33.371 DEBUG:teuthology.parallel:result is None 2024-04-15T15:19:33.377 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.5 2024-04-15T15:19:33.377 DEBUG:teuthology.parallel:result is None 2024-04-15T15:19:33.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.387+0000 7ff0b4552640 1 -- 172.21.15.105:0/382641105 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7ff0ac0cf8c0 con 0x7ff0ac15b8b0 2024-04-15T15:19:33.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.388+0000 7ff0a2ffd640 1 -- 172.21.15.105:0/382641105 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff09c05d050 con 0x7ff0ac15b8b0 2024-04-15T15:19:33.390 INFO:teuthology.orchestra.run.smithi105.stdout:55834574854 2024-04-15T15:19:33.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.390+0000 7ff0b4552640 1 -- 172.21.15.105:0/382641105 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff09006c9c0 msgr2=0x7ff09006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:33.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.390+0000 7ff0b4552640 1 --2- 172.21.15.105:0/382641105 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff09006c9c0 0x7ff09006ee80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff0a8000f60 tx=0x7ff0a8007c70 comp rx=0 tx=0).stop 2024-04-15T15:19:33.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.390+0000 7ff0b4552640 1 -- 172.21.15.105:0/382641105 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0ac15b8b0 msgr2=0x7ff0ac1406d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:33.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.390+0000 7ff0b4552640 1 --2- 172.21.15.105:0/382641105 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0ac15b8b0 0x7ff0ac1406d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff09c00c400 tx=0x7ff09c00c8c0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.390+0000 7ff0b4552640 1 -- 172.21.15.105:0/382641105 shutdown_connections 2024-04-15T15:19:33.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.390+0000 7ff0b4552640 1 --2- 172.21.15.105:0/382641105 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0ac1637a0 0x7ff0ac14fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.390+0000 7ff0b4552640 1 --2- 172.21.15.105:0/382641105 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff09006c9c0 0x7ff09006ee80 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.390+0000 7ff0b4552640 1 --2- 172.21.15.105:0/382641105 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0ac15b8b0 0x7ff0ac1406d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.390+0000 7ff0b4552640 1 --2- 172.21.15.105:0/382641105 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0ac0b4b20 0x7ff0ac14ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.390+0000 7ff0b4552640 1 -- 172.21.15.105:0/382641105 >> 172.21.15.105:0/382641105 conn(0x7ff0ac06bde0 msgr2=0x7ff0ac15ccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:33.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.390+0000 7ff0b4552640 1 -- 172.21.15.105:0/382641105 shutdown_connections 2024-04-15T15:19:33.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.390+0000 7ff0b4552640 1 -- 172.21.15.105:0/382641105 wait complete. 2024-04-15T15:19:33.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.394+0000 7f2931074640 1 -- 172.21.15.105:0/3232439012 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f292c0d0ae0 con 0x7f292c161e40 2024-04-15T15:19:33.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.395+0000 7f29137fe640 1 -- 172.21.15.105:0/3232439012 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f292408dff0 con 0x7f292c161e40 2024-04-15T15:19:33.397 INFO:teuthology.orchestra.run.smithi105.stdout:55834574855 2024-04-15T15:19:33.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.397+0000 7f2931074640 1 -- 172.21.15.105:0/3232439012 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f291806c8f0 msgr2=0x7f291806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:33.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.397+0000 7f2931074640 1 --2- 172.21.15.105:0/3232439012 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f291806c8f0 0x7f291806edb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2920009670 tx=0x7f2920006bf0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.397+0000 7f2931074640 1 -- 172.21.15.105:0/3232439012 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f292c161e40 msgr2=0x7f292c0b9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:33.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.397+0000 7f2931074640 1 --2- 172.21.15.105:0/3232439012 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f292c161e40 0x7f292c0b9280 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2924009f40 tx=0x7f2924004a40 comp rx=0 tx=0).stop 2024-04-15T15:19:33.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.397+0000 7f2931074640 1 -- 172.21.15.105:0/3232439012 shutdown_connections 2024-04-15T15:19:33.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.397+0000 7f2931074640 1 --2- 172.21.15.105:0/3232439012 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f292c0d0150 0x7f292c0b7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.397+0000 7f2931074640 1 --2- 172.21.15.105:0/3232439012 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f291806c8f0 0x7f291806edb0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.397+0000 7f2931074640 1 --2- 172.21.15.105:0/3232439012 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f292c161e40 0x7f292c0b9280 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.397+0000 7f2931074640 1 --2- 172.21.15.105:0/3232439012 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f292c0cfcf0 0x7f292c152a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.397+0000 7f2931074640 1 -- 172.21.15.105:0/3232439012 >> 172.21.15.105:0/3232439012 conn(0x7f292c08bfd0 msgr2=0x7f292c0d0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:33.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.397+0000 7f2931074640 1 -- 172.21.15.105:0/3232439012 shutdown_connections 2024-04-15T15:19:33.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.397+0000 7f2931074640 1 -- 172.21.15.105:0/3232439012 wait complete. 2024-04-15T15:19:33.404 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.4 2024-04-15T15:19:33.404 DEBUG:teuthology.parallel:result is None 2024-04-15T15:19:33.410 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.0 2024-04-15T15:19:33.411 DEBUG:teuthology.parallel:result is None 2024-04-15T15:19:33.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.416+0000 7f5a570f7640 1 -- 172.21.15.105:0/1962303513 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f5a38005850 con 0x7f5a480e4d30 2024-04-15T15:19:33.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.416+0000 7f5a46ffd640 1 -- 172.21.15.105:0/1962303513 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5a4c059fc0 con 0x7f5a480e4d30 2024-04-15T15:19:33.418 INFO:teuthology.orchestra.run.smithi105.stdout:55834574855 2024-04-15T15:19:33.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.419+0000 7f5a570f7640 1 -- 172.21.15.105:0/1962303513 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f5a3406c920 msgr2=0x7f5a3406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:33.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.419+0000 7f5a570f7640 1 --2- 172.21.15.105:0/1962303513 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f5a3406c920 0x7f5a3406ede0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5a40036000 tx=0x7f5a40008030 comp rx=0 tx=0).stop 2024-04-15T15:19:33.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.419+0000 7f5a570f7640 1 -- 172.21.15.105:0/1962303513 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a480e4d30 msgr2=0x7f5a480d2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:33.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.419+0000 7f5a570f7640 1 --2- 172.21.15.105:0/1962303513 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a480e4d30 0x7f5a480d2d70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c00c780 tx=0x7f5a4c00cb80 comp rx=0 tx=0).stop 2024-04-15T15:19:33.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.419+0000 7f5a570f7640 1 -- 172.21.15.105:0/1962303513 shutdown_connections 2024-04-15T15:19:33.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.419+0000 7f5a570f7640 1 --2- 172.21.15.105:0/1962303513 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f5a3406c920 0x7f5a3406ede0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.419+0000 7f5a570f7640 1 --2- 172.21.15.105:0/1962303513 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a480d32b0 0x7f5a48008000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.419+0000 7f5a570f7640 1 --2- 172.21.15.105:0/1962303513 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a480e4d30 0x7f5a480d2d70 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.419+0000 7f5a570f7640 1 --2- 172.21.15.105:0/1962303513 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5a480e12b0 0x7f5a480d2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.419+0000 7f5a570f7640 1 -- 172.21.15.105:0/1962303513 >> 172.21.15.105:0/1962303513 conn(0x7f5a48020040 msgr2=0x7f5a4800c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:33.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.419+0000 7f5a570f7640 1 -- 172.21.15.105:0/1962303513 shutdown_connections 2024-04-15T15:19:33.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.419+0000 7f5a570f7640 1 -- 172.21.15.105:0/1962303513 wait complete. 2024-04-15T15:19:33.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.422+0000 7fbc8d874640 1 -- 172.21.15.105:0/392281873 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fbc60005850 con 0x7fbc88159c10 2024-04-15T15:19:33.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.422+0000 7fbc7ffff640 1 -- 172.21.15.105:0/392281873 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbc88151f70 con 0x7fbc88159c10 2024-04-15T15:19:33.424 INFO:teuthology.orchestra.run.smithi105.stdout:55834574854 2024-04-15T15:19:33.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.424+0000 7fbc8d874640 1 -- 172.21.15.105:0/392281873 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fbc6c06ccf0 msgr2=0x7fbc6c06f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:33.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.424+0000 7fbc8d874640 1 --2- 172.21.15.105:0/392281873 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fbc6c06ccf0 0x7fbc6c06f1b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbc8806cee0 tx=0x7fbc80008040 comp rx=0 tx=0).stop 2024-04-15T15:19:33.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.424+0000 7fbc8d874640 1 -- 172.21.15.105:0/392281873 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc88159c10 msgr2=0x7fbc88151a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:33.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.424+0000 7fbc8d874640 1 --2- 172.21.15.105:0/392281873 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc88159c10 0x7fbc88151a30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbc70031000 tx=0x7fbc70001360 comp rx=0 tx=0).stop 2024-04-15T15:19:33.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.424+0000 7fbc8d874640 1 -- 172.21.15.105:0/392281873 shutdown_connections 2024-04-15T15:19:33.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.424+0000 7fbc8d874640 1 --2- 172.21.15.105:0/392281873 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbc8815f650 0x7fbc881603a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.424+0000 7fbc8d874640 1 --2- 172.21.15.105:0/392281873 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fbc6c06ccf0 0x7fbc6c06f1b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.424+0000 7fbc8d874640 1 --2- 172.21.15.105:0/392281873 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc88159c10 0x7fbc88151a30 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.424+0000 7fbc8d874640 1 --2- 172.21.15.105:0/392281873 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc68004800 0x7fbc880b7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.424+0000 7fbc8d874640 1 -- 172.21.15.105:0/392281873 >> 172.21.15.105:0/392281873 conn(0x7fbc880d25a0 msgr2=0x7fbc8806e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:33.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.425+0000 7fbc8d874640 1 -- 172.21.15.105:0/392281873 shutdown_connections 2024-04-15T15:19:33.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.425+0000 7fbc8d874640 1 -- 172.21.15.105:0/392281873 wait complete. 2024-04-15T15:19:33.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.425+0000 7f4ef93a1640 1 -- 172.21.15.105:0/102749193 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f4eec057480 con 0x7f4ee40051c0 2024-04-15T15:19:33.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.425+0000 7f4ef0ff9640 1 -- 172.21.15.105:0/102749193 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4ef4068f40 con 0x7f4ee40051c0 2024-04-15T15:19:33.428 INFO:teuthology.orchestra.run.smithi105.stdout:55834574856 2024-04-15T15:19:33.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.428+0000 7f4ef93a1640 1 -- 172.21.15.105:0/102749193 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f4ed806c940 msgr2=0x7f4ed806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:33.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.428+0000 7f4ef93a1640 1 --2- 172.21.15.105:0/102749193 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f4ed806c940 0x7f4ed806ee00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4eec007df0 tx=0x7f4ee0008040 comp rx=0 tx=0).stop 2024-04-15T15:19:33.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.428+0000 7f4ef93a1640 1 -- 172.21.15.105:0/102749193 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ee40051c0 msgr2=0x7f4eec0de840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:33.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.428+0000 7f4ef93a1640 1 --2- 172.21.15.105:0/102749193 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ee40051c0 0x7f4eec0de840 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4ef40654e0 tx=0x7f4ef40724b0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.429+0000 7f4ef93a1640 1 -- 172.21.15.105:0/102749193 shutdown_connections 2024-04-15T15:19:33.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.429+0000 7f4ef93a1640 1 --2- 172.21.15.105:0/102749193 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4eec052b40 0x7f4eec052ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.429+0000 7f4ef93a1640 1 --2- 172.21.15.105:0/102749193 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f4ed806c940 0x7f4ed806ee00 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.429+0000 7f4ef93a1640 1 --2- 172.21.15.105:0/102749193 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4eec0d2b10 0x7f4eec0df4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.429+0000 7f4ef93a1640 1 --2- 172.21.15.105:0/102749193 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ee40051c0 0x7f4eec0de840 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.429+0000 7f4ef93a1640 1 -- 172.21.15.105:0/102749193 >> 172.21.15.105:0/102749193 conn(0x7f4eec020040 msgr2=0x7f4eec008160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:33.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.429+0000 7f4ef93a1640 1 -- 172.21.15.105:0/102749193 shutdown_connections 2024-04-15T15:19:33.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.429+0000 7f4ef93a1640 1 -- 172.21.15.105:0/102749193 wait complete. 2024-04-15T15:19:33.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.431+0000 7f94f9192640 1 -- 172.21.15.105:0/855079544 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f94ec033a20 con 0x7f94ec033ef0 2024-04-15T15:19:33.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.431+0000 7f94f0ff9640 1 -- 172.21.15.105:0/855079544 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f94e807fff0 con 0x7f94ec033ef0 2024-04-15T15:19:33.433 INFO:teuthology.orchestra.run.smithi105.stdout:55834574855 2024-04-15T15:19:33.434 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.1 2024-04-15T15:19:33.434 DEBUG:teuthology.parallel:result is None 2024-04-15T15:19:33.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.433+0000 7f94f9192640 1 -- 172.21.15.105:0/855079544 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f94d806c8f0 msgr2=0x7f94d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:33.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.433+0000 7f94f9192640 1 --2- 172.21.15.105:0/855079544 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f94d806c8f0 0x7f94d806edb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f94ec0e1b40 tx=0x7f94e00090e0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.433+0000 7f94f9192640 1 -- 172.21.15.105:0/855079544 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f94ec033ef0 msgr2=0x7f94ec04aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:33.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.433+0000 7f94f9192640 1 --2- 172.21.15.105:0/855079544 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f94ec033ef0 0x7f94ec04aa30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f94e800de50 tx=0x7f94e80046c0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.433+0000 7f94f9192640 1 -- 172.21.15.105:0/855079544 shutdown_connections 2024-04-15T15:19:33.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.433+0000 7f94f9192640 1 --2- 172.21.15.105:0/855079544 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f94ec053c70 0x7f94ec04e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.433+0000 7f94f9192640 1 --2- 172.21.15.105:0/855079544 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f94d806c8f0 0x7f94d806edb0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.433+0000 7f94f9192640 1 --2- 172.21.15.105:0/855079544 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f94ec0df220 0x7f94ec00a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.433+0000 7f94f9192640 1 --2- 172.21.15.105:0/855079544 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f94ec033ef0 0x7f94ec04aa30 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.433+0000 7f94f9192640 1 -- 172.21.15.105:0/855079544 >> 172.21.15.105:0/855079544 conn(0x7f94ec009fb0 msgr2=0x7f94ec0dc810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:33.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.433+0000 7f94f9192640 1 -- 172.21.15.105:0/855079544 shutdown_connections 2024-04-15T15:19:33.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.433+0000 7f94f9192640 1 -- 172.21.15.105:0/855079544 wait complete. 2024-04-15T15:19:33.439 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.7 2024-04-15T15:19:33.440 DEBUG:teuthology.parallel:result is None 2024-04-15T15:19:33.447 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.2 2024-04-15T15:19:33.447 DEBUG:teuthology.parallel:result is None 2024-04-15T15:19:33.449 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.3 2024-04-15T15:19:33.449 DEBUG:teuthology.parallel:result is None 2024-04-15T15:19:33.449 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-15T15:19:33.449 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-15T15:19:33.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.591+0000 7f85133f2640 1 Processor -- start 2024-04-15T15:19:33.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.591+0000 7f85133f2640 1 -- start start 2024-04-15T15:19:33.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.592+0000 7f85133f2640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f850c15ecf0 0x7f850c06e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:33.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.592+0000 7f85133f2640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f850c0cd360 0x7f850c162960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:33.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.592+0000 7f85133f2640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f850c06d2a0 con 0x7f850c0d0f10 2024-04-15T15:19:33.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.592+0000 7f85133f2640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f850c06d580 con 0x7f850c0cd360 2024-04-15T15:19:33.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.592+0000 7f85133f2640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f850c06d860 con 0x7f850c15ecf0 2024-04-15T15:19:33.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.592+0000 7f851198f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f850c0cd360 0x7f850c162960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:33.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.592+0000 7f851098d640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f850c0d0f10 0x7f850c0ccc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41042/0 (socket says 172.21.15.105:41042) 2024-04-15T15:19:33.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.592+0000 7f851098d640 1 -- 172.21.15.105:0/2295805917 learned_addr learned my addr 172.21.15.105:0/2295805917 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:19:33.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.593+0000 7f851118e640 1 --2- 172.21.15.105:0/2295805917 >> v2:172.21.15.154:3300/0 conn(0x7f850c15ecf0 0x7f850c06e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:33.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.593+0000 7f851198f640 1 -- 172.21.15.105:0/2295805917 >> v2:172.21.15.154:3300/0 conn(0x7f850c15ecf0 msgr2=0x7f850c06e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:33.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.593+0000 7f851198f640 1 --2- 172.21.15.105:0/2295805917 >> v2:172.21.15.154:3300/0 conn(0x7f850c15ecf0 0x7f850c06e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.593+0000 7f851198f640 1 -- 172.21.15.105:0/2295805917 >> v1:172.21.15.105:6789/0 conn(0x7f850c0d0f10 legacy=0x7f850c0ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:33.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.593+0000 7f851198f640 1 -- 172.21.15.105:0/2295805917 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f850c06db40 con 0x7f850c0cd360 2024-04-15T15:19:33.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.593+0000 7f8503fff640 1 -- 172.21.15.105:0/2295805917 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2682172041 0 0) 0x7f850c06d2a0 con 0x7f850c0d0f10 2024-04-15T15:19:33.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.593+0000 7f851198f640 1 --2- 172.21.15.105:0/2295805917 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f850c0cd360 0x7f850c162960 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f850c075520 tx=0x7f850402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6826867251f0e607 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:33.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.594+0000 7f8503fff640 1 -- 172.21.15.105:0/2295805917 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8504039070 con 0x7f850c0cd360 2024-04-15T15:19:33.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.594+0000 7f8503fff640 1 -- 172.21.15.105:0/2295805917 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f850402cc80 con 0x7f850c0cd360 2024-04-15T15:19:33.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.594+0000 7f8503fff640 1 -- 172.21.15.105:0/2295805917 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8504034660 con 0x7f850c0cd360 2024-04-15T15:19:33.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.594+0000 7f85133f2640 1 -- 172.21.15.105:0/2295805917 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f850c0cd360 msgr2=0x7f850c162960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:33.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.594+0000 7f85133f2640 1 --2- 172.21.15.105:0/2295805917 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f850c0cd360 0x7f850c162960 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f850c075520 tx=0x7f850402bed0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.594+0000 7f85133f2640 1 -- 172.21.15.105:0/2295805917 shutdown_connections 2024-04-15T15:19:33.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.594+0000 7f85133f2640 1 --2- 172.21.15.105:0/2295805917 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f850c0cd360 0x7f850c162960 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.594+0000 7f85133f2640 1 --2- 172.21.15.105:0/2295805917 >> v2:172.21.15.154:3300/0 conn(0x7f850c15ecf0 0x7f850c06e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.594+0000 7f85133f2640 1 -- 172.21.15.105:0/2295805917 >> 172.21.15.105:0/2295805917 conn(0x7f850c0d2600 msgr2=0x7f850c0d2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:33.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.594+0000 7f85133f2640 1 -- 172.21.15.105:0/2295805917 shutdown_connections 2024-04-15T15:19:33.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.594+0000 7f85133f2640 1 -- 172.21.15.105:0/2295805917 wait complete. 2024-04-15T15:19:33.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.595+0000 7f85133f2640 1 Processor -- start 2024-04-15T15:19:33.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.595+0000 7f85133f2640 1 -- start start 2024-04-15T15:19:33.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.596+0000 7f85133f2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f850c0cd360 0x7f850c0b5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:33.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.596+0000 7f85133f2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f850c0d0f10 0x7f850c0b5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:33.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.596+0000 7f85133f2640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f850c15ecf0 0x7f850c14b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:33.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.596+0000 7f85133f2640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f850c15f0d0 con 0x7f850c0d0f10 2024-04-15T15:19:33.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.596+0000 7f85133f2640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f850c0d1370 con 0x7f850c0cd360 2024-04-15T15:19:33.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.596+0000 7f85133f2640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f850c15f250 con 0x7f850c15ecf0 2024-04-15T15:19:33.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.596+0000 7f851098d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f850c0d0f10 0x7f850c0b5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:33.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.596+0000 7f851098d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f850c0d0f10 0x7f850c0b5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39838/0 (socket says 172.21.15.105:39838) 2024-04-15T15:19:33.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.596+0000 7f851098d640 1 -- 172.21.15.105:0/168398753 learned_addr learned my addr 172.21.15.105:0/168398753 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:33.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.596+0000 7f851118e640 1 --2- 172.21.15.105:0/168398753 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f850c0cd360 0x7f850c0b5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:33.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.596+0000 7f851198f640 1 --2- 172.21.15.105:0/168398753 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f850c15ecf0 0x7f850c14b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:33.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.596+0000 7f851098d640 1 -- 172.21.15.105:0/168398753 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f850c15ecf0 msgr2=0x7f850c14b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:33.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.596+0000 7f851098d640 1 --2- 172.21.15.105:0/168398753 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f850c15ecf0 0x7f850c14b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.596+0000 7f851098d640 1 -- 172.21.15.105:0/168398753 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f850c0cd360 msgr2=0x7f850c0b5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:33.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.596+0000 7f851098d640 1 --2- 172.21.15.105:0/168398753 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f850c0cd360 0x7f850c0b5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.596+0000 7f851098d640 1 -- 172.21.15.105:0/168398753 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f850c14c610 con 0x7f850c0d0f10 2024-04-15T15:19:33.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.596+0000 7f851118e640 1 --2- 172.21.15.105:0/168398753 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f850c0cd360 0x7f850c0b5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:33.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.597+0000 7f851098d640 1 --2- 172.21.15.105:0/168398753 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f850c0d0f10 0x7f850c0b5760 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f84fc00c4d0 tx=0x7f84fc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:33.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.597+0000 7f8501ffb640 1 -- 172.21.15.105:0/168398753 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84fc018070 con 0x7f850c0d0f10 2024-04-15T15:19:33.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.597+0000 7f8501ffb640 1 -- 172.21.15.105:0/168398753 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84fc007500 con 0x7f850c0d0f10 2024-04-15T15:19:33.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.597+0000 7f8501ffb640 1 -- 172.21.15.105:0/168398753 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84fc004a20 con 0x7f850c0d0f10 2024-04-15T15:19:33.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.597+0000 7f85133f2640 1 -- 172.21.15.105:0/168398753 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f850c14c8f0 con 0x7f850c0d0f10 2024-04-15T15:19:33.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.597+0000 7f85133f2640 1 -- 172.21.15.105:0/168398753 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f850c0c75a0 con 0x7f850c0d0f10 2024-04-15T15:19:33.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.598+0000 7f85133f2640 1 -- 172.21.15.105:0/168398753 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f850c0c70b0 con 0x7f850c0d0f10 2024-04-15T15:19:33.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.598+0000 7f8501ffb640 1 -- 172.21.15.105:0/168398753 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f84fc01b070 con 0x7f850c0d0f10 2024-04-15T15:19:33.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.599+0000 7f8501ffb640 1 --2- 172.21.15.105:0/168398753 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f84f006c8f0 0x7f84f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:33.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.599+0000 7f8501ffb640 1 -- 172.21.15.105:0/168398753 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f84fc08ae90 con 0x7f850c0d0f10 2024-04-15T15:19:33.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.599+0000 7f851118e640 1 --2- 172.21.15.105:0/168398753 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f84f006c8f0 0x7f84f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:33.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.599+0000 7f851118e640 1 --2- 172.21.15.105:0/168398753 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f84f006c8f0 0x7f84f006edb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8508006fd0 tx=0x7f8508008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:33.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.603+0000 7f8501ffb640 1 -- 172.21.15.105:0/168398753 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f84fc0592c0 con 0x7f850c0d0f10 2024-04-15T15:19:33.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.778+0000 7f85133f2640 1 -- 172.21.15.105:0/168398753 --> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f850c068a40 con 0x7f84f006c8f0 2024-04-15T15:19:33.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.784+0000 7f8501ffb640 1 -- 172.21.15.105:0/168398753 <== mgr.4107 v2:172.21.15.105:6832/3506549322 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251836 (secure 0 0 0) 0x7f850c068a40 con 0x7f84f006c8f0 2024-04-15T15:19:33.789 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:19:33.790 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-04-15T15:19:33.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.787+0000 7f85133f2640 1 -- 172.21.15.105:0/168398753 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f84f006c8f0 msgr2=0x7f84f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:33.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.787+0000 7f85133f2640 1 --2- 172.21.15.105:0/168398753 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f84f006c8f0 0x7f84f006edb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8508006fd0 tx=0x7f8508008040 comp rx=0 tx=0).stop 2024-04-15T15:19:33.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.787+0000 7f85133f2640 1 -- 172.21.15.105:0/168398753 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f850c0d0f10 msgr2=0x7f850c0b5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:33.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.787+0000 7f85133f2640 1 --2- 172.21.15.105:0/168398753 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f850c0d0f10 0x7f850c0b5760 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f84fc00c4d0 tx=0x7f84fc00c990 comp rx=0 tx=0).stop 2024-04-15T15:19:33.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.787+0000 7f85133f2640 1 -- 172.21.15.105:0/168398753 shutdown_connections 2024-04-15T15:19:33.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.787+0000 7f85133f2640 1 --2- 172.21.15.105:0/168398753 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f850c15ecf0 0x7f850c14b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.787+0000 7f85133f2640 1 --2- 172.21.15.105:0/168398753 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f84f006c8f0 0x7f84f006edb0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.787+0000 7f85133f2640 1 --2- 172.21.15.105:0/168398753 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f850c0d0f10 0x7f850c0b5760 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.787+0000 7f85133f2640 1 --2- 172.21.15.105:0/168398753 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f850c0cd360 0x7f850c0b5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.787+0000 7f85133f2640 1 -- 172.21.15.105:0/168398753 >> 172.21.15.105:0/168398753 conn(0x7f850c0d2600 msgr2=0x7f850c070050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:33.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.787+0000 7f85133f2640 1 -- 172.21.15.105:0/168398753 shutdown_connections 2024-04-15T15:19:33.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.787+0000 7f85133f2640 1 -- 172.21.15.105:0/168398753 wait complete. 2024-04-15T15:19:33.800 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-04-15T15:19:33.735404+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219024,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755696,"statfs":{"total":773094113280,"available":772869832704,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001646"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.724996+0000","last_change":"2024-04-15T15:19:26.440158+0000","last_active":"2024-04-15T15:19:27.724996+0000","last_peered":"2024-04-15T15:19:27.724996+0000","last_clean":"2024-04-15T15:19:27.724996+0000","last_became_active":"2024-04-15T15:19:26.439860+0000","last_became_peered":"2024-04-15T15:19:26.439860+0000","last_unstale":"2024-04-15T15:19:27.724996+0000","last_undegraded":"2024-04-15T15:19:27.724996+0000","last_fullsized":"2024-04-15T15:19:27.724996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:39:29.689447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.440059+0000","last_change":"2024-04-15T15:19:26.426289+0000","last_active":"2024-04-15T15:19:27.440059+0000","last_peered":"2024-04-15T15:19:27.440059+0000","last_clean":"2024-04-15T15:19:27.440059+0000","last_became_active":"2024-04-15T15:19:26.425937+0000","last_became_peered":"2024-04-15T15:19:26.425937+0000","last_unstale":"2024-04-15T15:19:27.440059+0000","last_undegraded":"2024-04-15T15:19:27.440059+0000","last_fullsized":"2024-04-15T15:19:27.440059+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:06:20.973059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.773961+0000","last_change":"2024-04-15T15:19:26.425640+0000","last_active":"2024-04-15T15:19:27.773961+0000","last_peered":"2024-04-15T15:19:27.773961+0000","last_clean":"2024-04-15T15:19:27.773961+0000","last_became_active":"2024-04-15T15:19:26.425228+0000","last_became_peered":"2024-04-15T15:19:26.425228+0000","last_unstale":"2024-04-15T15:19:27.773961+0000","last_undegraded":"2024-04-15T15:19:27.773961+0000","last_fullsized":"2024-04-15T15:19:27.773961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:17:45.061394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774465+0000","last_change":"2024-04-15T15:19:26.439598+0000","last_active":"2024-04-15T15:19:27.774465+0000","last_peered":"2024-04-15T15:19:27.774465+0000","last_clean":"2024-04-15T15:19:27.774465+0000","last_became_active":"2024-04-15T15:19:26.439290+0000","last_became_peered":"2024-04-15T15:19:26.439290+0000","last_unstale":"2024-04-15T15:19:27.774465+0000","last_undegraded":"2024-04-15T15:19:27.774465+0000","last_fullsized":"2024-04-15T15:19:27.774465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:35:32.148665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.724710+0000","last_change":"2024-04-15T15:19:26.433863+0000","last_active":"2024-04-15T15:19:27.724710+0000","last_peered":"2024-04-15T15:19:27.724710+0000","last_clean":"2024-04-15T15:19:27.724710+0000","last_became_active":"2024-04-15T15:19:26.433559+0000","last_became_peered":"2024-04-15T15:19:26.433559+0000","last_unstale":"2024-04-15T15:19:27.724710+0000","last_undegraded":"2024-04-15T15:19:27.724710+0000","last_fullsized":"2024-04-15T15:19:27.724710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T03:07:46.810235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774361+0000","last_change":"2024-04-15T15:19:26.442278+0000","last_active":"2024-04-15T15:19:27.774361+0000","last_peered":"2024-04-15T15:19:27.774361+0000","last_clean":"2024-04-15T15:19:27.774361+0000","last_became_active":"2024-04-15T15:19:26.441910+0000","last_became_peered":"2024-04-15T15:19:26.441910+0000","last_unstale":"2024-04-15T15:19:27.774361+0000","last_undegraded":"2024-04-15T15:19:27.774361+0000","last_fullsized":"2024-04-15T15:19:27.774361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:05:03.041116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.724443+0000","last_change":"2024-04-15T15:19:26.430368+0000","last_active":"2024-04-15T15:19:27.724443+0000","last_peered":"2024-04-15T15:19:27.724443+0000","last_clean":"2024-04-15T15:19:27.724443+0000","last_became_active":"2024-04-15T15:19:26.430083+0000","last_became_peered":"2024-04-15T15:19:26.430083+0000","last_unstale":"2024-04-15T15:19:27.724443+0000","last_undegraded":"2024-04-15T15:19:27.724443+0000","last_fullsized":"2024-04-15T15:19:27.724443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:52:38.985490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.417632+0000","last_change":"2024-04-15T15:19:26.444027+0000","last_active":"2024-04-15T15:19:27.417632+0000","last_peered":"2024-04-15T15:19:27.417632+0000","last_clean":"2024-04-15T15:19:27.417632+0000","last_became_active":"2024-04-15T15:19:26.443699+0000","last_became_peered":"2024-04-15T15:19:26.443699+0000","last_unstale":"2024-04-15T15:19:27.417632+0000","last_undegraded":"2024-04-15T15:19:27.417632+0000","last_fullsized":"2024-04-15T15:19:27.417632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:19:31.027967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.725018+0000","last_change":"2024-04-15T15:19:26.443898+0000","last_active":"2024-04-15T15:19:27.725018+0000","last_peered":"2024-04-15T15:19:27.725018+0000","last_clean":"2024-04-15T15:19:27.725018+0000","last_became_active":"2024-04-15T15:19:26.443600+0000","last_became_peered":"2024-04-15T15:19:26.443600+0000","last_unstale":"2024-04-15T15:19:27.725018+0000","last_undegraded":"2024-04-15T15:19:27.725018+0000","last_fullsized":"2024-04-15T15:19:27.725018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:07:04.317951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.420914+0000","last_change":"2024-04-15T15:19:26.445705+0000","last_active":"2024-04-15T15:19:27.420914+0000","last_peered":"2024-04-15T15:19:27.420914+0000","last_clean":"2024-04-15T15:19:27.420914+0000","last_became_active":"2024-04-15T15:19:26.445421+0000","last_became_peered":"2024-04-15T15:19:26.445421+0000","last_unstale":"2024-04-15T15:19:27.420914+0000","last_undegraded":"2024-04-15T15:19:27.420914+0000","last_fullsized":"2024-04-15T15:19:27.420914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:06:20.973059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.420805+0000","last_change":"2024-04-15T15:19:26.441881+0000","last_active":"2024-04-15T15:19:27.420805+0000","last_peered":"2024-04-15T15:19:27.420805+0000","last_clean":"2024-04-15T15:19:27.420805+0000","last_became_active":"2024-04-15T15:19:26.441610+0000","last_became_peered":"2024-04-15T15:19:26.441610+0000","last_unstale":"2024-04-15T15:19:27.420805+0000","last_undegraded":"2024-04-15T15:19:27.420805+0000","last_fullsized":"2024-04-15T15:19:27.420805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:55:46.969871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.421116+0000","last_change":"2024-04-15T15:19:26.443171+0000","last_active":"2024-04-15T15:19:27.421116+0000","last_peered":"2024-04-15T15:19:27.421116+0000","last_clean":"2024-04-15T15:19:27.421116+0000","last_became_active":"2024-04-15T15:19:26.442896+0000","last_became_peered":"2024-04-15T15:19:26.442896+0000","last_unstale":"2024-04-15T15:19:27.421116+0000","last_undegraded":"2024-04-15T15:19:27.421116+0000","last_fullsized":"2024-04-15T15:19:27.421116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:31:12.317442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.417978+0000","last_change":"2024-04-15T15:19:26.442097+0000","last_active":"2024-04-15T15:19:27.417978+0000","last_peered":"2024-04-15T15:19:27.417978+0000","last_clean":"2024-04-15T15:19:27.417978+0000","last_became_active":"2024-04-15T15:19:26.441823+0000","last_became_peered":"2024-04-15T15:19:26.441823+0000","last_unstale":"2024-04-15T15:19:27.417978+0000","last_undegraded":"2024-04-15T15:19:27.417978+0000","last_fullsized":"2024-04-15T15:19:27.417978+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:05:03.041116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.773787+0000","last_change":"2024-04-15T15:19:26.428713+0000","last_active":"2024-04-15T15:19:27.773787+0000","last_peered":"2024-04-15T15:19:27.773787+0000","last_clean":"2024-04-15T15:19:27.773787+0000","last_became_active":"2024-04-15T15:19:26.428434+0000","last_became_peered":"2024-04-15T15:19:26.428434+0000","last_unstale":"2024-04-15T15:19:27.773787+0000","last_undegraded":"2024-04-15T15:19:27.773787+0000","last_fullsized":"2024-04-15T15:19:27.773787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:03:15.458694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.439725+0000","last_change":"2024-04-15T15:19:26.436898+0000","last_active":"2024-04-15T15:19:27.439725+0000","last_peered":"2024-04-15T15:19:27.439725+0000","last_clean":"2024-04-15T15:19:27.439725+0000","last_became_active":"2024-04-15T15:19:26.436643+0000","last_became_peered":"2024-04-15T15:19:26.436643+0000","last_unstale":"2024-04-15T15:19:27.439725+0000","last_undegraded":"2024-04-15T15:19:27.439725+0000","last_fullsized":"2024-04-15T15:19:27.439725+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:57:45.149303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774246+0000","last_change":"2024-04-15T15:19:26.439598+0000","last_active":"2024-04-15T15:19:27.774246+0000","last_peered":"2024-04-15T15:19:27.774246+0000","last_clean":"2024-04-15T15:19:27.774246+0000","last_became_active":"2024-04-15T15:19:26.439335+0000","last_became_peered":"2024-04-15T15:19:26.439335+0000","last_unstale":"2024-04-15T15:19:27.774246+0000","last_undegraded":"2024-04-15T15:19:27.774246+0000","last_fullsized":"2024-04-15T15:19:27.774246+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:55:46.969871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774820+0000","last_change":"2024-04-15T15:19:26.445594+0000","last_active":"2024-04-15T15:19:27.774820+0000","last_peered":"2024-04-15T15:19:27.774820+0000","last_clean":"2024-04-15T15:19:27.774820+0000","last_became_active":"2024-04-15T15:19:26.445306+0000","last_became_peered":"2024-04-15T15:19:26.445306+0000","last_unstale":"2024-04-15T15:19:27.774820+0000","last_undegraded":"2024-04-15T15:19:27.774820+0000","last_fullsized":"2024-04-15T15:19:27.774820+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:47:07.377973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.554342+0000","last_change":"2024-04-15T15:19:26.424093+0000","last_active":"2024-04-15T15:19:27.554342+0000","last_peered":"2024-04-15T15:19:27.554342+0000","last_clean":"2024-04-15T15:19:27.554342+0000","last_became_active":"2024-04-15T15:19:26.423816+0000","last_became_peered":"2024-04-15T15:19:26.423816+0000","last_unstale":"2024-04-15T15:19:27.554342+0000","last_undegraded":"2024-04-15T15:19:27.554342+0000","last_fullsized":"2024-04-15T15:19:27.554342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:52:03.007433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.725002+0000","last_change":"2024-04-15T15:19:26.437633+0000","last_active":"2024-04-15T15:19:27.725002+0000","last_peered":"2024-04-15T15:19:27.725002+0000","last_clean":"2024-04-15T15:19:27.725002+0000","last_became_active":"2024-04-15T15:19:26.437000+0000","last_became_peered":"2024-04-15T15:19:26.437000+0000","last_unstale":"2024-04-15T15:19:27.725002+0000","last_undegraded":"2024-04-15T15:19:27.725002+0000","last_fullsized":"2024-04-15T15:19:27.725002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:49:01.189045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.420621+0000","last_change":"2024-04-15T15:19:26.431490+0000","last_active":"2024-04-15T15:19:27.420621+0000","last_peered":"2024-04-15T15:19:27.420621+0000","last_clean":"2024-04-15T15:19:27.420621+0000","last_became_active":"2024-04-15T15:19:26.431235+0000","last_became_peered":"2024-04-15T15:19:26.431235+0000","last_unstale":"2024-04-15T15:19:27.420621+0000","last_undegraded":"2024-04-15T15:19:27.420621+0000","last_fullsized":"2024-04-15T15:19:27.420621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:07:02.244933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774078+0000","last_change":"2024-04-15T15:19:26.433079+0000","last_active":"2024-04-15T15:19:27.774078+0000","last_peered":"2024-04-15T15:19:27.774078+0000","last_clean":"2024-04-15T15:19:27.774078+0000","last_became_active":"2024-04-15T15:19:26.432075+0000","last_became_peered":"2024-04-15T15:19:26.432075+0000","last_unstale":"2024-04-15T15:19:27.774078+0000","last_undegraded":"2024-04-15T15:19:27.774078+0000","last_fullsized":"2024-04-15T15:19:27.774078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:55:46.969871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774692+0000","last_change":"2024-04-15T15:19:26.442575+0000","last_active":"2024-04-15T15:19:27.774692+0000","last_peered":"2024-04-15T15:19:27.774692+0000","last_clean":"2024-04-15T15:19:27.774692+0000","last_became_active":"2024-04-15T15:19:26.441316+0000","last_became_peered":"2024-04-15T15:19:26.441316+0000","last_unstale":"2024-04-15T15:19:27.774692+0000","last_undegraded":"2024-04-15T15:19:27.774692+0000","last_fullsized":"2024-04-15T15:19:27.774692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T03:07:46.810235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.417731+0000","last_change":"2024-04-15T15:19:26.437545+0000","last_active":"2024-04-15T15:19:27.417731+0000","last_peered":"2024-04-15T15:19:27.417731+0000","last_clean":"2024-04-15T15:19:27.417731+0000","last_became_active":"2024-04-15T15:19:26.437018+0000","last_became_peered":"2024-04-15T15:19:26.437018+0000","last_unstale":"2024-04-15T15:19:27.417731+0000","last_undegraded":"2024-04-15T15:19:27.417731+0000","last_fullsized":"2024-04-15T15:19:27.417731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:57:45.149303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774315+0000","last_change":"2024-04-15T15:19:26.438924+0000","last_active":"2024-04-15T15:19:27.774315+0000","last_peered":"2024-04-15T15:19:27.774315+0000","last_clean":"2024-04-15T15:19:27.774315+0000","last_became_active":"2024-04-15T15:19:26.438615+0000","last_became_peered":"2024-04-15T15:19:26.438615+0000","last_unstale":"2024-04-15T15:19:27.774315+0000","last_undegraded":"2024-04-15T15:19:27.774315+0000","last_fullsized":"2024-04-15T15:19:27.774315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:06:20.973059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.417837+0000","last_change":"2024-04-15T15:19:26.446195+0000","last_active":"2024-04-15T15:19:27.417837+0000","last_peered":"2024-04-15T15:19:27.417837+0000","last_clean":"2024-04-15T15:19:27.417837+0000","last_became_active":"2024-04-15T15:19:26.445872+0000","last_became_peered":"2024-04-15T15:19:26.445872+0000","last_unstale":"2024-04-15T15:19:27.417837+0000","last_undegraded":"2024-04-15T15:19:27.417837+0000","last_fullsized":"2024-04-15T15:19:27.417837+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:07:02.244933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.421026+0000","last_change":"2024-04-15T15:19:26.429073+0000","last_active":"2024-04-15T15:19:27.421026+0000","last_peered":"2024-04-15T15:19:27.421026+0000","last_clean":"2024-04-15T15:19:27.421026+0000","last_became_active":"2024-04-15T15:19:26.428755+0000","last_became_peered":"2024-04-15T15:19:26.428755+0000","last_unstale":"2024-04-15T15:19:27.421026+0000","last_undegraded":"2024-04-15T15:19:27.421026+0000","last_fullsized":"2024-04-15T15:19:27.421026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:35:32.148665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.439805+0000","last_change":"2024-04-15T15:19:26.432582+0000","last_active":"2024-04-15T15:19:27.439805+0000","last_peered":"2024-04-15T15:19:27.439805+0000","last_clean":"2024-04-15T15:19:27.439805+0000","last_became_active":"2024-04-15T15:19:26.432334+0000","last_became_peered":"2024-04-15T15:19:26.432334+0000","last_unstale":"2024-04-15T15:19:27.439805+0000","last_undegraded":"2024-04-15T15:19:27.439805+0000","last_fullsized":"2024-04-15T15:19:27.439805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:07:02.244933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.418110+0000","last_change":"2024-04-15T15:19:26.436504+0000","last_active":"2024-04-15T15:19:27.418110+0000","last_peered":"2024-04-15T15:19:27.418110+0000","last_clean":"2024-04-15T15:19:27.418110+0000","last_became_active":"2024-04-15T15:19:26.436237+0000","last_became_peered":"2024-04-15T15:19:26.436237+0000","last_unstale":"2024-04-15T15:19:27.418110+0000","last_undegraded":"2024-04-15T15:19:27.418110+0000","last_fullsized":"2024-04-15T15:19:27.418110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:15:06.003211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774078+0000","last_change":"2024-04-15T15:19:26.429813+0000","last_active":"2024-04-15T15:19:27.774078+0000","last_peered":"2024-04-15T15:19:27.774078+0000","last_clean":"2024-04-15T15:19:27.774078+0000","last_became_active":"2024-04-15T15:19:26.429507+0000","last_became_peered":"2024-04-15T15:19:26.429507+0000","last_unstale":"2024-04-15T15:19:27.774078+0000","last_undegraded":"2024-04-15T15:19:27.774078+0000","last_fullsized":"2024-04-15T15:19:27.774078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:06:20.973059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774710+0000","last_change":"2024-04-15T15:19:26.442277+0000","last_active":"2024-04-15T15:19:27.774710+0000","last_peered":"2024-04-15T15:19:27.774710+0000","last_clean":"2024-04-15T15:19:27.774710+0000","last_became_active":"2024-04-15T15:19:26.441630+0000","last_became_peered":"2024-04-15T15:19:26.441630+0000","last_unstale":"2024-04-15T15:19:27.774710+0000","last_undegraded":"2024-04-15T15:19:27.774710+0000","last_fullsized":"2024-04-15T15:19:27.774710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:06:21.060519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.554197+0000","last_change":"2024-04-15T15:19:26.439042+0000","last_active":"2024-04-15T15:19:27.554197+0000","last_peered":"2024-04-15T15:19:27.554197+0000","last_clean":"2024-04-15T15:19:27.554197+0000","last_became_active":"2024-04-15T15:19:26.438709+0000","last_became_peered":"2024-04-15T15:19:26.438709+0000","last_unstale":"2024-04-15T15:19:27.554197+0000","last_undegraded":"2024-04-15T15:19:27.554197+0000","last_fullsized":"2024-04-15T15:19:27.554197+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:21:56.467880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.420350+0000","last_change":"2024-04-15T15:19:26.423461+0000","last_active":"2024-04-15T15:19:27.420350+0000","last_peered":"2024-04-15T15:19:27.420350+0000","last_clean":"2024-04-15T15:19:27.420350+0000","last_became_active":"2024-04-15T15:19:26.423171+0000","last_became_peered":"2024-04-15T15:19:26.423171+0000","last_unstale":"2024-04-15T15:19:27.420350+0000","last_undegraded":"2024-04-15T15:19:27.420350+0000","last_fullsized":"2024-04-15T15:19:27.420350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:57:45.149303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.420526+0000","last_change":"2024-04-15T15:19:26.431616+0000","last_active":"2024-04-15T15:19:27.420526+0000","last_peered":"2024-04-15T15:19:27.420526+0000","last_clean":"2024-04-15T15:19:27.420526+0000","last_became_active":"2024-04-15T15:19:26.430999+0000","last_became_peered":"2024-04-15T15:19:26.430999+0000","last_unstale":"2024-04-15T15:19:27.420526+0000","last_undegraded":"2024-04-15T15:19:27.420526+0000","last_fullsized":"2024-04-15T15:19:27.420526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:55:34.679094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.418188+0000","last_change":"2024-04-15T15:19:25.418290+0000","last_active":"2024-04-15T15:19:27.418188+0000","last_peered":"2024-04-15T15:19:27.418188+0000","last_clean":"2024-04-15T15:19:27.418188+0000","last_became_active":"2024-04-15T15:19:25.418036+0000","last_became_peered":"2024-04-15T15:19:25.418036+0000","last_unstale":"2024-04-15T15:19:27.418188+0000","last_undegraded":"2024-04-15T15:19:27.418188+0000","last_fullsized":"2024-04-15T15:19:27.418188+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:35:36.249861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774176+0000","last_change":"2024-04-15T15:19:26.431800+0000","last_active":"2024-04-15T15:19:27.774176+0000","last_peered":"2024-04-15T15:19:27.774176+0000","last_clean":"2024-04-15T15:19:27.774176+0000","last_became_active":"2024-04-15T15:19:26.431535+0000","last_became_peered":"2024-04-15T15:19:26.431535+0000","last_unstale":"2024-04-15T15:19:27.774176+0000","last_undegraded":"2024-04-15T15:19:27.774176+0000","last_fullsized":"2024-04-15T15:19:27.774176+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:05:03.041116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.421045+0000","last_change":"2024-04-15T15:19:25.422350+0000","last_active":"2024-04-15T15:19:27.421045+0000","last_peered":"2024-04-15T15:19:27.421045+0000","last_clean":"2024-04-15T15:19:27.421045+0000","last_became_active":"2024-04-15T15:19:25.422091+0000","last_became_peered":"2024-04-15T15:19:25.422091+0000","last_unstale":"2024-04-15T15:19:27.421045+0000","last_undegraded":"2024-04-15T15:19:27.421045+0000","last_fullsized":"2024-04-15T15:19:27.421045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:55:33.675151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.417601+0000","last_change":"2024-04-15T15:19:25.427262+0000","last_active":"2024-04-15T15:19:27.417601+0000","last_peered":"2024-04-15T15:19:27.417601+0000","last_clean":"2024-04-15T15:19:27.417601+0000","last_became_active":"2024-04-15T15:19:25.427057+0000","last_became_peered":"2024-04-15T15:19:25.427057+0000","last_unstale":"2024-04-15T15:19:27.417601+0000","last_undegraded":"2024-04-15T15:19:27.417601+0000","last_fullsized":"2024-04-15T15:19:27.417601+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:39:21.336486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.417662+0000","last_change":"2024-04-15T15:19:26.434134+0000","last_active":"2024-04-15T15:19:27.417662+0000","last_peered":"2024-04-15T15:19:27.417662+0000","last_clean":"2024-04-15T15:19:27.417662+0000","last_became_active":"2024-04-15T15:19:26.433852+0000","last_became_peered":"2024-04-15T15:19:26.433852+0000","last_unstale":"2024-04-15T15:19:27.417662+0000","last_undegraded":"2024-04-15T15:19:27.417662+0000","last_fullsized":"2024-04-15T15:19:27.417662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:17:45.061394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774370+0000","last_change":"2024-04-15T15:19:25.423330+0000","last_active":"2024-04-15T15:19:27.774370+0000","last_peered":"2024-04-15T15:19:27.774370+0000","last_clean":"2024-04-15T15:19:27.774370+0000","last_became_active":"2024-04-15T15:19:25.423045+0000","last_became_peered":"2024-04-15T15:19:25.423045+0000","last_unstale":"2024-04-15T15:19:27.774370+0000","last_undegraded":"2024-04-15T15:19:27.774370+0000","last_fullsized":"2024-04-15T15:19:27.774370+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:55:33.675151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.439704+0000","last_change":"2024-04-15T15:19:26.432451+0000","last_active":"2024-04-15T15:19:27.439704+0000","last_peered":"2024-04-15T15:19:27.439704+0000","last_clean":"2024-04-15T15:19:27.439704+0000","last_became_active":"2024-04-15T15:19:26.432197+0000","last_became_peered":"2024-04-15T15:19:26.432197+0000","last_unstale":"2024-04-15T15:19:27.439704+0000","last_undegraded":"2024-04-15T15:19:27.439704+0000","last_fullsized":"2024-04-15T15:19:27.439704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:17:45.061394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.420612+0000","last_change":"2024-04-15T15:19:25.423680+0000","last_active":"2024-04-15T15:19:27.420612+0000","last_peered":"2024-04-15T15:19:27.420612+0000","last_clean":"2024-04-15T15:19:27.420612+0000","last_became_active":"2024-04-15T15:19:25.423361+0000","last_became_peered":"2024-04-15T15:19:25.423361+0000","last_unstale":"2024-04-15T15:19:27.420612+0000","last_undegraded":"2024-04-15T15:19:27.420612+0000","last_fullsized":"2024-04-15T15:19:27.420612+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:30:23.903641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774647+0000","last_change":"2024-04-15T15:19:26.446276+0000","last_active":"2024-04-15T15:19:27.774647+0000","last_peered":"2024-04-15T15:19:27.774647+0000","last_clean":"2024-04-15T15:19:27.774647+0000","last_became_active":"2024-04-15T15:19:26.445956+0000","last_became_peered":"2024-04-15T15:19:26.445956+0000","last_unstale":"2024-04-15T15:19:27.774647+0000","last_undegraded":"2024-04-15T15:19:27.774647+0000","last_fullsized":"2024-04-15T15:19:27.774647+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:15:06.003211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774626+0000","last_change":"2024-04-15T15:19:25.427557+0000","last_active":"2024-04-15T15:19:27.774626+0000","last_peered":"2024-04-15T15:19:27.774626+0000","last_clean":"2024-04-15T15:19:27.774626+0000","last_became_active":"2024-04-15T15:19:25.427264+0000","last_became_peered":"2024-04-15T15:19:25.427264+0000","last_unstale":"2024-04-15T15:19:27.774626+0000","last_undegraded":"2024-04-15T15:19:27.774626+0000","last_fullsized":"2024-04-15T15:19:27.774626+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:30:23.903641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.773986+0000","last_change":"2024-04-15T15:19:26.443195+0000","last_active":"2024-04-15T15:19:27.773986+0000","last_peered":"2024-04-15T15:19:27.773986+0000","last_clean":"2024-04-15T15:19:27.773986+0000","last_became_active":"2024-04-15T15:19:26.442937+0000","last_became_peered":"2024-04-15T15:19:26.442937+0000","last_unstale":"2024-04-15T15:19:27.773986+0000","last_undegraded":"2024-04-15T15:19:27.773986+0000","last_fullsized":"2024-04-15T15:19:27.773986+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:56:54.270550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.439846+0000","last_change":"2024-04-15T15:19:25.419319+0000","last_active":"2024-04-15T15:19:27.439846+0000","last_peered":"2024-04-15T15:19:27.439846+0000","last_clean":"2024-04-15T15:19:27.439846+0000","last_became_active":"2024-04-15T15:19:25.419070+0000","last_became_peered":"2024-04-15T15:19:25.419070+0000","last_unstale":"2024-04-15T15:19:27.439846+0000","last_undegraded":"2024-04-15T15:19:27.439846+0000","last_fullsized":"2024-04-15T15:19:27.439846+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:21:02.027132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.421169+0000","last_change":"2024-04-15T15:19:26.428785+0000","last_active":"2024-04-15T15:19:27.421169+0000","last_peered":"2024-04-15T15:19:27.421169+0000","last_clean":"2024-04-15T15:19:27.421169+0000","last_became_active":"2024-04-15T15:19:26.428518+0000","last_became_peered":"2024-04-15T15:19:26.428518+0000","last_unstale":"2024-04-15T15:19:27.421169+0000","last_undegraded":"2024-04-15T15:19:27.421169+0000","last_fullsized":"2024-04-15T15:19:27.421169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:50:04.102480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.420737+0000","last_change":"2024-04-15T15:19:25.421107+0000","last_active":"2024-04-15T15:19:27.420737+0000","last_peered":"2024-04-15T15:19:27.420737+0000","last_clean":"2024-04-15T15:19:27.420737+0000","last_became_active":"2024-04-15T15:19:25.420203+0000","last_became_peered":"2024-04-15T15:19:25.420203+0000","last_unstale":"2024-04-15T15:19:27.420737+0000","last_undegraded":"2024-04-15T15:19:27.420737+0000","last_fullsized":"2024-04-15T15:19:27.420737+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:21:02.027132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.724752+0000","last_change":"2024-04-15T15:19:26.437300+0000","last_active":"2024-04-15T15:19:27.724752+0000","last_peered":"2024-04-15T15:19:27.724752+0000","last_clean":"2024-04-15T15:19:27.724752+0000","last_became_active":"2024-04-15T15:19:26.436626+0000","last_became_peered":"2024-04-15T15:19:26.436626+0000","last_unstale":"2024-04-15T15:19:27.724752+0000","last_undegraded":"2024-04-15T15:19:27.724752+0000","last_fullsized":"2024-04-15T15:19:27.724752+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:06:21.060519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774152+0000","last_change":"2024-04-15T15:19:25.423086+0000","last_active":"2024-04-15T15:19:27.774152+0000","last_peered":"2024-04-15T15:19:27.774152+0000","last_clean":"2024-04-15T15:19:27.774152+0000","last_became_active":"2024-04-15T15:19:25.422796+0000","last_became_peered":"2024-04-15T15:19:25.422796+0000","last_unstale":"2024-04-15T15:19:27.774152+0000","last_undegraded":"2024-04-15T15:19:27.774152+0000","last_fullsized":"2024-04-15T15:19:27.774152+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:20:16.625168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.773922+0000","last_change":"2024-04-15T15:19:26.433472+0000","last_active":"2024-04-15T15:19:27.773922+0000","last_peered":"2024-04-15T15:19:27.773922+0000","last_clean":"2024-04-15T15:19:27.773922+0000","last_became_active":"2024-04-15T15:19:26.432886+0000","last_became_peered":"2024-04-15T15:19:26.432886+0000","last_unstale":"2024-04-15T15:19:27.773922+0000","last_undegraded":"2024-04-15T15:19:27.773922+0000","last_fullsized":"2024-04-15T15:19:27.773922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:19:31.027967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774842+0000","last_change":"2024-04-15T15:19:25.419868+0000","last_active":"2024-04-15T15:19:27.774842+0000","last_peered":"2024-04-15T15:19:27.774842+0000","last_clean":"2024-04-15T15:19:27.774842+0000","last_became_active":"2024-04-15T15:19:25.419589+0000","last_became_peered":"2024-04-15T15:19:25.419589+0000","last_unstale":"2024-04-15T15:19:27.774842+0000","last_undegraded":"2024-04-15T15:19:27.774842+0000","last_fullsized":"2024-04-15T15:19:27.774842+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:08:25.359023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.724748+0000","last_change":"2024-04-15T15:19:26.446074+0000","last_active":"2024-04-15T15:19:27.724748+0000","last_peered":"2024-04-15T15:19:27.724748+0000","last_clean":"2024-04-15T15:19:27.724748+0000","last_became_active":"2024-04-15T15:19:26.445793+0000","last_became_peered":"2024-04-15T15:19:26.445793+0000","last_unstale":"2024-04-15T15:19:27.724748+0000","last_undegraded":"2024-04-15T15:19:27.724748+0000","last_fullsized":"2024-04-15T15:19:27.724748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:38:37.460808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.420477+0000","last_change":"2024-04-15T15:19:25.420817+0000","last_active":"2024-04-15T15:19:27.420477+0000","last_peered":"2024-04-15T15:19:27.420477+0000","last_clean":"2024-04-15T15:19:27.420477+0000","last_became_active":"2024-04-15T15:19:25.420546+0000","last_became_peered":"2024-04-15T15:19:25.420546+0000","last_unstale":"2024-04-15T15:19:27.420477+0000","last_undegraded":"2024-04-15T15:19:27.420477+0000","last_fullsized":"2024-04-15T15:19:27.420477+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:36:11.468459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.420351+0000","last_change":"2024-04-15T15:19:26.422633+0000","last_active":"2024-04-15T15:19:27.420351+0000","last_peered":"2024-04-15T15:19:27.420351+0000","last_clean":"2024-04-15T15:19:27.420351+0000","last_became_active":"2024-04-15T15:19:26.422315+0000","last_became_peered":"2024-04-15T15:19:26.422315+0000","last_unstale":"2024-04-15T15:19:27.420351+0000","last_undegraded":"2024-04-15T15:19:27.420351+0000","last_fullsized":"2024-04-15T15:19:27.420351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:56:54.270550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.554124+0000","last_change":"2024-04-15T15:19:25.414012+0000","last_active":"2024-04-15T15:19:27.554124+0000","last_peered":"2024-04-15T15:19:27.554124+0000","last_clean":"2024-04-15T15:19:27.554124+0000","last_became_active":"2024-04-15T15:19:25.413714+0000","last_became_peered":"2024-04-15T15:19:25.413714+0000","last_unstale":"2024-04-15T15:19:27.554124+0000","last_undegraded":"2024-04-15T15:19:27.554124+0000","last_fullsized":"2024-04-15T15:19:27.554124+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:39:21.336486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.724458+0000","last_change":"2024-04-15T15:19:26.442998+0000","last_active":"2024-04-15T15:19:27.724458+0000","last_peered":"2024-04-15T15:19:27.724458+0000","last_clean":"2024-04-15T15:19:27.724458+0000","last_became_active":"2024-04-15T15:19:26.442205+0000","last_became_peered":"2024-04-15T15:19:26.442205+0000","last_unstale":"2024-04-15T15:19:27.724458+0000","last_undegraded":"2024-04-15T15:19:27.724458+0000","last_fullsized":"2024-04-15T15:19:27.724458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:15:06.003211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.421043+0000","last_change":"2024-04-15T15:19:25.423057+0000","last_active":"2024-04-15T15:19:27.421043+0000","last_peered":"2024-04-15T15:19:27.421043+0000","last_clean":"2024-04-15T15:19:27.421043+0000","last_became_active":"2024-04-15T15:19:25.422760+0000","last_became_peered":"2024-04-15T15:19:25.422760+0000","last_unstale":"2024-04-15T15:19:27.421043+0000","last_undegraded":"2024-04-15T15:19:27.421043+0000","last_fullsized":"2024-04-15T15:19:27.421043+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:00:10.713962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.724934+0000","last_change":"2024-04-15T15:19:26.439301+0000","last_active":"2024-04-15T15:19:27.724934+0000","last_peered":"2024-04-15T15:19:27.724934+0000","last_clean":"2024-04-15T15:19:27.724934+0000","last_became_active":"2024-04-15T15:19:26.439016+0000","last_became_peered":"2024-04-15T15:19:26.439016+0000","last_unstale":"2024-04-15T15:19:27.724934+0000","last_undegraded":"2024-04-15T15:19:27.724934+0000","last_fullsized":"2024-04-15T15:19:27.724934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:47:07.377973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.554369+0000","last_change":"2024-04-15T15:19:25.416350+0000","last_active":"2024-04-15T15:19:27.554369+0000","last_peered":"2024-04-15T15:19:27.554369+0000","last_clean":"2024-04-15T15:19:27.554369+0000","last_became_active":"2024-04-15T15:19:25.415907+0000","last_became_peered":"2024-04-15T15:19:25.415907+0000","last_unstale":"2024-04-15T15:19:27.554369+0000","last_undegraded":"2024-04-15T15:19:27.554369+0000","last_fullsized":"2024-04-15T15:19:27.554369+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:55:33.675151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.725169+0000","last_change":"2024-04-15T15:19:26.445601+0000","last_active":"2024-04-15T15:19:27.725169+0000","last_peered":"2024-04-15T15:19:27.725169+0000","last_clean":"2024-04-15T15:19:27.725169+0000","last_became_active":"2024-04-15T15:19:26.445320+0000","last_became_peered":"2024-04-15T15:19:26.445320+0000","last_unstale":"2024-04-15T15:19:27.725169+0000","last_undegraded":"2024-04-15T15:19:27.725169+0000","last_fullsized":"2024-04-15T15:19:27.725169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:50:29.196579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.439918+0000","last_change":"2024-04-15T15:19:25.424681+0000","last_active":"2024-04-15T15:19:27.439918+0000","last_peered":"2024-04-15T15:19:27.439918+0000","last_clean":"2024-04-15T15:19:27.439918+0000","last_became_active":"2024-04-15T15:19:25.424420+0000","last_became_peered":"2024-04-15T15:19:25.424420+0000","last_unstale":"2024-04-15T15:19:27.439918+0000","last_undegraded":"2024-04-15T15:19:27.439918+0000","last_fullsized":"2024-04-15T15:19:27.439918+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:50:03.098538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774833+0000","last_change":"2024-04-15T15:19:26.441743+0000","last_active":"2024-04-15T15:19:27.774833+0000","last_peered":"2024-04-15T15:19:27.774833+0000","last_clean":"2024-04-15T15:19:27.774833+0000","last_became_active":"2024-04-15T15:19:26.441156+0000","last_became_peered":"2024-04-15T15:19:26.441156+0000","last_unstale":"2024-04-15T15:19:27.774833+0000","last_undegraded":"2024-04-15T15:19:27.774833+0000","last_fullsized":"2024-04-15T15:19:27.774833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:39:29.689447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.554294+0000","last_change":"2024-04-15T15:19:25.408454+0000","last_active":"2024-04-15T15:19:27.554294+0000","last_peered":"2024-04-15T15:19:27.554294+0000","last_clean":"2024-04-15T15:19:27.554294+0000","last_became_active":"2024-04-15T15:19:25.408188+0000","last_became_peered":"2024-04-15T15:19:25.408188+0000","last_unstale":"2024-04-15T15:19:27.554294+0000","last_undegraded":"2024-04-15T15:19:27.554294+0000","last_fullsized":"2024-04-15T15:19:27.554294+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:30:23.903641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.724769+0000","last_change":"2024-04-15T15:19:26.443981+0000","last_active":"2024-04-15T15:19:27.724769+0000","last_peered":"2024-04-15T15:19:27.724769+0000","last_clean":"2024-04-15T15:19:27.724769+0000","last_became_active":"2024-04-15T15:19:26.443177+0000","last_became_peered":"2024-04-15T15:19:26.443177+0000","last_unstale":"2024-04-15T15:19:27.724769+0000","last_undegraded":"2024-04-15T15:19:27.724769+0000","last_fullsized":"2024-04-15T15:19:27.724769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:32:49.335713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.724389+0000","last_change":"2024-04-15T15:19:14.878269+0000","last_active":"2024-04-15T15:19:27.724389+0000","last_peered":"2024-04-15T15:19:27.724389+0000","last_clean":"2024-04-15T15:19:27.724389+0000","last_became_active":"2024-04-15T15:19:14.877367+0000","last_became_peered":"2024-04-15T15:19:14.877367+0000","last_unstale":"2024-04-15T15:19:27.724389+0000","last_undegraded":"2024-04-15T15:19:27.724389+0000","last_fullsized":"2024-04-15T15:19:27.724389+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:13.827511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:13.827511+0000","last_clean_scrub_stamp":"2024-04-15T15:19:13.827511+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:32:55.902561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774286+0000","last_change":"2024-04-15T15:19:25.415380+0000","last_active":"2024-04-15T15:19:27.774286+0000","last_peered":"2024-04-15T15:19:27.774286+0000","last_clean":"2024-04-15T15:19:27.774286+0000","last_became_active":"2024-04-15T15:19:25.415098+0000","last_became_peered":"2024-04-15T15:19:25.415098+0000","last_unstale":"2024-04-15T15:19:27.774286+0000","last_undegraded":"2024-04-15T15:19:27.774286+0000","last_fullsized":"2024-04-15T15:19:27.774286+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:36:11.468459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.417927+0000","last_change":"2024-04-15T15:19:26.445122+0000","last_active":"2024-04-15T15:19:27.417927+0000","last_peered":"2024-04-15T15:19:27.417927+0000","last_clean":"2024-04-15T15:19:27.417927+0000","last_became_active":"2024-04-15T15:19:26.444748+0000","last_became_peered":"2024-04-15T15:19:26.444748+0000","last_unstale":"2024-04-15T15:19:27.417927+0000","last_undegraded":"2024-04-15T15:19:27.417927+0000","last_fullsized":"2024-04-15T15:19:27.417927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:55:46.969871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.724312+0000","last_change":"2024-04-15T15:19:25.422817+0000","last_active":"2024-04-15T15:19:27.724312+0000","last_peered":"2024-04-15T15:19:27.724312+0000","last_clean":"2024-04-15T15:19:27.724312+0000","last_became_active":"2024-04-15T15:19:25.422382+0000","last_became_peered":"2024-04-15T15:19:25.422382+0000","last_unstale":"2024-04-15T15:19:27.724312+0000","last_undegraded":"2024-04-15T15:19:27.724312+0000","last_fullsized":"2024-04-15T15:19:27.724312+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:21:02.027132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.773822+0000","last_change":"2024-04-15T15:19:26.431108+0000","last_active":"2024-04-15T15:19:27.773822+0000","last_peered":"2024-04-15T15:19:27.773822+0000","last_clean":"2024-04-15T15:19:27.773822+0000","last_became_active":"2024-04-15T15:19:26.430820+0000","last_became_peered":"2024-04-15T15:19:26.430820+0000","last_unstale":"2024-04-15T15:19:27.773822+0000","last_undegraded":"2024-04-15T15:19:27.773822+0000","last_fullsized":"2024-04-15T15:19:27.773822+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:23:17.976379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.554269+0000","last_change":"2024-04-15T15:19:25.416081+0000","last_active":"2024-04-15T15:19:27.554269+0000","last_peered":"2024-04-15T15:19:27.554269+0000","last_clean":"2024-04-15T15:19:27.554269+0000","last_became_active":"2024-04-15T15:19:25.415568+0000","last_became_peered":"2024-04-15T15:19:25.415568+0000","last_unstale":"2024-04-15T15:19:27.554269+0000","last_undegraded":"2024-04-15T15:19:27.554269+0000","last_fullsized":"2024-04-15T15:19:27.554269+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:36:11.468459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.439731+0000","last_change":"2024-04-15T15:19:26.429663+0000","last_active":"2024-04-15T15:19:27.439731+0000","last_peered":"2024-04-15T15:19:27.439731+0000","last_clean":"2024-04-15T15:19:27.439731+0000","last_became_active":"2024-04-15T15:19:26.429416+0000","last_became_peered":"2024-04-15T15:19:26.429416+0000","last_unstale":"2024-04-15T15:19:27.439731+0000","last_undegraded":"2024-04-15T15:19:27.439731+0000","last_fullsized":"2024-04-15T15:19:27.439731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:56:54.270550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774841+0000","last_change":"2024-04-15T15:19:25.414043+0000","last_active":"2024-04-15T15:19:27.774841+0000","last_peered":"2024-04-15T15:19:27.774841+0000","last_clean":"2024-04-15T15:19:27.774841+0000","last_became_active":"2024-04-15T15:19:25.413732+0000","last_became_peered":"2024-04-15T15:19:25.413732+0000","last_unstale":"2024-04-15T15:19:27.774841+0000","last_undegraded":"2024-04-15T15:19:27.774841+0000","last_fullsized":"2024-04-15T15:19:27.774841+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:21:02.027132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.420842+0000","last_change":"2024-04-15T15:19:26.432152+0000","last_active":"2024-04-15T15:19:27.420842+0000","last_peered":"2024-04-15T15:19:27.420842+0000","last_clean":"2024-04-15T15:19:27.420842+0000","last_became_active":"2024-04-15T15:19:26.431798+0000","last_became_peered":"2024-04-15T15:19:26.431798+0000","last_unstale":"2024-04-15T15:19:27.420842+0000","last_undegraded":"2024-04-15T15:19:27.420842+0000","last_fullsized":"2024-04-15T15:19:27.420842+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:21:03.031074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.439877+0000","last_change":"2024-04-15T15:19:25.420180+0000","last_active":"2024-04-15T15:19:27.439877+0000","last_peered":"2024-04-15T15:19:27.439877+0000","last_clean":"2024-04-15T15:19:27.439877+0000","last_became_active":"2024-04-15T15:19:25.419908+0000","last_became_peered":"2024-04-15T15:19:25.419908+0000","last_unstale":"2024-04-15T15:19:27.439877+0000","last_undegraded":"2024-04-15T15:19:27.439877+0000","last_fullsized":"2024-04-15T15:19:27.439877+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:36:11.468459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774793+0000","last_change":"2024-04-15T15:19:26.440109+0000","last_active":"2024-04-15T15:19:27.774793+0000","last_peered":"2024-04-15T15:19:27.774793+0000","last_clean":"2024-04-15T15:19:27.774793+0000","last_became_active":"2024-04-15T15:19:26.439411+0000","last_became_peered":"2024-04-15T15:19:26.439411+0000","last_unstale":"2024-04-15T15:19:27.774793+0000","last_undegraded":"2024-04-15T15:19:27.774793+0000","last_fullsized":"2024-04-15T15:19:27.774793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:32:49.335713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.725007+0000","last_change":"2024-04-15T15:19:25.422506+0000","last_active":"2024-04-15T15:19:27.725007+0000","last_peered":"2024-04-15T15:19:27.725007+0000","last_clean":"2024-04-15T15:19:27.725007+0000","last_became_active":"2024-04-15T15:19:25.422124+0000","last_became_peered":"2024-04-15T15:19:25.422124+0000","last_unstale":"2024-04-15T15:19:27.725007+0000","last_undegraded":"2024-04-15T15:19:27.725007+0000","last_fullsized":"2024-04-15T15:19:27.725007+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:52:02.003490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.439836+0000","last_change":"2024-04-15T15:19:26.429959+0000","last_active":"2024-04-15T15:19:27.439836+0000","last_peered":"2024-04-15T15:19:27.439836+0000","last_clean":"2024-04-15T15:19:27.439836+0000","last_became_active":"2024-04-15T15:19:26.429641+0000","last_became_peered":"2024-04-15T15:19:26.429641+0000","last_unstale":"2024-04-15T15:19:27.439836+0000","last_undegraded":"2024-04-15T15:19:27.439836+0000","last_fullsized":"2024-04-15T15:19:27.439836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:55:46.969871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.441601+0000","last_change":"2024-04-15T15:19:25.413501+0000","last_active":"2024-04-15T15:19:27.441601+0000","last_peered":"2024-04-15T15:19:27.441601+0000","last_clean":"2024-04-15T15:19:27.441601+0000","last_became_active":"2024-04-15T15:19:25.413246+0000","last_became_peered":"2024-04-15T15:19:25.413246+0000","last_unstale":"2024-04-15T15:19:27.441601+0000","last_undegraded":"2024-04-15T15:19:27.441601+0000","last_fullsized":"2024-04-15T15:19:27.441601+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:31:11.313499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.417952+0000","last_change":"2024-04-15T15:19:26.431760+0000","last_active":"2024-04-15T15:19:27.417952+0000","last_peered":"2024-04-15T15:19:27.417952+0000","last_clean":"2024-04-15T15:19:27.417952+0000","last_became_active":"2024-04-15T15:19:26.431488+0000","last_became_peered":"2024-04-15T15:19:26.431488+0000","last_unstale":"2024-04-15T15:19:27.417952+0000","last_undegraded":"2024-04-15T15:19:27.417952+0000","last_fullsized":"2024-04-15T15:19:27.417952+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:06:20.973059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.420725+0000","last_change":"2024-04-15T15:19:25.417327+0000","last_active":"2024-04-15T15:19:27.420725+0000","last_peered":"2024-04-15T15:19:27.420725+0000","last_clean":"2024-04-15T15:19:27.420725+0000","last_became_active":"2024-04-15T15:19:25.416474+0000","last_became_peered":"2024-04-15T15:19:25.416474+0000","last_unstale":"2024-04-15T15:19:27.420725+0000","last_undegraded":"2024-04-15T15:19:27.420725+0000","last_fullsized":"2024-04-15T15:19:27.420725+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:08:25.359023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774050+0000","last_change":"2024-04-15T15:19:26.427252+0000","last_active":"2024-04-15T15:19:27.774050+0000","last_peered":"2024-04-15T15:19:27.774050+0000","last_clean":"2024-04-15T15:19:27.774050+0000","last_became_active":"2024-04-15T15:19:26.427004+0000","last_became_peered":"2024-04-15T15:19:26.427004+0000","last_unstale":"2024-04-15T15:19:27.774050+0000","last_undegraded":"2024-04-15T15:19:27.774050+0000","last_fullsized":"2024-04-15T15:19:27.774050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:07:02.244933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.724316+0000","last_change":"2024-04-15T15:19:25.422257+0000","last_active":"2024-04-15T15:19:27.724316+0000","last_peered":"2024-04-15T15:19:27.724316+0000","last_clean":"2024-04-15T15:19:27.724316+0000","last_became_active":"2024-04-15T15:19:25.421856+0000","last_became_peered":"2024-04-15T15:19:25.421856+0000","last_unstale":"2024-04-15T15:19:27.724316+0000","last_undegraded":"2024-04-15T15:19:27.724316+0000","last_fullsized":"2024-04-15T15:19:27.724316+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:08:25.359023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.420621+0000","last_change":"2024-04-15T15:19:26.430222+0000","last_active":"2024-04-15T15:19:27.420621+0000","last_peered":"2024-04-15T15:19:27.420621+0000","last_clean":"2024-04-15T15:19:27.420621+0000","last_became_active":"2024-04-15T15:19:26.429923+0000","last_became_peered":"2024-04-15T15:19:26.429923+0000","last_unstale":"2024-04-15T15:19:27.420621+0000","last_undegraded":"2024-04-15T15:19:27.420621+0000","last_fullsized":"2024-04-15T15:19:27.420621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:08:26.362966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.439788+0000","last_change":"2024-04-15T15:19:25.418348+0000","last_active":"2024-04-15T15:19:27.439788+0000","last_peered":"2024-04-15T15:19:27.439788+0000","last_clean":"2024-04-15T15:19:27.439788+0000","last_became_active":"2024-04-15T15:19:25.418080+0000","last_became_peered":"2024-04-15T15:19:25.418080+0000","last_unstale":"2024-04-15T15:19:27.439788+0000","last_undegraded":"2024-04-15T15:19:27.439788+0000","last_fullsized":"2024-04-15T15:19:27.439788+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:55:33.675151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.417808+0000","last_change":"2024-04-15T15:19:26.428800+0000","last_active":"2024-04-15T15:19:27.417808+0000","last_peered":"2024-04-15T15:19:27.417808+0000","last_clean":"2024-04-15T15:19:27.417808+0000","last_became_active":"2024-04-15T15:19:26.428528+0000","last_became_peered":"2024-04-15T15:19:26.428528+0000","last_unstale":"2024-04-15T15:19:27.417808+0000","last_undegraded":"2024-04-15T15:19:27.417808+0000","last_fullsized":"2024-04-15T15:19:27.417808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:56:54.270550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.439924+0000","last_change":"2024-04-15T15:19:25.416097+0000","last_active":"2024-04-15T15:19:27.439924+0000","last_peered":"2024-04-15T15:19:27.439924+0000","last_clean":"2024-04-15T15:19:27.439924+0000","last_became_active":"2024-04-15T15:19:25.415550+0000","last_became_peered":"2024-04-15T15:19:25.415550+0000","last_unstale":"2024-04-15T15:19:27.439924+0000","last_undegraded":"2024-04-15T15:19:27.439924+0000","last_fullsized":"2024-04-15T15:19:27.439924+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:08:25.359023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.418104+0000","last_change":"2024-04-15T15:19:26.429710+0000","last_active":"2024-04-15T15:19:27.418104+0000","last_peered":"2024-04-15T15:19:27.418104+0000","last_clean":"2024-04-15T15:19:27.418104+0000","last_became_active":"2024-04-15T15:19:26.429429+0000","last_became_peered":"2024-04-15T15:19:26.429429+0000","last_unstale":"2024-04-15T15:19:27.418104+0000","last_undegraded":"2024-04-15T15:19:27.418104+0000","last_fullsized":"2024-04-15T15:19:27.418104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:52:38.985490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.724834+0000","last_change":"2024-04-15T15:19:25.425223+0000","last_active":"2024-04-15T15:19:27.724834+0000","last_peered":"2024-04-15T15:19:27.724834+0000","last_clean":"2024-04-15T15:19:27.724834+0000","last_became_active":"2024-04-15T15:19:25.424548+0000","last_became_peered":"2024-04-15T15:19:25.424548+0000","last_unstale":"2024-04-15T15:19:27.724834+0000","last_undegraded":"2024-04-15T15:19:27.724834+0000","last_fullsized":"2024-04-15T15:19:27.724834+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:50:03.098538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.420985+0000","last_change":"2024-04-15T15:19:26.430650+0000","last_active":"2024-04-15T15:19:27.420985+0000","last_peered":"2024-04-15T15:19:27.420985+0000","last_clean":"2024-04-15T15:19:27.420985+0000","last_became_active":"2024-04-15T15:19:26.430359+0000","last_became_peered":"2024-04-15T15:19:26.430359+0000","last_unstale":"2024-04-15T15:19:27.420985+0000","last_undegraded":"2024-04-15T15:19:27.420985+0000","last_fullsized":"2024-04-15T15:19:27.420985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:05:03.041116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774721+0000","last_change":"2024-04-15T15:19:26.445875+0000","last_active":"2024-04-15T15:19:27.774721+0000","last_peered":"2024-04-15T15:19:27.774721+0000","last_clean":"2024-04-15T15:19:27.774721+0000","last_became_active":"2024-04-15T15:19:26.444235+0000","last_became_peered":"2024-04-15T15:19:26.444235+0000","last_unstale":"2024-04-15T15:19:27.774721+0000","last_undegraded":"2024-04-15T15:19:27.774721+0000","last_fullsized":"2024-04-15T15:19:27.774721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:38:37.460808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.724639+0000","last_change":"2024-04-15T15:19:25.425067+0000","last_active":"2024-04-15T15:19:27.724639+0000","last_peered":"2024-04-15T15:19:27.724639+0000","last_clean":"2024-04-15T15:19:27.724639+0000","last_became_active":"2024-04-15T15:19:25.424761+0000","last_became_peered":"2024-04-15T15:19:25.424761+0000","last_unstale":"2024-04-15T15:19:27.724639+0000","last_undegraded":"2024-04-15T15:19:27.724639+0000","last_fullsized":"2024-04-15T15:19:27.724639+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:31:11.313499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774515+0000","last_change":"2024-04-15T15:19:26.444448+0000","last_active":"2024-04-15T15:19:27.774515+0000","last_peered":"2024-04-15T15:19:27.774515+0000","last_clean":"2024-04-15T15:19:27.774515+0000","last_became_active":"2024-04-15T15:19:26.444185+0000","last_became_peered":"2024-04-15T15:19:26.444185+0000","last_unstale":"2024-04-15T15:19:27.774515+0000","last_undegraded":"2024-04-15T15:19:27.774515+0000","last_fullsized":"2024-04-15T15:19:27.774515+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:52:38.985490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.724888+0000","last_change":"2024-04-15T15:19:25.427837+0000","last_active":"2024-04-15T15:19:27.724888+0000","last_peered":"2024-04-15T15:19:27.724888+0000","last_clean":"2024-04-15T15:19:27.724888+0000","last_became_active":"2024-04-15T15:19:25.427483+0000","last_became_peered":"2024-04-15T15:19:25.427483+0000","last_unstale":"2024-04-15T15:19:27.724888+0000","last_undegraded":"2024-04-15T15:19:27.724888+0000","last_fullsized":"2024-04-15T15:19:27.724888+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:21:55.463937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.773977+0000","last_change":"2024-04-15T15:19:26.443862+0000","last_active":"2024-04-15T15:19:27.773977+0000","last_peered":"2024-04-15T15:19:27.773977+0000","last_clean":"2024-04-15T15:19:27.773977+0000","last_became_active":"2024-04-15T15:19:26.442956+0000","last_became_peered":"2024-04-15T15:19:26.442956+0000","last_unstale":"2024-04-15T15:19:27.773977+0000","last_undegraded":"2024-04-15T15:19:27.773977+0000","last_fullsized":"2024-04-15T15:19:27.773977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:57:45.149303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.418121+0000","last_change":"2024-04-15T15:19:25.419914+0000","last_active":"2024-04-15T15:19:27.418121+0000","last_peered":"2024-04-15T15:19:27.418121+0000","last_clean":"2024-04-15T15:19:27.418121+0000","last_became_active":"2024-04-15T15:19:25.419638+0000","last_became_peered":"2024-04-15T15:19:25.419638+0000","last_unstale":"2024-04-15T15:19:27.418121+0000","last_undegraded":"2024-04-15T15:19:27.418121+0000","last_fullsized":"2024-04-15T15:19:27.418121+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:30:23.903641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.418082+0000","last_change":"2024-04-15T15:19:26.442167+0000","last_active":"2024-04-15T15:19:27.418082+0000","last_peered":"2024-04-15T15:19:27.418082+0000","last_clean":"2024-04-15T15:19:27.418082+0000","last_became_active":"2024-04-15T15:19:26.441866+0000","last_became_peered":"2024-04-15T15:19:26.441866+0000","last_unstale":"2024-04-15T15:19:27.418082+0000","last_undegraded":"2024-04-15T15:19:27.418082+0000","last_fullsized":"2024-04-15T15:19:27.418082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:35:32.148665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774091+0000","last_change":"2024-04-15T15:19:25.419672+0000","last_active":"2024-04-15T15:19:27.774091+0000","last_peered":"2024-04-15T15:19:27.774091+0000","last_clean":"2024-04-15T15:19:27.774091+0000","last_became_active":"2024-04-15T15:19:25.419412+0000","last_became_peered":"2024-04-15T15:19:25.419412+0000","last_unstale":"2024-04-15T15:19:27.774091+0000","last_undegraded":"2024-04-15T15:19:27.774091+0000","last_fullsized":"2024-04-15T15:19:27.774091+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:35:36.249861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-15T15:19:33.804 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-15T15:19:33.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.941+0000 7f19a9660640 1 Processor -- start 2024-04-15T15:19:33.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.941+0000 7f19a9660640 1 -- start start 2024-04-15T15:19:33.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.942+0000 7f19a9660640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19a4163130 0x7f19a40cf820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:33.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.942+0000 7f19a9660640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f19a40d0ef0 0x7f19a40d1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:33.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.942+0000 7f19a9660640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19a415edb0 con 0x7f19a415e9d0 2024-04-15T15:19:33.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.942+0000 7f19a9660640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f19a40d18c0 con 0x7f19a40d0ef0 2024-04-15T15:19:33.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.942+0000 7f19a9660640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f19a40cef90 con 0x7f19a4163130 2024-04-15T15:19:33.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.942+0000 7f19a2ffd640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f19a415e9d0 0x7f19a40d3f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41054/0 (socket says 172.21.15.105:41054) 2024-04-15T15:19:33.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.942+0000 7f19a2ffd640 1 -- 172.21.15.105:0/3594295469 learned_addr learned my addr 172.21.15.105:0/3594295469 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:19:33.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.943+0000 7f19a1ffb640 1 -- 172.21.15.105:0/3594295469 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1221567826 0 0) 0x7f1994001490 con 0x7f19a415e9d0 2024-04-15T15:19:33.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.943+0000 7f19a1ffb640 1 -- 172.21.15.105:0/3594295469 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1990003520 con 0x7f19a415e9d0 2024-04-15T15:19:33.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.943+0000 7f19a27fc640 1 --2- 172.21.15.105:0/3594295469 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19a4163130 0x7f19a40cf820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:33.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.943+0000 7f19a37fe640 1 --2- 172.21.15.105:0/3594295469 >> v2:172.21.15.154:3300/0 conn(0x7f19a40d0ef0 0x7f19a40d1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:33.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.943+0000 7f19a1ffb640 1 -- 172.21.15.105:0/3594295469 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2746845769 0 0) 0x7f1990003520 con 0x7f19a415e9d0 2024-04-15T15:19:33.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.943+0000 7f19a1ffb640 1 -- 172.21.15.105:0/3594295469 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19a4163130 msgr2=0x7f19a40cf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:33.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.943+0000 7f19a1ffb640 1 --2- 172.21.15.105:0/3594295469 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19a4163130 0x7f19a40cf820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.943+0000 7f19a1ffb640 1 -- 172.21.15.105:0/3594295469 >> v2:172.21.15.154:3300/0 conn(0x7f19a40d0ef0 msgr2=0x7f19a40d1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:33.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.943+0000 7f19a1ffb640 1 --2- 172.21.15.105:0/3594295469 >> v2:172.21.15.154:3300/0 conn(0x7f19a40d0ef0 0x7f19a40d1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.943+0000 7f19a1ffb640 1 -- 172.21.15.105:0/3594295469 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19a40ba280 con 0x7f19a415e9d0 2024-04-15T15:19:33.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.943+0000 7f19a27fc640 1 --2- 172.21.15.105:0/3594295469 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19a4163130 0x7f19a40cf820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:19:33.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.943+0000 7f19a1ffb640 1 -- 172.21.15.105:0/3594295469 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7f1994001bc0 con 0x7f19a415e9d0 2024-04-15T15:19:33.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.943+0000 7f19a1ffb640 1 -- 172.21.15.105:0/3594295469 >> v1:172.21.15.105:6789/0 conn(0x7f19a415e9d0 legacy=0x7f19a40d3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:33.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.943+0000 7f19a1ffb640 1 --2- 172.21.15.105:0/3594295469 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19900039b0 0x7f1990003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:33.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.943+0000 7f19a1ffb640 1 --2- 172.21.15.105:0/3594295469 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19a4163130 0x7f1990004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:33.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.943+0000 7f19a1ffb640 1 --2- 172.21.15.105:0/3594295469 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1990004d40 0x7f19900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:33.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.943+0000 7f19a1ffb640 1 -- 172.21.15.105:0/3594295469 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f19a40d18c0 con 0x7f19900039b0 2024-04-15T15:19:33.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.943+0000 7f19a1ffb640 1 -- 172.21.15.105:0/3594295469 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f19a40cef90 con 0x7f1990004d40 2024-04-15T15:19:33.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.943+0000 7f19a1ffb640 1 -- 172.21.15.105:0/3594295469 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f199000bac0 con 0x7f19a4163130 2024-04-15T15:19:33.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.943+0000 7f19a2ffd640 1 --2- 172.21.15.105:0/3594295469 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19900039b0 0x7f1990003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:33.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.943+0000 7f19a27fc640 1 --2- 172.21.15.105:0/3594295469 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19a4163130 0x7f1990004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:33.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.943+0000 7f19a37fe640 1 --2- 172.21.15.105:0/3594295469 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1990004d40 0x7f19900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:33.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.943+0000 7f19a2ffd640 1 -- 172.21.15.105:0/3594295469 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19a4163130 msgr2=0x7f1990004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:33.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.943+0000 7f19a2ffd640 1 --2- 172.21.15.105:0/3594295469 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19a4163130 0x7f1990004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.943+0000 7f19a2ffd640 1 -- 172.21.15.105:0/3594295469 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1990004d40 msgr2=0x7f19900051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:33.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.943+0000 7f19a2ffd640 1 --2- 172.21.15.105:0/3594295469 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1990004d40 0x7f19900051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.943+0000 7f19a2ffd640 1 -- 172.21.15.105:0/3594295469 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1990003520 con 0x7f19900039b0 2024-04-15T15:19:33.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.943+0000 7f19a37fe640 1 --2- 172.21.15.105:0/3594295469 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1990004d40 0x7f19900051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:19:33.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.944+0000 7f19a2ffd640 1 --2- 172.21.15.105:0/3594295469 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19900039b0 0x7f1990003dd0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f199400aa70 tx=0x7f1990005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6534af2db6e09bb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:33.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.944+0000 7f19a1ffb640 1 -- 172.21.15.105:0/3594295469 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1994038070 con 0x7f19900039b0 2024-04-15T15:19:33.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.944+0000 7f19a1ffb640 1 -- 172.21.15.105:0/3594295469 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19940072f0 con 0x7f19900039b0 2024-04-15T15:19:33.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.944+0000 7f19a1ffb640 1 -- 172.21.15.105:0/3594295469 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f199400a690 con 0x7f19900039b0 2024-04-15T15:19:33.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.945+0000 7f19a9660640 1 -- 172.21.15.105:0/3594295469 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19900039b0 msgr2=0x7f1990003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:33.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.945+0000 7f19a9660640 1 --2- 172.21.15.105:0/3594295469 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19900039b0 0x7f1990003dd0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f199400aa70 tx=0x7f1990005830 comp rx=0 tx=0).stop 2024-04-15T15:19:33.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.945+0000 7f19a37fe640 1 -- 172.21.15.105:0/3594295469 reap_dead start 2024-04-15T15:19:33.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.945+0000 7f19a9660640 1 -- 172.21.15.105:0/3594295469 shutdown_connections 2024-04-15T15:19:33.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.945+0000 7f19a9660640 1 -- 172.21.15.105:0/3594295469 >> 172.21.15.105:0/3594295469 conn(0x7f19a4067720 msgr2=0x7f19a4067b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:33.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.945+0000 7f19a9660640 1 -- 172.21.15.105:0/3594295469 shutdown_connections 2024-04-15T15:19:33.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.945+0000 7f19a9660640 1 -- 172.21.15.105:0/3594295469 wait complete. 2024-04-15T15:19:33.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.946+0000 7f19a9660640 1 Processor -- start 2024-04-15T15:19:33.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.946+0000 7f19a9660640 1 -- start start 2024-04-15T15:19:33.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.947+0000 7f19a9660640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19900039b0 0x7f19a40b6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:33.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.947+0000 7f19a9660640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19a414fba0 0x7f19a40ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:33.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.947+0000 7f19a9660640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19a40ace10 0x7f19a41f7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:33.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.947+0000 7f19a9660640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f19a40cf820 con 0x7f19a414fba0 2024-04-15T15:19:33.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.947+0000 7f19a9660640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f19a40cf250 con 0x7f19a40ace10 2024-04-15T15:19:33.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.947+0000 7f19a9660640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f19a40cf9a0 con 0x7f19900039b0 2024-04-15T15:19:33.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.947+0000 7f19a27fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19a414fba0 0x7f19a40ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:33.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.947+0000 7f19a27fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19a414fba0 0x7f19a40ac8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39854/0 (socket says 172.21.15.105:39854) 2024-04-15T15:19:33.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.947+0000 7f19a27fc640 1 -- 172.21.15.105:0/4022345299 learned_addr learned my addr 172.21.15.105:0/4022345299 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:33.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.947+0000 7f19a2ffd640 1 --2- 172.21.15.105:0/4022345299 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19900039b0 0x7f19a40b6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:33.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.947+0000 7f19a27fc640 1 -- 172.21.15.105:0/4022345299 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19900039b0 msgr2=0x7f19a40b6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:33.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.947+0000 7f19a27fc640 1 --2- 172.21.15.105:0/4022345299 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19900039b0 0x7f19a40b6bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.947+0000 7f19a27fc640 1 -- 172.21.15.105:0/4022345299 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19a40ace10 msgr2=0x7f19a41f7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:33.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.947+0000 7f19a27fc640 1 --2- 172.21.15.105:0/4022345299 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19a40ace10 0x7f19a41f7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.947+0000 7f19a27fc640 1 -- 172.21.15.105:0/4022345299 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19a40ab150 con 0x7f19a414fba0 2024-04-15T15:19:33.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.947+0000 7f19a27fc640 1 --2- 172.21.15.105:0/4022345299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19a414fba0 0x7f19a40ac8d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f198c0115f0 tx=0x7f198c011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:33.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.947+0000 7f199b7fe640 1 -- 172.21.15.105:0/4022345299 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198c019070 con 0x7f19a414fba0 2024-04-15T15:19:33.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.947+0000 7f199b7fe640 1 -- 172.21.15.105:0/4022345299 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f198c004220 con 0x7f19a414fba0 2024-04-15T15:19:33.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.947+0000 7f19a9660640 1 -- 172.21.15.105:0/4022345299 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19a40ab430 con 0x7f19a414fba0 2024-04-15T15:19:33.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.947+0000 7f199b7fe640 1 -- 172.21.15.105:0/4022345299 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198c00f040 con 0x7f19a414fba0 2024-04-15T15:19:33.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.947+0000 7f19a9660640 1 -- 172.21.15.105:0/4022345299 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19a40ab9f0 con 0x7f19a414fba0 2024-04-15T15:19:33.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.948+0000 7f19a9660640 1 -- 172.21.15.105:0/4022345299 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19a40ad230 con 0x7f19a414fba0 2024-04-15T15:19:33.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.949+0000 7f199b7fe640 1 -- 172.21.15.105:0/4022345299 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f198c015670 con 0x7f19a414fba0 2024-04-15T15:19:33.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.949+0000 7f199b7fe640 1 --2- 172.21.15.105:0/4022345299 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f198406c8f0 0x7f198406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:33.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.949+0000 7f199b7fe640 1 -- 172.21.15.105:0/4022345299 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f198c08bd90 con 0x7f19a414fba0 2024-04-15T15:19:33.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.949+0000 7f19a2ffd640 1 --2- 172.21.15.105:0/4022345299 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f198406c8f0 0x7f198406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:33.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.950+0000 7f19a2ffd640 1 --2- 172.21.15.105:0/4022345299 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f198406c8f0 0x7f198406edb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1994035000 tx=0x7f1994030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:33.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:33.954+0000 7f199b7fe640 1 -- 172.21.15.105:0/4022345299 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f198c05a1c0 con 0x7f19a414fba0 2024-04-15T15:19:34.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.126+0000 7f19a9660640 1 -- 172.21.15.105:0/4022345299 --> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f19a40d2540 con 0x7f198406c8f0 2024-04-15T15:19:34.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.132+0000 7f199b7fe640 1 -- 172.21.15.105:0/4022345299 <== mgr.4107 v2:172.21.15.105:6832/3506549322 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251836 (secure 0 0 0) 0x7f19a40d2540 con 0x7f198406c8f0 2024-04-15T15:19:34.136 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:19:34.137 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-04-15T15:19:34.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.134+0000 7f19a9660640 1 -- 172.21.15.105:0/4022345299 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f198406c8f0 msgr2=0x7f198406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:34.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.134+0000 7f19a9660640 1 --2- 172.21.15.105:0/4022345299 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f198406c8f0 0x7f198406edb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1994035000 tx=0x7f1994030000 comp rx=0 tx=0).stop 2024-04-15T15:19:34.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.134+0000 7f19a9660640 1 -- 172.21.15.105:0/4022345299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19a414fba0 msgr2=0x7f19a40ac8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:34.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.134+0000 7f19a9660640 1 --2- 172.21.15.105:0/4022345299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19a414fba0 0x7f19a40ac8d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f198c0115f0 tx=0x7f198c011ab0 comp rx=0 tx=0).stop 2024-04-15T15:19:34.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.134+0000 7f19a9660640 1 -- 172.21.15.105:0/4022345299 shutdown_connections 2024-04-15T15:19:34.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.134+0000 7f19a9660640 1 --2- 172.21.15.105:0/4022345299 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f198406c8f0 0x7f198406edb0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:34.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.134+0000 7f19a9660640 1 --2- 172.21.15.105:0/4022345299 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19a40ace10 0x7f19a41f7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:34.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.134+0000 7f19a9660640 1 --2- 172.21.15.105:0/4022345299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19a414fba0 0x7f19a40ac8d0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:34.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.134+0000 7f19a9660640 1 --2- 172.21.15.105:0/4022345299 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19900039b0 0x7f19a40b6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:34.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.134+0000 7f19a9660640 1 -- 172.21.15.105:0/4022345299 >> 172.21.15.105:0/4022345299 conn(0x7f19a4067720 msgr2=0x7f19a40b33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:34.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.134+0000 7f19a9660640 1 -- 172.21.15.105:0/4022345299 shutdown_connections 2024-04-15T15:19:34.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.135+0000 7f19a9660640 1 -- 172.21.15.105:0/4022345299 wait complete. 2024-04-15T15:19:34.149 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-04-15T15:19:33.735404+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219024,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755696,"statfs":{"total":773094113280,"available":772869832704,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001646"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.724996+0000","last_change":"2024-04-15T15:19:26.440158+0000","last_active":"2024-04-15T15:19:27.724996+0000","last_peered":"2024-04-15T15:19:27.724996+0000","last_clean":"2024-04-15T15:19:27.724996+0000","last_became_active":"2024-04-15T15:19:26.439860+0000","last_became_peered":"2024-04-15T15:19:26.439860+0000","last_unstale":"2024-04-15T15:19:27.724996+0000","last_undegraded":"2024-04-15T15:19:27.724996+0000","last_fullsized":"2024-04-15T15:19:27.724996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:39:29.689447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.440059+0000","last_change":"2024-04-15T15:19:26.426289+0000","last_active":"2024-04-15T15:19:27.440059+0000","last_peered":"2024-04-15T15:19:27.440059+0000","last_clean":"2024-04-15T15:19:27.440059+0000","last_became_active":"2024-04-15T15:19:26.425937+0000","last_became_peered":"2024-04-15T15:19:26.425937+0000","last_unstale":"2024-04-15T15:19:27.440059+0000","last_undegraded":"2024-04-15T15:19:27.440059+0000","last_fullsized":"2024-04-15T15:19:27.440059+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:06:20.973059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.773961+0000","last_change":"2024-04-15T15:19:26.425640+0000","last_active":"2024-04-15T15:19:27.773961+0000","last_peered":"2024-04-15T15:19:27.773961+0000","last_clean":"2024-04-15T15:19:27.773961+0000","last_became_active":"2024-04-15T15:19:26.425228+0000","last_became_peered":"2024-04-15T15:19:26.425228+0000","last_unstale":"2024-04-15T15:19:27.773961+0000","last_undegraded":"2024-04-15T15:19:27.773961+0000","last_fullsized":"2024-04-15T15:19:27.773961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:17:45.061394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774465+0000","last_change":"2024-04-15T15:19:26.439598+0000","last_active":"2024-04-15T15:19:27.774465+0000","last_peered":"2024-04-15T15:19:27.774465+0000","last_clean":"2024-04-15T15:19:27.774465+0000","last_became_active":"2024-04-15T15:19:26.439290+0000","last_became_peered":"2024-04-15T15:19:26.439290+0000","last_unstale":"2024-04-15T15:19:27.774465+0000","last_undegraded":"2024-04-15T15:19:27.774465+0000","last_fullsized":"2024-04-15T15:19:27.774465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:35:32.148665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.724710+0000","last_change":"2024-04-15T15:19:26.433863+0000","last_active":"2024-04-15T15:19:27.724710+0000","last_peered":"2024-04-15T15:19:27.724710+0000","last_clean":"2024-04-15T15:19:27.724710+0000","last_became_active":"2024-04-15T15:19:26.433559+0000","last_became_peered":"2024-04-15T15:19:26.433559+0000","last_unstale":"2024-04-15T15:19:27.724710+0000","last_undegraded":"2024-04-15T15:19:27.724710+0000","last_fullsized":"2024-04-15T15:19:27.724710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T03:07:46.810235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774361+0000","last_change":"2024-04-15T15:19:26.442278+0000","last_active":"2024-04-15T15:19:27.774361+0000","last_peered":"2024-04-15T15:19:27.774361+0000","last_clean":"2024-04-15T15:19:27.774361+0000","last_became_active":"2024-04-15T15:19:26.441910+0000","last_became_peered":"2024-04-15T15:19:26.441910+0000","last_unstale":"2024-04-15T15:19:27.774361+0000","last_undegraded":"2024-04-15T15:19:27.774361+0000","last_fullsized":"2024-04-15T15:19:27.774361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:05:03.041116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.724443+0000","last_change":"2024-04-15T15:19:26.430368+0000","last_active":"2024-04-15T15:19:27.724443+0000","last_peered":"2024-04-15T15:19:27.724443+0000","last_clean":"2024-04-15T15:19:27.724443+0000","last_became_active":"2024-04-15T15:19:26.430083+0000","last_became_peered":"2024-04-15T15:19:26.430083+0000","last_unstale":"2024-04-15T15:19:27.724443+0000","last_undegraded":"2024-04-15T15:19:27.724443+0000","last_fullsized":"2024-04-15T15:19:27.724443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:52:38.985490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.417632+0000","last_change":"2024-04-15T15:19:26.444027+0000","last_active":"2024-04-15T15:19:27.417632+0000","last_peered":"2024-04-15T15:19:27.417632+0000","last_clean":"2024-04-15T15:19:27.417632+0000","last_became_active":"2024-04-15T15:19:26.443699+0000","last_became_peered":"2024-04-15T15:19:26.443699+0000","last_unstale":"2024-04-15T15:19:27.417632+0000","last_undegraded":"2024-04-15T15:19:27.417632+0000","last_fullsized":"2024-04-15T15:19:27.417632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:19:31.027967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.725018+0000","last_change":"2024-04-15T15:19:26.443898+0000","last_active":"2024-04-15T15:19:27.725018+0000","last_peered":"2024-04-15T15:19:27.725018+0000","last_clean":"2024-04-15T15:19:27.725018+0000","last_became_active":"2024-04-15T15:19:26.443600+0000","last_became_peered":"2024-04-15T15:19:26.443600+0000","last_unstale":"2024-04-15T15:19:27.725018+0000","last_undegraded":"2024-04-15T15:19:27.725018+0000","last_fullsized":"2024-04-15T15:19:27.725018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:07:04.317951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.420914+0000","last_change":"2024-04-15T15:19:26.445705+0000","last_active":"2024-04-15T15:19:27.420914+0000","last_peered":"2024-04-15T15:19:27.420914+0000","last_clean":"2024-04-15T15:19:27.420914+0000","last_became_active":"2024-04-15T15:19:26.445421+0000","last_became_peered":"2024-04-15T15:19:26.445421+0000","last_unstale":"2024-04-15T15:19:27.420914+0000","last_undegraded":"2024-04-15T15:19:27.420914+0000","last_fullsized":"2024-04-15T15:19:27.420914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:06:20.973059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.420805+0000","last_change":"2024-04-15T15:19:26.441881+0000","last_active":"2024-04-15T15:19:27.420805+0000","last_peered":"2024-04-15T15:19:27.420805+0000","last_clean":"2024-04-15T15:19:27.420805+0000","last_became_active":"2024-04-15T15:19:26.441610+0000","last_became_peered":"2024-04-15T15:19:26.441610+0000","last_unstale":"2024-04-15T15:19:27.420805+0000","last_undegraded":"2024-04-15T15:19:27.420805+0000","last_fullsized":"2024-04-15T15:19:27.420805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:55:46.969871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.421116+0000","last_change":"2024-04-15T15:19:26.443171+0000","last_active":"2024-04-15T15:19:27.421116+0000","last_peered":"2024-04-15T15:19:27.421116+0000","last_clean":"2024-04-15T15:19:27.421116+0000","last_became_active":"2024-04-15T15:19:26.442896+0000","last_became_peered":"2024-04-15T15:19:26.442896+0000","last_unstale":"2024-04-15T15:19:27.421116+0000","last_undegraded":"2024-04-15T15:19:27.421116+0000","last_fullsized":"2024-04-15T15:19:27.421116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:31:12.317442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.417978+0000","last_change":"2024-04-15T15:19:26.442097+0000","last_active":"2024-04-15T15:19:27.417978+0000","last_peered":"2024-04-15T15:19:27.417978+0000","last_clean":"2024-04-15T15:19:27.417978+0000","last_became_active":"2024-04-15T15:19:26.441823+0000","last_became_peered":"2024-04-15T15:19:26.441823+0000","last_unstale":"2024-04-15T15:19:27.417978+0000","last_undegraded":"2024-04-15T15:19:27.417978+0000","last_fullsized":"2024-04-15T15:19:27.417978+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:05:03.041116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.773787+0000","last_change":"2024-04-15T15:19:26.428713+0000","last_active":"2024-04-15T15:19:27.773787+0000","last_peered":"2024-04-15T15:19:27.773787+0000","last_clean":"2024-04-15T15:19:27.773787+0000","last_became_active":"2024-04-15T15:19:26.428434+0000","last_became_peered":"2024-04-15T15:19:26.428434+0000","last_unstale":"2024-04-15T15:19:27.773787+0000","last_undegraded":"2024-04-15T15:19:27.773787+0000","last_fullsized":"2024-04-15T15:19:27.773787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:03:15.458694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.439725+0000","last_change":"2024-04-15T15:19:26.436898+0000","last_active":"2024-04-15T15:19:27.439725+0000","last_peered":"2024-04-15T15:19:27.439725+0000","last_clean":"2024-04-15T15:19:27.439725+0000","last_became_active":"2024-04-15T15:19:26.436643+0000","last_became_peered":"2024-04-15T15:19:26.436643+0000","last_unstale":"2024-04-15T15:19:27.439725+0000","last_undegraded":"2024-04-15T15:19:27.439725+0000","last_fullsized":"2024-04-15T15:19:27.439725+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:57:45.149303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774246+0000","last_change":"2024-04-15T15:19:26.439598+0000","last_active":"2024-04-15T15:19:27.774246+0000","last_peered":"2024-04-15T15:19:27.774246+0000","last_clean":"2024-04-15T15:19:27.774246+0000","last_became_active":"2024-04-15T15:19:26.439335+0000","last_became_peered":"2024-04-15T15:19:26.439335+0000","last_unstale":"2024-04-15T15:19:27.774246+0000","last_undegraded":"2024-04-15T15:19:27.774246+0000","last_fullsized":"2024-04-15T15:19:27.774246+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:55:46.969871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774820+0000","last_change":"2024-04-15T15:19:26.445594+0000","last_active":"2024-04-15T15:19:27.774820+0000","last_peered":"2024-04-15T15:19:27.774820+0000","last_clean":"2024-04-15T15:19:27.774820+0000","last_became_active":"2024-04-15T15:19:26.445306+0000","last_became_peered":"2024-04-15T15:19:26.445306+0000","last_unstale":"2024-04-15T15:19:27.774820+0000","last_undegraded":"2024-04-15T15:19:27.774820+0000","last_fullsized":"2024-04-15T15:19:27.774820+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:47:07.377973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.554342+0000","last_change":"2024-04-15T15:19:26.424093+0000","last_active":"2024-04-15T15:19:27.554342+0000","last_peered":"2024-04-15T15:19:27.554342+0000","last_clean":"2024-04-15T15:19:27.554342+0000","last_became_active":"2024-04-15T15:19:26.423816+0000","last_became_peered":"2024-04-15T15:19:26.423816+0000","last_unstale":"2024-04-15T15:19:27.554342+0000","last_undegraded":"2024-04-15T15:19:27.554342+0000","last_fullsized":"2024-04-15T15:19:27.554342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:52:03.007433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.725002+0000","last_change":"2024-04-15T15:19:26.437633+0000","last_active":"2024-04-15T15:19:27.725002+0000","last_peered":"2024-04-15T15:19:27.725002+0000","last_clean":"2024-04-15T15:19:27.725002+0000","last_became_active":"2024-04-15T15:19:26.437000+0000","last_became_peered":"2024-04-15T15:19:26.437000+0000","last_unstale":"2024-04-15T15:19:27.725002+0000","last_undegraded":"2024-04-15T15:19:27.725002+0000","last_fullsized":"2024-04-15T15:19:27.725002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:49:01.189045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.420621+0000","last_change":"2024-04-15T15:19:26.431490+0000","last_active":"2024-04-15T15:19:27.420621+0000","last_peered":"2024-04-15T15:19:27.420621+0000","last_clean":"2024-04-15T15:19:27.420621+0000","last_became_active":"2024-04-15T15:19:26.431235+0000","last_became_peered":"2024-04-15T15:19:26.431235+0000","last_unstale":"2024-04-15T15:19:27.420621+0000","last_undegraded":"2024-04-15T15:19:27.420621+0000","last_fullsized":"2024-04-15T15:19:27.420621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:07:02.244933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774078+0000","last_change":"2024-04-15T15:19:26.433079+0000","last_active":"2024-04-15T15:19:27.774078+0000","last_peered":"2024-04-15T15:19:27.774078+0000","last_clean":"2024-04-15T15:19:27.774078+0000","last_became_active":"2024-04-15T15:19:26.432075+0000","last_became_peered":"2024-04-15T15:19:26.432075+0000","last_unstale":"2024-04-15T15:19:27.774078+0000","last_undegraded":"2024-04-15T15:19:27.774078+0000","last_fullsized":"2024-04-15T15:19:27.774078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:55:46.969871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774692+0000","last_change":"2024-04-15T15:19:26.442575+0000","last_active":"2024-04-15T15:19:27.774692+0000","last_peered":"2024-04-15T15:19:27.774692+0000","last_clean":"2024-04-15T15:19:27.774692+0000","last_became_active":"2024-04-15T15:19:26.441316+0000","last_became_peered":"2024-04-15T15:19:26.441316+0000","last_unstale":"2024-04-15T15:19:27.774692+0000","last_undegraded":"2024-04-15T15:19:27.774692+0000","last_fullsized":"2024-04-15T15:19:27.774692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T03:07:46.810235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.417731+0000","last_change":"2024-04-15T15:19:26.437545+0000","last_active":"2024-04-15T15:19:27.417731+0000","last_peered":"2024-04-15T15:19:27.417731+0000","last_clean":"2024-04-15T15:19:27.417731+0000","last_became_active":"2024-04-15T15:19:26.437018+0000","last_became_peered":"2024-04-15T15:19:26.437018+0000","last_unstale":"2024-04-15T15:19:27.417731+0000","last_undegraded":"2024-04-15T15:19:27.417731+0000","last_fullsized":"2024-04-15T15:19:27.417731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:57:45.149303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774315+0000","last_change":"2024-04-15T15:19:26.438924+0000","last_active":"2024-04-15T15:19:27.774315+0000","last_peered":"2024-04-15T15:19:27.774315+0000","last_clean":"2024-04-15T15:19:27.774315+0000","last_became_active":"2024-04-15T15:19:26.438615+0000","last_became_peered":"2024-04-15T15:19:26.438615+0000","last_unstale":"2024-04-15T15:19:27.774315+0000","last_undegraded":"2024-04-15T15:19:27.774315+0000","last_fullsized":"2024-04-15T15:19:27.774315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:06:20.973059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.417837+0000","last_change":"2024-04-15T15:19:26.446195+0000","last_active":"2024-04-15T15:19:27.417837+0000","last_peered":"2024-04-15T15:19:27.417837+0000","last_clean":"2024-04-15T15:19:27.417837+0000","last_became_active":"2024-04-15T15:19:26.445872+0000","last_became_peered":"2024-04-15T15:19:26.445872+0000","last_unstale":"2024-04-15T15:19:27.417837+0000","last_undegraded":"2024-04-15T15:19:27.417837+0000","last_fullsized":"2024-04-15T15:19:27.417837+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:07:02.244933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.421026+0000","last_change":"2024-04-15T15:19:26.429073+0000","last_active":"2024-04-15T15:19:27.421026+0000","last_peered":"2024-04-15T15:19:27.421026+0000","last_clean":"2024-04-15T15:19:27.421026+0000","last_became_active":"2024-04-15T15:19:26.428755+0000","last_became_peered":"2024-04-15T15:19:26.428755+0000","last_unstale":"2024-04-15T15:19:27.421026+0000","last_undegraded":"2024-04-15T15:19:27.421026+0000","last_fullsized":"2024-04-15T15:19:27.421026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:35:32.148665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.439805+0000","last_change":"2024-04-15T15:19:26.432582+0000","last_active":"2024-04-15T15:19:27.439805+0000","last_peered":"2024-04-15T15:19:27.439805+0000","last_clean":"2024-04-15T15:19:27.439805+0000","last_became_active":"2024-04-15T15:19:26.432334+0000","last_became_peered":"2024-04-15T15:19:26.432334+0000","last_unstale":"2024-04-15T15:19:27.439805+0000","last_undegraded":"2024-04-15T15:19:27.439805+0000","last_fullsized":"2024-04-15T15:19:27.439805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:07:02.244933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.418110+0000","last_change":"2024-04-15T15:19:26.436504+0000","last_active":"2024-04-15T15:19:27.418110+0000","last_peered":"2024-04-15T15:19:27.418110+0000","last_clean":"2024-04-15T15:19:27.418110+0000","last_became_active":"2024-04-15T15:19:26.436237+0000","last_became_peered":"2024-04-15T15:19:26.436237+0000","last_unstale":"2024-04-15T15:19:27.418110+0000","last_undegraded":"2024-04-15T15:19:27.418110+0000","last_fullsized":"2024-04-15T15:19:27.418110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:15:06.003211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774078+0000","last_change":"2024-04-15T15:19:26.429813+0000","last_active":"2024-04-15T15:19:27.774078+0000","last_peered":"2024-04-15T15:19:27.774078+0000","last_clean":"2024-04-15T15:19:27.774078+0000","last_became_active":"2024-04-15T15:19:26.429507+0000","last_became_peered":"2024-04-15T15:19:26.429507+0000","last_unstale":"2024-04-15T15:19:27.774078+0000","last_undegraded":"2024-04-15T15:19:27.774078+0000","last_fullsized":"2024-04-15T15:19:27.774078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:06:20.973059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774710+0000","last_change":"2024-04-15T15:19:26.442277+0000","last_active":"2024-04-15T15:19:27.774710+0000","last_peered":"2024-04-15T15:19:27.774710+0000","last_clean":"2024-04-15T15:19:27.774710+0000","last_became_active":"2024-04-15T15:19:26.441630+0000","last_became_peered":"2024-04-15T15:19:26.441630+0000","last_unstale":"2024-04-15T15:19:27.774710+0000","last_undegraded":"2024-04-15T15:19:27.774710+0000","last_fullsized":"2024-04-15T15:19:27.774710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:06:21.060519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.554197+0000","last_change":"2024-04-15T15:19:26.439042+0000","last_active":"2024-04-15T15:19:27.554197+0000","last_peered":"2024-04-15T15:19:27.554197+0000","last_clean":"2024-04-15T15:19:27.554197+0000","last_became_active":"2024-04-15T15:19:26.438709+0000","last_became_peered":"2024-04-15T15:19:26.438709+0000","last_unstale":"2024-04-15T15:19:27.554197+0000","last_undegraded":"2024-04-15T15:19:27.554197+0000","last_fullsized":"2024-04-15T15:19:27.554197+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:21:56.467880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.420350+0000","last_change":"2024-04-15T15:19:26.423461+0000","last_active":"2024-04-15T15:19:27.420350+0000","last_peered":"2024-04-15T15:19:27.420350+0000","last_clean":"2024-04-15T15:19:27.420350+0000","last_became_active":"2024-04-15T15:19:26.423171+0000","last_became_peered":"2024-04-15T15:19:26.423171+0000","last_unstale":"2024-04-15T15:19:27.420350+0000","last_undegraded":"2024-04-15T15:19:27.420350+0000","last_fullsized":"2024-04-15T15:19:27.420350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:57:45.149303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.420526+0000","last_change":"2024-04-15T15:19:26.431616+0000","last_active":"2024-04-15T15:19:27.420526+0000","last_peered":"2024-04-15T15:19:27.420526+0000","last_clean":"2024-04-15T15:19:27.420526+0000","last_became_active":"2024-04-15T15:19:26.430999+0000","last_became_peered":"2024-04-15T15:19:26.430999+0000","last_unstale":"2024-04-15T15:19:27.420526+0000","last_undegraded":"2024-04-15T15:19:27.420526+0000","last_fullsized":"2024-04-15T15:19:27.420526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:55:34.679094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.418188+0000","last_change":"2024-04-15T15:19:25.418290+0000","last_active":"2024-04-15T15:19:27.418188+0000","last_peered":"2024-04-15T15:19:27.418188+0000","last_clean":"2024-04-15T15:19:27.418188+0000","last_became_active":"2024-04-15T15:19:25.418036+0000","last_became_peered":"2024-04-15T15:19:25.418036+0000","last_unstale":"2024-04-15T15:19:27.418188+0000","last_undegraded":"2024-04-15T15:19:27.418188+0000","last_fullsized":"2024-04-15T15:19:27.418188+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:35:36.249861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774176+0000","last_change":"2024-04-15T15:19:26.431800+0000","last_active":"2024-04-15T15:19:27.774176+0000","last_peered":"2024-04-15T15:19:27.774176+0000","last_clean":"2024-04-15T15:19:27.774176+0000","last_became_active":"2024-04-15T15:19:26.431535+0000","last_became_peered":"2024-04-15T15:19:26.431535+0000","last_unstale":"2024-04-15T15:19:27.774176+0000","last_undegraded":"2024-04-15T15:19:27.774176+0000","last_fullsized":"2024-04-15T15:19:27.774176+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:05:03.041116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.421045+0000","last_change":"2024-04-15T15:19:25.422350+0000","last_active":"2024-04-15T15:19:27.421045+0000","last_peered":"2024-04-15T15:19:27.421045+0000","last_clean":"2024-04-15T15:19:27.421045+0000","last_became_active":"2024-04-15T15:19:25.422091+0000","last_became_peered":"2024-04-15T15:19:25.422091+0000","last_unstale":"2024-04-15T15:19:27.421045+0000","last_undegraded":"2024-04-15T15:19:27.421045+0000","last_fullsized":"2024-04-15T15:19:27.421045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:55:33.675151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.417601+0000","last_change":"2024-04-15T15:19:25.427262+0000","last_active":"2024-04-15T15:19:27.417601+0000","last_peered":"2024-04-15T15:19:27.417601+0000","last_clean":"2024-04-15T15:19:27.417601+0000","last_became_active":"2024-04-15T15:19:25.427057+0000","last_became_peered":"2024-04-15T15:19:25.427057+0000","last_unstale":"2024-04-15T15:19:27.417601+0000","last_undegraded":"2024-04-15T15:19:27.417601+0000","last_fullsized":"2024-04-15T15:19:27.417601+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:39:21.336486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.417662+0000","last_change":"2024-04-15T15:19:26.434134+0000","last_active":"2024-04-15T15:19:27.417662+0000","last_peered":"2024-04-15T15:19:27.417662+0000","last_clean":"2024-04-15T15:19:27.417662+0000","last_became_active":"2024-04-15T15:19:26.433852+0000","last_became_peered":"2024-04-15T15:19:26.433852+0000","last_unstale":"2024-04-15T15:19:27.417662+0000","last_undegraded":"2024-04-15T15:19:27.417662+0000","last_fullsized":"2024-04-15T15:19:27.417662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:17:45.061394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774370+0000","last_change":"2024-04-15T15:19:25.423330+0000","last_active":"2024-04-15T15:19:27.774370+0000","last_peered":"2024-04-15T15:19:27.774370+0000","last_clean":"2024-04-15T15:19:27.774370+0000","last_became_active":"2024-04-15T15:19:25.423045+0000","last_became_peered":"2024-04-15T15:19:25.423045+0000","last_unstale":"2024-04-15T15:19:27.774370+0000","last_undegraded":"2024-04-15T15:19:27.774370+0000","last_fullsized":"2024-04-15T15:19:27.774370+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:55:33.675151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.439704+0000","last_change":"2024-04-15T15:19:26.432451+0000","last_active":"2024-04-15T15:19:27.439704+0000","last_peered":"2024-04-15T15:19:27.439704+0000","last_clean":"2024-04-15T15:19:27.439704+0000","last_became_active":"2024-04-15T15:19:26.432197+0000","last_became_peered":"2024-04-15T15:19:26.432197+0000","last_unstale":"2024-04-15T15:19:27.439704+0000","last_undegraded":"2024-04-15T15:19:27.439704+0000","last_fullsized":"2024-04-15T15:19:27.439704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:17:45.061394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.420612+0000","last_change":"2024-04-15T15:19:25.423680+0000","last_active":"2024-04-15T15:19:27.420612+0000","last_peered":"2024-04-15T15:19:27.420612+0000","last_clean":"2024-04-15T15:19:27.420612+0000","last_became_active":"2024-04-15T15:19:25.423361+0000","last_became_peered":"2024-04-15T15:19:25.423361+0000","last_unstale":"2024-04-15T15:19:27.420612+0000","last_undegraded":"2024-04-15T15:19:27.420612+0000","last_fullsized":"2024-04-15T15:19:27.420612+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:30:23.903641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774647+0000","last_change":"2024-04-15T15:19:26.446276+0000","last_active":"2024-04-15T15:19:27.774647+0000","last_peered":"2024-04-15T15:19:27.774647+0000","last_clean":"2024-04-15T15:19:27.774647+0000","last_became_active":"2024-04-15T15:19:26.445956+0000","last_became_peered":"2024-04-15T15:19:26.445956+0000","last_unstale":"2024-04-15T15:19:27.774647+0000","last_undegraded":"2024-04-15T15:19:27.774647+0000","last_fullsized":"2024-04-15T15:19:27.774647+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:15:06.003211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774626+0000","last_change":"2024-04-15T15:19:25.427557+0000","last_active":"2024-04-15T15:19:27.774626+0000","last_peered":"2024-04-15T15:19:27.774626+0000","last_clean":"2024-04-15T15:19:27.774626+0000","last_became_active":"2024-04-15T15:19:25.427264+0000","last_became_peered":"2024-04-15T15:19:25.427264+0000","last_unstale":"2024-04-15T15:19:27.774626+0000","last_undegraded":"2024-04-15T15:19:27.774626+0000","last_fullsized":"2024-04-15T15:19:27.774626+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:30:23.903641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.773986+0000","last_change":"2024-04-15T15:19:26.443195+0000","last_active":"2024-04-15T15:19:27.773986+0000","last_peered":"2024-04-15T15:19:27.773986+0000","last_clean":"2024-04-15T15:19:27.773986+0000","last_became_active":"2024-04-15T15:19:26.442937+0000","last_became_peered":"2024-04-15T15:19:26.442937+0000","last_unstale":"2024-04-15T15:19:27.773986+0000","last_undegraded":"2024-04-15T15:19:27.773986+0000","last_fullsized":"2024-04-15T15:19:27.773986+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:56:54.270550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.439846+0000","last_change":"2024-04-15T15:19:25.419319+0000","last_active":"2024-04-15T15:19:27.439846+0000","last_peered":"2024-04-15T15:19:27.439846+0000","last_clean":"2024-04-15T15:19:27.439846+0000","last_became_active":"2024-04-15T15:19:25.419070+0000","last_became_peered":"2024-04-15T15:19:25.419070+0000","last_unstale":"2024-04-15T15:19:27.439846+0000","last_undegraded":"2024-04-15T15:19:27.439846+0000","last_fullsized":"2024-04-15T15:19:27.439846+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:21:02.027132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.421169+0000","last_change":"2024-04-15T15:19:26.428785+0000","last_active":"2024-04-15T15:19:27.421169+0000","last_peered":"2024-04-15T15:19:27.421169+0000","last_clean":"2024-04-15T15:19:27.421169+0000","last_became_active":"2024-04-15T15:19:26.428518+0000","last_became_peered":"2024-04-15T15:19:26.428518+0000","last_unstale":"2024-04-15T15:19:27.421169+0000","last_undegraded":"2024-04-15T15:19:27.421169+0000","last_fullsized":"2024-04-15T15:19:27.421169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:50:04.102480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.420737+0000","last_change":"2024-04-15T15:19:25.421107+0000","last_active":"2024-04-15T15:19:27.420737+0000","last_peered":"2024-04-15T15:19:27.420737+0000","last_clean":"2024-04-15T15:19:27.420737+0000","last_became_active":"2024-04-15T15:19:25.420203+0000","last_became_peered":"2024-04-15T15:19:25.420203+0000","last_unstale":"2024-04-15T15:19:27.420737+0000","last_undegraded":"2024-04-15T15:19:27.420737+0000","last_fullsized":"2024-04-15T15:19:27.420737+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:21:02.027132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.724752+0000","last_change":"2024-04-15T15:19:26.437300+0000","last_active":"2024-04-15T15:19:27.724752+0000","last_peered":"2024-04-15T15:19:27.724752+0000","last_clean":"2024-04-15T15:19:27.724752+0000","last_became_active":"2024-04-15T15:19:26.436626+0000","last_became_peered":"2024-04-15T15:19:26.436626+0000","last_unstale":"2024-04-15T15:19:27.724752+0000","last_undegraded":"2024-04-15T15:19:27.724752+0000","last_fullsized":"2024-04-15T15:19:27.724752+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:06:21.060519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774152+0000","last_change":"2024-04-15T15:19:25.423086+0000","last_active":"2024-04-15T15:19:27.774152+0000","last_peered":"2024-04-15T15:19:27.774152+0000","last_clean":"2024-04-15T15:19:27.774152+0000","last_became_active":"2024-04-15T15:19:25.422796+0000","last_became_peered":"2024-04-15T15:19:25.422796+0000","last_unstale":"2024-04-15T15:19:27.774152+0000","last_undegraded":"2024-04-15T15:19:27.774152+0000","last_fullsized":"2024-04-15T15:19:27.774152+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:20:16.625168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.773922+0000","last_change":"2024-04-15T15:19:26.433472+0000","last_active":"2024-04-15T15:19:27.773922+0000","last_peered":"2024-04-15T15:19:27.773922+0000","last_clean":"2024-04-15T15:19:27.773922+0000","last_became_active":"2024-04-15T15:19:26.432886+0000","last_became_peered":"2024-04-15T15:19:26.432886+0000","last_unstale":"2024-04-15T15:19:27.773922+0000","last_undegraded":"2024-04-15T15:19:27.773922+0000","last_fullsized":"2024-04-15T15:19:27.773922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:19:31.027967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774842+0000","last_change":"2024-04-15T15:19:25.419868+0000","last_active":"2024-04-15T15:19:27.774842+0000","last_peered":"2024-04-15T15:19:27.774842+0000","last_clean":"2024-04-15T15:19:27.774842+0000","last_became_active":"2024-04-15T15:19:25.419589+0000","last_became_peered":"2024-04-15T15:19:25.419589+0000","last_unstale":"2024-04-15T15:19:27.774842+0000","last_undegraded":"2024-04-15T15:19:27.774842+0000","last_fullsized":"2024-04-15T15:19:27.774842+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:08:25.359023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.724748+0000","last_change":"2024-04-15T15:19:26.446074+0000","last_active":"2024-04-15T15:19:27.724748+0000","last_peered":"2024-04-15T15:19:27.724748+0000","last_clean":"2024-04-15T15:19:27.724748+0000","last_became_active":"2024-04-15T15:19:26.445793+0000","last_became_peered":"2024-04-15T15:19:26.445793+0000","last_unstale":"2024-04-15T15:19:27.724748+0000","last_undegraded":"2024-04-15T15:19:27.724748+0000","last_fullsized":"2024-04-15T15:19:27.724748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:38:37.460808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.420477+0000","last_change":"2024-04-15T15:19:25.420817+0000","last_active":"2024-04-15T15:19:27.420477+0000","last_peered":"2024-04-15T15:19:27.420477+0000","last_clean":"2024-04-15T15:19:27.420477+0000","last_became_active":"2024-04-15T15:19:25.420546+0000","last_became_peered":"2024-04-15T15:19:25.420546+0000","last_unstale":"2024-04-15T15:19:27.420477+0000","last_undegraded":"2024-04-15T15:19:27.420477+0000","last_fullsized":"2024-04-15T15:19:27.420477+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:36:11.468459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.420351+0000","last_change":"2024-04-15T15:19:26.422633+0000","last_active":"2024-04-15T15:19:27.420351+0000","last_peered":"2024-04-15T15:19:27.420351+0000","last_clean":"2024-04-15T15:19:27.420351+0000","last_became_active":"2024-04-15T15:19:26.422315+0000","last_became_peered":"2024-04-15T15:19:26.422315+0000","last_unstale":"2024-04-15T15:19:27.420351+0000","last_undegraded":"2024-04-15T15:19:27.420351+0000","last_fullsized":"2024-04-15T15:19:27.420351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:56:54.270550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.554124+0000","last_change":"2024-04-15T15:19:25.414012+0000","last_active":"2024-04-15T15:19:27.554124+0000","last_peered":"2024-04-15T15:19:27.554124+0000","last_clean":"2024-04-15T15:19:27.554124+0000","last_became_active":"2024-04-15T15:19:25.413714+0000","last_became_peered":"2024-04-15T15:19:25.413714+0000","last_unstale":"2024-04-15T15:19:27.554124+0000","last_undegraded":"2024-04-15T15:19:27.554124+0000","last_fullsized":"2024-04-15T15:19:27.554124+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:39:21.336486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.724458+0000","last_change":"2024-04-15T15:19:26.442998+0000","last_active":"2024-04-15T15:19:27.724458+0000","last_peered":"2024-04-15T15:19:27.724458+0000","last_clean":"2024-04-15T15:19:27.724458+0000","last_became_active":"2024-04-15T15:19:26.442205+0000","last_became_peered":"2024-04-15T15:19:26.442205+0000","last_unstale":"2024-04-15T15:19:27.724458+0000","last_undegraded":"2024-04-15T15:19:27.724458+0000","last_fullsized":"2024-04-15T15:19:27.724458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:15:06.003211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.421043+0000","last_change":"2024-04-15T15:19:25.423057+0000","last_active":"2024-04-15T15:19:27.421043+0000","last_peered":"2024-04-15T15:19:27.421043+0000","last_clean":"2024-04-15T15:19:27.421043+0000","last_became_active":"2024-04-15T15:19:25.422760+0000","last_became_peered":"2024-04-15T15:19:25.422760+0000","last_unstale":"2024-04-15T15:19:27.421043+0000","last_undegraded":"2024-04-15T15:19:27.421043+0000","last_fullsized":"2024-04-15T15:19:27.421043+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:00:10.713962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.724934+0000","last_change":"2024-04-15T15:19:26.439301+0000","last_active":"2024-04-15T15:19:27.724934+0000","last_peered":"2024-04-15T15:19:27.724934+0000","last_clean":"2024-04-15T15:19:27.724934+0000","last_became_active":"2024-04-15T15:19:26.439016+0000","last_became_peered":"2024-04-15T15:19:26.439016+0000","last_unstale":"2024-04-15T15:19:27.724934+0000","last_undegraded":"2024-04-15T15:19:27.724934+0000","last_fullsized":"2024-04-15T15:19:27.724934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:47:07.377973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.554369+0000","last_change":"2024-04-15T15:19:25.416350+0000","last_active":"2024-04-15T15:19:27.554369+0000","last_peered":"2024-04-15T15:19:27.554369+0000","last_clean":"2024-04-15T15:19:27.554369+0000","last_became_active":"2024-04-15T15:19:25.415907+0000","last_became_peered":"2024-04-15T15:19:25.415907+0000","last_unstale":"2024-04-15T15:19:27.554369+0000","last_undegraded":"2024-04-15T15:19:27.554369+0000","last_fullsized":"2024-04-15T15:19:27.554369+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:55:33.675151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.725169+0000","last_change":"2024-04-15T15:19:26.445601+0000","last_active":"2024-04-15T15:19:27.725169+0000","last_peered":"2024-04-15T15:19:27.725169+0000","last_clean":"2024-04-15T15:19:27.725169+0000","last_became_active":"2024-04-15T15:19:26.445320+0000","last_became_peered":"2024-04-15T15:19:26.445320+0000","last_unstale":"2024-04-15T15:19:27.725169+0000","last_undegraded":"2024-04-15T15:19:27.725169+0000","last_fullsized":"2024-04-15T15:19:27.725169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:50:29.196579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.439918+0000","last_change":"2024-04-15T15:19:25.424681+0000","last_active":"2024-04-15T15:19:27.439918+0000","last_peered":"2024-04-15T15:19:27.439918+0000","last_clean":"2024-04-15T15:19:27.439918+0000","last_became_active":"2024-04-15T15:19:25.424420+0000","last_became_peered":"2024-04-15T15:19:25.424420+0000","last_unstale":"2024-04-15T15:19:27.439918+0000","last_undegraded":"2024-04-15T15:19:27.439918+0000","last_fullsized":"2024-04-15T15:19:27.439918+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:50:03.098538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774833+0000","last_change":"2024-04-15T15:19:26.441743+0000","last_active":"2024-04-15T15:19:27.774833+0000","last_peered":"2024-04-15T15:19:27.774833+0000","last_clean":"2024-04-15T15:19:27.774833+0000","last_became_active":"2024-04-15T15:19:26.441156+0000","last_became_peered":"2024-04-15T15:19:26.441156+0000","last_unstale":"2024-04-15T15:19:27.774833+0000","last_undegraded":"2024-04-15T15:19:27.774833+0000","last_fullsized":"2024-04-15T15:19:27.774833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:39:29.689447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.554294+0000","last_change":"2024-04-15T15:19:25.408454+0000","last_active":"2024-04-15T15:19:27.554294+0000","last_peered":"2024-04-15T15:19:27.554294+0000","last_clean":"2024-04-15T15:19:27.554294+0000","last_became_active":"2024-04-15T15:19:25.408188+0000","last_became_peered":"2024-04-15T15:19:25.408188+0000","last_unstale":"2024-04-15T15:19:27.554294+0000","last_undegraded":"2024-04-15T15:19:27.554294+0000","last_fullsized":"2024-04-15T15:19:27.554294+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:30:23.903641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.724769+0000","last_change":"2024-04-15T15:19:26.443981+0000","last_active":"2024-04-15T15:19:27.724769+0000","last_peered":"2024-04-15T15:19:27.724769+0000","last_clean":"2024-04-15T15:19:27.724769+0000","last_became_active":"2024-04-15T15:19:26.443177+0000","last_became_peered":"2024-04-15T15:19:26.443177+0000","last_unstale":"2024-04-15T15:19:27.724769+0000","last_undegraded":"2024-04-15T15:19:27.724769+0000","last_fullsized":"2024-04-15T15:19:27.724769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:32:49.335713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.724389+0000","last_change":"2024-04-15T15:19:14.878269+0000","last_active":"2024-04-15T15:19:27.724389+0000","last_peered":"2024-04-15T15:19:27.724389+0000","last_clean":"2024-04-15T15:19:27.724389+0000","last_became_active":"2024-04-15T15:19:14.877367+0000","last_became_peered":"2024-04-15T15:19:14.877367+0000","last_unstale":"2024-04-15T15:19:27.724389+0000","last_undegraded":"2024-04-15T15:19:27.724389+0000","last_fullsized":"2024-04-15T15:19:27.724389+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:13.827511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:13.827511+0000","last_clean_scrub_stamp":"2024-04-15T15:19:13.827511+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:32:55.902561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774286+0000","last_change":"2024-04-15T15:19:25.415380+0000","last_active":"2024-04-15T15:19:27.774286+0000","last_peered":"2024-04-15T15:19:27.774286+0000","last_clean":"2024-04-15T15:19:27.774286+0000","last_became_active":"2024-04-15T15:19:25.415098+0000","last_became_peered":"2024-04-15T15:19:25.415098+0000","last_unstale":"2024-04-15T15:19:27.774286+0000","last_undegraded":"2024-04-15T15:19:27.774286+0000","last_fullsized":"2024-04-15T15:19:27.774286+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:36:11.468459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.417927+0000","last_change":"2024-04-15T15:19:26.445122+0000","last_active":"2024-04-15T15:19:27.417927+0000","last_peered":"2024-04-15T15:19:27.417927+0000","last_clean":"2024-04-15T15:19:27.417927+0000","last_became_active":"2024-04-15T15:19:26.444748+0000","last_became_peered":"2024-04-15T15:19:26.444748+0000","last_unstale":"2024-04-15T15:19:27.417927+0000","last_undegraded":"2024-04-15T15:19:27.417927+0000","last_fullsized":"2024-04-15T15:19:27.417927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:55:46.969871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.724312+0000","last_change":"2024-04-15T15:19:25.422817+0000","last_active":"2024-04-15T15:19:27.724312+0000","last_peered":"2024-04-15T15:19:27.724312+0000","last_clean":"2024-04-15T15:19:27.724312+0000","last_became_active":"2024-04-15T15:19:25.422382+0000","last_became_peered":"2024-04-15T15:19:25.422382+0000","last_unstale":"2024-04-15T15:19:27.724312+0000","last_undegraded":"2024-04-15T15:19:27.724312+0000","last_fullsized":"2024-04-15T15:19:27.724312+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:21:02.027132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.773822+0000","last_change":"2024-04-15T15:19:26.431108+0000","last_active":"2024-04-15T15:19:27.773822+0000","last_peered":"2024-04-15T15:19:27.773822+0000","last_clean":"2024-04-15T15:19:27.773822+0000","last_became_active":"2024-04-15T15:19:26.430820+0000","last_became_peered":"2024-04-15T15:19:26.430820+0000","last_unstale":"2024-04-15T15:19:27.773822+0000","last_undegraded":"2024-04-15T15:19:27.773822+0000","last_fullsized":"2024-04-15T15:19:27.773822+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:23:17.976379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.554269+0000","last_change":"2024-04-15T15:19:25.416081+0000","last_active":"2024-04-15T15:19:27.554269+0000","last_peered":"2024-04-15T15:19:27.554269+0000","last_clean":"2024-04-15T15:19:27.554269+0000","last_became_active":"2024-04-15T15:19:25.415568+0000","last_became_peered":"2024-04-15T15:19:25.415568+0000","last_unstale":"2024-04-15T15:19:27.554269+0000","last_undegraded":"2024-04-15T15:19:27.554269+0000","last_fullsized":"2024-04-15T15:19:27.554269+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:36:11.468459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.439731+0000","last_change":"2024-04-15T15:19:26.429663+0000","last_active":"2024-04-15T15:19:27.439731+0000","last_peered":"2024-04-15T15:19:27.439731+0000","last_clean":"2024-04-15T15:19:27.439731+0000","last_became_active":"2024-04-15T15:19:26.429416+0000","last_became_peered":"2024-04-15T15:19:26.429416+0000","last_unstale":"2024-04-15T15:19:27.439731+0000","last_undegraded":"2024-04-15T15:19:27.439731+0000","last_fullsized":"2024-04-15T15:19:27.439731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:56:54.270550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774841+0000","last_change":"2024-04-15T15:19:25.414043+0000","last_active":"2024-04-15T15:19:27.774841+0000","last_peered":"2024-04-15T15:19:27.774841+0000","last_clean":"2024-04-15T15:19:27.774841+0000","last_became_active":"2024-04-15T15:19:25.413732+0000","last_became_peered":"2024-04-15T15:19:25.413732+0000","last_unstale":"2024-04-15T15:19:27.774841+0000","last_undegraded":"2024-04-15T15:19:27.774841+0000","last_fullsized":"2024-04-15T15:19:27.774841+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:21:02.027132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.420842+0000","last_change":"2024-04-15T15:19:26.432152+0000","last_active":"2024-04-15T15:19:27.420842+0000","last_peered":"2024-04-15T15:19:27.420842+0000","last_clean":"2024-04-15T15:19:27.420842+0000","last_became_active":"2024-04-15T15:19:26.431798+0000","last_became_peered":"2024-04-15T15:19:26.431798+0000","last_unstale":"2024-04-15T15:19:27.420842+0000","last_undegraded":"2024-04-15T15:19:27.420842+0000","last_fullsized":"2024-04-15T15:19:27.420842+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:21:03.031074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.439877+0000","last_change":"2024-04-15T15:19:25.420180+0000","last_active":"2024-04-15T15:19:27.439877+0000","last_peered":"2024-04-15T15:19:27.439877+0000","last_clean":"2024-04-15T15:19:27.439877+0000","last_became_active":"2024-04-15T15:19:25.419908+0000","last_became_peered":"2024-04-15T15:19:25.419908+0000","last_unstale":"2024-04-15T15:19:27.439877+0000","last_undegraded":"2024-04-15T15:19:27.439877+0000","last_fullsized":"2024-04-15T15:19:27.439877+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:36:11.468459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774793+0000","last_change":"2024-04-15T15:19:26.440109+0000","last_active":"2024-04-15T15:19:27.774793+0000","last_peered":"2024-04-15T15:19:27.774793+0000","last_clean":"2024-04-15T15:19:27.774793+0000","last_became_active":"2024-04-15T15:19:26.439411+0000","last_became_peered":"2024-04-15T15:19:26.439411+0000","last_unstale":"2024-04-15T15:19:27.774793+0000","last_undegraded":"2024-04-15T15:19:27.774793+0000","last_fullsized":"2024-04-15T15:19:27.774793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:32:49.335713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.725007+0000","last_change":"2024-04-15T15:19:25.422506+0000","last_active":"2024-04-15T15:19:27.725007+0000","last_peered":"2024-04-15T15:19:27.725007+0000","last_clean":"2024-04-15T15:19:27.725007+0000","last_became_active":"2024-04-15T15:19:25.422124+0000","last_became_peered":"2024-04-15T15:19:25.422124+0000","last_unstale":"2024-04-15T15:19:27.725007+0000","last_undegraded":"2024-04-15T15:19:27.725007+0000","last_fullsized":"2024-04-15T15:19:27.725007+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:52:02.003490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.439836+0000","last_change":"2024-04-15T15:19:26.429959+0000","last_active":"2024-04-15T15:19:27.439836+0000","last_peered":"2024-04-15T15:19:27.439836+0000","last_clean":"2024-04-15T15:19:27.439836+0000","last_became_active":"2024-04-15T15:19:26.429641+0000","last_became_peered":"2024-04-15T15:19:26.429641+0000","last_unstale":"2024-04-15T15:19:27.439836+0000","last_undegraded":"2024-04-15T15:19:27.439836+0000","last_fullsized":"2024-04-15T15:19:27.439836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:55:46.969871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.441601+0000","last_change":"2024-04-15T15:19:25.413501+0000","last_active":"2024-04-15T15:19:27.441601+0000","last_peered":"2024-04-15T15:19:27.441601+0000","last_clean":"2024-04-15T15:19:27.441601+0000","last_became_active":"2024-04-15T15:19:25.413246+0000","last_became_peered":"2024-04-15T15:19:25.413246+0000","last_unstale":"2024-04-15T15:19:27.441601+0000","last_undegraded":"2024-04-15T15:19:27.441601+0000","last_fullsized":"2024-04-15T15:19:27.441601+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:31:11.313499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.417952+0000","last_change":"2024-04-15T15:19:26.431760+0000","last_active":"2024-04-15T15:19:27.417952+0000","last_peered":"2024-04-15T15:19:27.417952+0000","last_clean":"2024-04-15T15:19:27.417952+0000","last_became_active":"2024-04-15T15:19:26.431488+0000","last_became_peered":"2024-04-15T15:19:26.431488+0000","last_unstale":"2024-04-15T15:19:27.417952+0000","last_undegraded":"2024-04-15T15:19:27.417952+0000","last_fullsized":"2024-04-15T15:19:27.417952+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:06:20.973059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.420725+0000","last_change":"2024-04-15T15:19:25.417327+0000","last_active":"2024-04-15T15:19:27.420725+0000","last_peered":"2024-04-15T15:19:27.420725+0000","last_clean":"2024-04-15T15:19:27.420725+0000","last_became_active":"2024-04-15T15:19:25.416474+0000","last_became_peered":"2024-04-15T15:19:25.416474+0000","last_unstale":"2024-04-15T15:19:27.420725+0000","last_undegraded":"2024-04-15T15:19:27.420725+0000","last_fullsized":"2024-04-15T15:19:27.420725+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:08:25.359023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774050+0000","last_change":"2024-04-15T15:19:26.427252+0000","last_active":"2024-04-15T15:19:27.774050+0000","last_peered":"2024-04-15T15:19:27.774050+0000","last_clean":"2024-04-15T15:19:27.774050+0000","last_became_active":"2024-04-15T15:19:26.427004+0000","last_became_peered":"2024-04-15T15:19:26.427004+0000","last_unstale":"2024-04-15T15:19:27.774050+0000","last_undegraded":"2024-04-15T15:19:27.774050+0000","last_fullsized":"2024-04-15T15:19:27.774050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:07:02.244933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.724316+0000","last_change":"2024-04-15T15:19:25.422257+0000","last_active":"2024-04-15T15:19:27.724316+0000","last_peered":"2024-04-15T15:19:27.724316+0000","last_clean":"2024-04-15T15:19:27.724316+0000","last_became_active":"2024-04-15T15:19:25.421856+0000","last_became_peered":"2024-04-15T15:19:25.421856+0000","last_unstale":"2024-04-15T15:19:27.724316+0000","last_undegraded":"2024-04-15T15:19:27.724316+0000","last_fullsized":"2024-04-15T15:19:27.724316+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:08:25.359023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.420621+0000","last_change":"2024-04-15T15:19:26.430222+0000","last_active":"2024-04-15T15:19:27.420621+0000","last_peered":"2024-04-15T15:19:27.420621+0000","last_clean":"2024-04-15T15:19:27.420621+0000","last_became_active":"2024-04-15T15:19:26.429923+0000","last_became_peered":"2024-04-15T15:19:26.429923+0000","last_unstale":"2024-04-15T15:19:27.420621+0000","last_undegraded":"2024-04-15T15:19:27.420621+0000","last_fullsized":"2024-04-15T15:19:27.420621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:08:26.362966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.439788+0000","last_change":"2024-04-15T15:19:25.418348+0000","last_active":"2024-04-15T15:19:27.439788+0000","last_peered":"2024-04-15T15:19:27.439788+0000","last_clean":"2024-04-15T15:19:27.439788+0000","last_became_active":"2024-04-15T15:19:25.418080+0000","last_became_peered":"2024-04-15T15:19:25.418080+0000","last_unstale":"2024-04-15T15:19:27.439788+0000","last_undegraded":"2024-04-15T15:19:27.439788+0000","last_fullsized":"2024-04-15T15:19:27.439788+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:55:33.675151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.417808+0000","last_change":"2024-04-15T15:19:26.428800+0000","last_active":"2024-04-15T15:19:27.417808+0000","last_peered":"2024-04-15T15:19:27.417808+0000","last_clean":"2024-04-15T15:19:27.417808+0000","last_became_active":"2024-04-15T15:19:26.428528+0000","last_became_peered":"2024-04-15T15:19:26.428528+0000","last_unstale":"2024-04-15T15:19:27.417808+0000","last_undegraded":"2024-04-15T15:19:27.417808+0000","last_fullsized":"2024-04-15T15:19:27.417808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:56:54.270550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.439924+0000","last_change":"2024-04-15T15:19:25.416097+0000","last_active":"2024-04-15T15:19:27.439924+0000","last_peered":"2024-04-15T15:19:27.439924+0000","last_clean":"2024-04-15T15:19:27.439924+0000","last_became_active":"2024-04-15T15:19:25.415550+0000","last_became_peered":"2024-04-15T15:19:25.415550+0000","last_unstale":"2024-04-15T15:19:27.439924+0000","last_undegraded":"2024-04-15T15:19:27.439924+0000","last_fullsized":"2024-04-15T15:19:27.439924+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:08:25.359023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.418104+0000","last_change":"2024-04-15T15:19:26.429710+0000","last_active":"2024-04-15T15:19:27.418104+0000","last_peered":"2024-04-15T15:19:27.418104+0000","last_clean":"2024-04-15T15:19:27.418104+0000","last_became_active":"2024-04-15T15:19:26.429429+0000","last_became_peered":"2024-04-15T15:19:26.429429+0000","last_unstale":"2024-04-15T15:19:27.418104+0000","last_undegraded":"2024-04-15T15:19:27.418104+0000","last_fullsized":"2024-04-15T15:19:27.418104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:52:38.985490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.724834+0000","last_change":"2024-04-15T15:19:25.425223+0000","last_active":"2024-04-15T15:19:27.724834+0000","last_peered":"2024-04-15T15:19:27.724834+0000","last_clean":"2024-04-15T15:19:27.724834+0000","last_became_active":"2024-04-15T15:19:25.424548+0000","last_became_peered":"2024-04-15T15:19:25.424548+0000","last_unstale":"2024-04-15T15:19:27.724834+0000","last_undegraded":"2024-04-15T15:19:27.724834+0000","last_fullsized":"2024-04-15T15:19:27.724834+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:50:03.098538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.420985+0000","last_change":"2024-04-15T15:19:26.430650+0000","last_active":"2024-04-15T15:19:27.420985+0000","last_peered":"2024-04-15T15:19:27.420985+0000","last_clean":"2024-04-15T15:19:27.420985+0000","last_became_active":"2024-04-15T15:19:26.430359+0000","last_became_peered":"2024-04-15T15:19:26.430359+0000","last_unstale":"2024-04-15T15:19:27.420985+0000","last_undegraded":"2024-04-15T15:19:27.420985+0000","last_fullsized":"2024-04-15T15:19:27.420985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:05:03.041116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774721+0000","last_change":"2024-04-15T15:19:26.445875+0000","last_active":"2024-04-15T15:19:27.774721+0000","last_peered":"2024-04-15T15:19:27.774721+0000","last_clean":"2024-04-15T15:19:27.774721+0000","last_became_active":"2024-04-15T15:19:26.444235+0000","last_became_peered":"2024-04-15T15:19:26.444235+0000","last_unstale":"2024-04-15T15:19:27.774721+0000","last_undegraded":"2024-04-15T15:19:27.774721+0000","last_fullsized":"2024-04-15T15:19:27.774721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:38:37.460808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.724639+0000","last_change":"2024-04-15T15:19:25.425067+0000","last_active":"2024-04-15T15:19:27.724639+0000","last_peered":"2024-04-15T15:19:27.724639+0000","last_clean":"2024-04-15T15:19:27.724639+0000","last_became_active":"2024-04-15T15:19:25.424761+0000","last_became_peered":"2024-04-15T15:19:25.424761+0000","last_unstale":"2024-04-15T15:19:27.724639+0000","last_undegraded":"2024-04-15T15:19:27.724639+0000","last_fullsized":"2024-04-15T15:19:27.724639+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:31:11.313499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774515+0000","last_change":"2024-04-15T15:19:26.444448+0000","last_active":"2024-04-15T15:19:27.774515+0000","last_peered":"2024-04-15T15:19:27.774515+0000","last_clean":"2024-04-15T15:19:27.774515+0000","last_became_active":"2024-04-15T15:19:26.444185+0000","last_became_peered":"2024-04-15T15:19:26.444185+0000","last_unstale":"2024-04-15T15:19:27.774515+0000","last_undegraded":"2024-04-15T15:19:27.774515+0000","last_fullsized":"2024-04-15T15:19:27.774515+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:52:38.985490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.724888+0000","last_change":"2024-04-15T15:19:25.427837+0000","last_active":"2024-04-15T15:19:27.724888+0000","last_peered":"2024-04-15T15:19:27.724888+0000","last_clean":"2024-04-15T15:19:27.724888+0000","last_became_active":"2024-04-15T15:19:25.427483+0000","last_became_peered":"2024-04-15T15:19:25.427483+0000","last_unstale":"2024-04-15T15:19:27.724888+0000","last_undegraded":"2024-04-15T15:19:27.724888+0000","last_fullsized":"2024-04-15T15:19:27.724888+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:21:55.463937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.773977+0000","last_change":"2024-04-15T15:19:26.443862+0000","last_active":"2024-04-15T15:19:27.773977+0000","last_peered":"2024-04-15T15:19:27.773977+0000","last_clean":"2024-04-15T15:19:27.773977+0000","last_became_active":"2024-04-15T15:19:26.442956+0000","last_became_peered":"2024-04-15T15:19:26.442956+0000","last_unstale":"2024-04-15T15:19:27.773977+0000","last_undegraded":"2024-04-15T15:19:27.773977+0000","last_fullsized":"2024-04-15T15:19:27.773977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:57:45.149303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.418121+0000","last_change":"2024-04-15T15:19:25.419914+0000","last_active":"2024-04-15T15:19:27.418121+0000","last_peered":"2024-04-15T15:19:27.418121+0000","last_clean":"2024-04-15T15:19:27.418121+0000","last_became_active":"2024-04-15T15:19:25.419638+0000","last_became_peered":"2024-04-15T15:19:25.419638+0000","last_unstale":"2024-04-15T15:19:27.418121+0000","last_undegraded":"2024-04-15T15:19:27.418121+0000","last_fullsized":"2024-04-15T15:19:27.418121+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:30:23.903641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.418082+0000","last_change":"2024-04-15T15:19:26.442167+0000","last_active":"2024-04-15T15:19:27.418082+0000","last_peered":"2024-04-15T15:19:27.418082+0000","last_clean":"2024-04-15T15:19:27.418082+0000","last_became_active":"2024-04-15T15:19:26.441866+0000","last_became_peered":"2024-04-15T15:19:26.441866+0000","last_unstale":"2024-04-15T15:19:27.418082+0000","last_undegraded":"2024-04-15T15:19:27.418082+0000","last_fullsized":"2024-04-15T15:19:27.418082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:25.400451+0000","last_clean_scrub_stamp":"2024-04-15T15:19:25.400451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:35:32.148665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-15T15:19:27.774091+0000","last_change":"2024-04-15T15:19:25.419672+0000","last_active":"2024-04-15T15:19:27.774091+0000","last_peered":"2024-04-15T15:19:27.774091+0000","last_clean":"2024-04-15T15:19:27.774091+0000","last_became_active":"2024-04-15T15:19:25.419412+0000","last_became_peered":"2024-04-15T15:19:25.419412+0000","last_unstale":"2024-04-15T15:19:27.774091+0000","last_undegraded":"2024-04-15T15:19:27.774091+0000","last_fullsized":"2024-04-15T15:19:27.774091+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:24.396509+0000","last_clean_scrub_stamp":"2024-04-15T15:19:24.396509+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:35:36.249861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-15T15:19:34.152 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-15T15:19:34.152 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-15T15:19:34.152 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-15T15:19:34.152 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-15T15:19:34.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.288+0000 7f097fd50640 1 Processor -- start 2024-04-15T15:19:34.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.288+0000 7f097fd50640 1 -- start start 2024-04-15T15:19:34.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.289+0000 7f097fd50640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f097806c0d0 0x7f0978166570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:34.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.289+0000 7f097fd50640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09780b3c80 0x7f0978168ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:34.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.289+0000 7f097fd50640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09780c8270 con 0x7f09780d0540 2024-04-15T15:19:34.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.289+0000 7f097fd50640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f09780d0920 con 0x7f097806c0d0 2024-04-15T15:19:34.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.289+0000 7f097fd50640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f09780c7cb0 con 0x7f09780b3c80 2024-04-15T15:19:34.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.289+0000 7f097e2ed640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09780b3c80 0x7f0978168ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:34.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.289+0000 7f097d2eb640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f097806c0d0 0x7f0978166570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:34.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.289+0000 7f097daec640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f09780d0540 0x7f097806b9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.105:36896/0 (socket says 172.21.15.105:36896) 2024-04-15T15:19:34.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.289+0000 7f097e2ed640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09780b3c80 0x7f0978168ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.105:57244/0 (socket says 172.21.15.105:57244) 2024-04-15T15:19:34.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.289+0000 7f097daec640 1 -- 172.21.15.105:0/356749825 learned_addr learned my addr 172.21.15.105:0/356749825 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:19:34.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.290+0000 7f097e2ed640 1 -- 172.21.15.105:0/356749825 >> v2:172.21.15.154:3300/0 conn(0x7f097806c0d0 msgr2=0x7f0978166570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:34.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.290+0000 7f097e2ed640 1 --2- 172.21.15.105:0/356749825 >> v2:172.21.15.154:3300/0 conn(0x7f097806c0d0 0x7f0978166570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:34.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.290+0000 7f097e2ed640 1 -- 172.21.15.105:0/356749825 >> v1:172.21.15.154:6789/0 conn(0x7f09780d0540 legacy=0x7f097806b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:34.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.290+0000 7f097caea640 1 -- 172.21.15.105:0/356749825 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 946679906 0 0) 0x7f09780c8270 con 0x7f09780d0540 2024-04-15T15:19:34.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.290+0000 7f097e2ed640 1 -- 172.21.15.105:0/356749825 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09780c7f90 con 0x7f09780b3c80 2024-04-15T15:19:34.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.291+0000 7f097e2ed640 1 --2- 172.21.15.105:0/356749825 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09780b3c80 0x7f0978168ac0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f09700097a0 tx=0x7f097002b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5557e488a5957d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:34.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.291+0000 7f097caea640 1 -- 172.21.15.105:0/356749825 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0970035020 con 0x7f09780b3c80 2024-04-15T15:19:34.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.291+0000 7f097caea640 1 -- 172.21.15.105:0/356749825 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0970031830 con 0x7f09780b3c80 2024-04-15T15:19:34.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.291+0000 7f097caea640 1 -- 172.21.15.105:0/356749825 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0970031c00 con 0x7f09780b3c80 2024-04-15T15:19:34.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.291+0000 7f097fd50640 1 -- 172.21.15.105:0/356749825 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09780b3c80 msgr2=0x7f0978168ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:34.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.291+0000 7f097fd50640 1 --2- 172.21.15.105:0/356749825 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09780b3c80 0x7f0978168ac0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f09700097a0 tx=0x7f097002b8d0 comp rx=0 tx=0).stop 2024-04-15T15:19:34.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.291+0000 7f097fd50640 1 -- 172.21.15.105:0/356749825 shutdown_connections 2024-04-15T15:19:34.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.291+0000 7f097fd50640 1 --2- 172.21.15.105:0/356749825 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09780b3c80 0x7f0978168ac0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:34.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.291+0000 7f097fd50640 1 --2- 172.21.15.105:0/356749825 >> v2:172.21.15.154:3300/0 conn(0x7f097806c0d0 0x7f0978166570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:34.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.291+0000 7f097fd50640 1 -- 172.21.15.105:0/356749825 >> 172.21.15.105:0/356749825 conn(0x7f097809aa40 msgr2=0x7f09780b30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:34.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.291+0000 7f097fd50640 1 -- 172.21.15.105:0/356749825 shutdown_connections 2024-04-15T15:19:34.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.291+0000 7f097fd50640 1 -- 172.21.15.105:0/356749825 wait complete. 2024-04-15T15:19:34.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.292+0000 7f097fd50640 1 Processor -- start 2024-04-15T15:19:34.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.293+0000 7f097fd50640 1 -- start start 2024-04-15T15:19:34.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.293+0000 7f097fd50640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f097806c0d0 0x7f09781f4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:34.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.293+0000 7f097fd50640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f09780b3c80 0x7f097814c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:34.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.293+0000 7f097fd50640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09780d0540 0x7f09781f7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:34.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.293+0000 7f097fd50640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0978070e40 con 0x7f09780b3c80 2024-04-15T15:19:34.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.293+0000 7f097fd50640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f097806f180 con 0x7f097806c0d0 2024-04-15T15:19:34.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.293+0000 7f097fd50640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f0978070fc0 con 0x7f09780d0540 2024-04-15T15:19:34.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.293+0000 7f097d2eb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f09780b3c80 0x7f097814c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:34.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.293+0000 7f097d2eb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f09780b3c80 0x7f097814c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39864/0 (socket says 172.21.15.105:39864) 2024-04-15T15:19:34.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.293+0000 7f097d2eb640 1 -- 172.21.15.105:0/3836588501 learned_addr learned my addr 172.21.15.105:0/3836588501 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:34.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.294+0000 7f097e2ed640 1 --2- 172.21.15.105:0/3836588501 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09780d0540 0x7f09781f7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:34.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.294+0000 7f097daec640 1 --2- 172.21.15.105:0/3836588501 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f097806c0d0 0x7f09781f4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:34.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.294+0000 7f097d2eb640 1 -- 172.21.15.105:0/3836588501 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09780d0540 msgr2=0x7f09781f7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:34.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.294+0000 7f097d2eb640 1 --2- 172.21.15.105:0/3836588501 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09780d0540 0x7f09781f7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:34.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.294+0000 7f097d2eb640 1 -- 172.21.15.105:0/3836588501 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f097806c0d0 msgr2=0x7f09781f4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:34.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.294+0000 7f097d2eb640 1 --2- 172.21.15.105:0/3836588501 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f097806c0d0 0x7f09781f4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:34.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.294+0000 7f097d2eb640 1 -- 172.21.15.105:0/3836588501 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09781f8170 con 0x7f09780b3c80 2024-04-15T15:19:34.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.294+0000 7f097d2eb640 1 --2- 172.21.15.105:0/3836588501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f09780b3c80 0x7f097814c3e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f096800e870 tx=0x7f096800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:34.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.294+0000 7f096e7fc640 1 -- 172.21.15.105:0/3836588501 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0968017070 con 0x7f09780b3c80 2024-04-15T15:19:34.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.294+0000 7f096e7fc640 1 -- 172.21.15.105:0/3836588501 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f096800c940 con 0x7f09780b3c80 2024-04-15T15:19:34.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.294+0000 7f097fd50640 1 -- 172.21.15.105:0/3836588501 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09781f8450 con 0x7f09780b3c80 2024-04-15T15:19:34.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.294+0000 7f096e7fc640 1 -- 172.21.15.105:0/3836588501 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096800cd30 con 0x7f09780b3c80 2024-04-15T15:19:34.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.294+0000 7f097fd50640 1 -- 172.21.15.105:0/3836588501 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09781e7460 con 0x7f09780b3c80 2024-04-15T15:19:34.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.295+0000 7f097fd50640 1 -- 172.21.15.105:0/3836588501 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0978068a40 con 0x7f09780b3c80 2024-04-15T15:19:34.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.296+0000 7f096e7fc640 1 -- 172.21.15.105:0/3836588501 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f096801a070 con 0x7f09780b3c80 2024-04-15T15:19:34.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.296+0000 7f096e7fc640 1 --2- 172.21.15.105:0/3836588501 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f095c06c8f0 0x7f095c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:34.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.296+0000 7f096e7fc640 1 -- 172.21.15.105:0/3836588501 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f096808bd10 con 0x7f09780b3c80 2024-04-15T15:19:34.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.296+0000 7f097daec640 1 --2- 172.21.15.105:0/3836588501 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f095c06c8f0 0x7f095c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:34.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.296+0000 7f097daec640 1 --2- 172.21.15.105:0/3836588501 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f095c06c8f0 0x7f095c06edb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f09740012e0 tx=0x7f0974007ef0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:34.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.300+0000 7f096e7fc640 1 -- 172.21.15.105:0/3836588501 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f096805a1d0 con 0x7f09780b3c80 2024-04-15T15:19:34.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.517+0000 7f097fd50640 1 -- 172.21.15.105:0/3836588501 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f09780b39e0 con 0x7f09780b3c80 2024-04-15T15:19:34.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.518+0000 7f096e7fc640 1 -- 172.21.15.105:0/3836588501 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f0968059ad0 con 0x7f09780b3c80 2024-04-15T15:19:34.520 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:19:34.520 INFO:teuthology.orchestra.run.smithi105.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-15T15:19:34.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.520+0000 7f097fd50640 1 -- 172.21.15.105:0/3836588501 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f095c06c8f0 msgr2=0x7f095c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:34.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.520+0000 7f097fd50640 1 --2- 172.21.15.105:0/3836588501 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f095c06c8f0 0x7f095c06edb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f09740012e0 tx=0x7f0974007ef0 comp rx=0 tx=0).stop 2024-04-15T15:19:34.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.520+0000 7f097fd50640 1 -- 172.21.15.105:0/3836588501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f09780b3c80 msgr2=0x7f097814c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:34.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.520+0000 7f097fd50640 1 --2- 172.21.15.105:0/3836588501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f09780b3c80 0x7f097814c3e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f096800e870 tx=0x7f096800ed30 comp rx=0 tx=0).stop 2024-04-15T15:19:34.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.520+0000 7f097fd50640 1 -- 172.21.15.105:0/3836588501 shutdown_connections 2024-04-15T15:19:34.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.520+0000 7f097fd50640 1 --2- 172.21.15.105:0/3836588501 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09780d0540 0x7f09781f7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:34.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.520+0000 7f097fd50640 1 --2- 172.21.15.105:0/3836588501 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f095c06c8f0 0x7f095c06edb0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:34.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.520+0000 7f097fd50640 1 --2- 172.21.15.105:0/3836588501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f09780b3c80 0x7f097814c3e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:34.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.520+0000 7f097fd50640 1 --2- 172.21.15.105:0/3836588501 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f097806c0d0 0x7f09781f4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:34.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.520+0000 7f097fd50640 1 -- 172.21.15.105:0/3836588501 >> 172.21.15.105:0/3836588501 conn(0x7f097809aa40 msgr2=0x7f09780d1710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:34.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.520+0000 7f097fd50640 1 -- 172.21.15.105:0/3836588501 shutdown_connections 2024-04-15T15:19:34.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.520+0000 7f097fd50640 1 -- 172.21.15.105:0/3836588501 wait complete. 2024-04-15T15:19:34.532 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-15T15:19:34.533 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:19:34.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.633+0000 7fcda99a8640 1 Processor -- start 2024-04-15T15:19:34.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.633+0000 7fcda99a8640 1 -- start start 2024-04-15T15:19:34.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.634+0000 7fcda99a8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcda415ec60 0x7fcda40d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:34.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.634+0000 7fcda99a8640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fcda409b150 0x7fcda40cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:34.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.634+0000 7fcda99a8640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcda40714f0 con 0x7fcda409b780 2024-04-15T15:19:34.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.634+0000 7fcda99a8640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fcda406cad0 con 0x7fcda409b150 2024-04-15T15:19:34.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.634+0000 7fcda99a8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fcda40cd3b0 con 0x7fcda415ec60 2024-04-15T15:19:34.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.634+0000 7fcda37fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fcda409b150 0x7fcda40cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:34.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.634+0000 7fcda37fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fcda409b150 0x7fcda40cd840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39880/0 (socket says 172.21.15.105:39880) 2024-04-15T15:19:34.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.634+0000 7fcda37fe640 1 -- 172.21.15.105:0/1601499643 learned_addr learned my addr 172.21.15.105:0/1601499643 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:34.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.634+0000 7fcda2ffd640 1 --2- 172.21.15.105:0/1601499643 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcda415ec60 0x7fcda40d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:34.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.634+0000 7fcda37fe640 1 -- 172.21.15.105:0/1601499643 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcda415ec60 msgr2=0x7fcda40d1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:34.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.634+0000 7fcda37fe640 1 --2- 172.21.15.105:0/1601499643 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcda415ec60 0x7fcda40d1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:34.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.634+0000 7fcda37fe640 1 -- 172.21.15.105:0/1601499643 >> v1:172.21.15.154:6789/0 conn(0x7fcda409b780 legacy=0x7fcda409aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:34.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.634+0000 7fcda37fe640 1 -- 172.21.15.105:0/1601499643 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcda40d1940 con 0x7fcda409b150 2024-04-15T15:19:34.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.634+0000 7fcda2ffd640 1 --2- 172.21.15.105:0/1601499643 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcda415ec60 0x7fcda40d1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:34.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.635+0000 7fcda37fe640 1 --2- 172.21.15.105:0/1601499643 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcda409b150 0x7fcda40cd840 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c009e50 tx=0x7fcd9c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dded7228a822df2 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:34.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.635+0000 7fcda1ffb640 1 -- 172.21.15.105:0/1601499643 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd9c02ca70 con 0x7fcda409b150 2024-04-15T15:19:34.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.635+0000 7fcda1ffb640 1 -- 172.21.15.105:0/1601499643 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd9c02cc10 con 0x7fcda409b150 2024-04-15T15:19:34.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.635+0000 7fcda1ffb640 1 -- 172.21.15.105:0/1601499643 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd9c0343f0 con 0x7fcda409b150 2024-04-15T15:19:34.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.636+0000 7fcda99a8640 1 -- 172.21.15.105:0/1601499643 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcda409b150 msgr2=0x7fcda40cd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:34.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.636+0000 7fcda99a8640 1 --2- 172.21.15.105:0/1601499643 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcda409b150 0x7fcda40cd840 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c009e50 tx=0x7fcd9c02bed0 comp rx=0 tx=0).stop 2024-04-15T15:19:34.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.636+0000 7fcda99a8640 1 -- 172.21.15.105:0/1601499643 shutdown_connections 2024-04-15T15:19:34.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.636+0000 7fcda99a8640 1 --2- 172.21.15.105:0/1601499643 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcda409b150 0x7fcda40cd840 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:34.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.636+0000 7fcda99a8640 1 --2- 172.21.15.105:0/1601499643 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcda415ec60 0x7fcda40d1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:34.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.636+0000 7fcda99a8640 1 -- 172.21.15.105:0/1601499643 >> 172.21.15.105:0/1601499643 conn(0x7fcda40d2570 msgr2=0x7fcda40d2980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:34.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.636+0000 7fcda99a8640 1 -- 172.21.15.105:0/1601499643 shutdown_connections 2024-04-15T15:19:34.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.636+0000 7fcda99a8640 1 -- 172.21.15.105:0/1601499643 wait complete. 2024-04-15T15:19:34.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.637+0000 7fcda99a8640 1 Processor -- start 2024-04-15T15:19:34.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.637+0000 7fcda99a8640 1 -- start start 2024-04-15T15:19:34.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.638+0000 7fcda99a8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcda409b150 0x7fcda415d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:34.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.638+0000 7fcda99a8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcda409b780 0x7fcda40b6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:34.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.638+0000 7fcda99a8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcda409d460 0x7fcda40b6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:34.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.638+0000 7fcda99a8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcda40cdd80 con 0x7fcda409b780 2024-04-15T15:19:34.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.638+0000 7fcda99a8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fcda40cd040 con 0x7fcda409d460 2024-04-15T15:19:34.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.638+0000 7fcda99a8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fcda40cdf00 con 0x7fcda409b150 2024-04-15T15:19:34.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.638+0000 7fcda27fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcda409b780 0x7fcda40b6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:34.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.638+0000 7fcda27fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcda409b780 0x7fcda40b6850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39882/0 (socket says 172.21.15.105:39882) 2024-04-15T15:19:34.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.638+0000 7fcda27fc640 1 -- 172.21.15.105:0/4222423329 learned_addr learned my addr 172.21.15.105:0/4222423329 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:34.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.638+0000 7fcda2ffd640 1 --2- 172.21.15.105:0/4222423329 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcda409b150 0x7fcda415d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:34.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.638+0000 7fcda37fe640 1 --2- 172.21.15.105:0/4222423329 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcda409d460 0x7fcda40b6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:34.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.638+0000 7fcda27fc640 1 -- 172.21.15.105:0/4222423329 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcda409b150 msgr2=0x7fcda415d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:34.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.638+0000 7fcda27fc640 1 --2- 172.21.15.105:0/4222423329 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcda409b150 0x7fcda415d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:34.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.638+0000 7fcda27fc640 1 -- 172.21.15.105:0/4222423329 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcda409d460 msgr2=0x7fcda40b6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:34.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.638+0000 7fcda27fc640 1 --2- 172.21.15.105:0/4222423329 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcda409d460 0x7fcda40b6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:34.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.638+0000 7fcda27fc640 1 -- 172.21.15.105:0/4222423329 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcda415dc20 con 0x7fcda409b780 2024-04-15T15:19:34.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.638+0000 7fcda37fe640 1 --2- 172.21.15.105:0/4222423329 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcda409d460 0x7fcda40b6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:34.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.638+0000 7fcda27fc640 1 --2- 172.21.15.105:0/4222423329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcda409b780 0x7fcda40b6850 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcd9800c430 tx=0x7fcd9800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:34.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.639+0000 7fcd8bfff640 1 -- 172.21.15.105:0/4222423329 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd98011790 con 0x7fcda409b780 2024-04-15T15:19:34.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.639+0000 7fcd8bfff640 1 -- 172.21.15.105:0/4222423329 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd98017070 con 0x7fcda409b780 2024-04-15T15:19:34.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.639+0000 7fcda99a8640 1 -- 172.21.15.105:0/4222423329 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcda40a0190 con 0x7fcda409b780 2024-04-15T15:19:34.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.639+0000 7fcd8bfff640 1 -- 172.21.15.105:0/4222423329 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd98011bb0 con 0x7fcda409b780 2024-04-15T15:19:34.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.639+0000 7fcda99a8640 1 -- 172.21.15.105:0/4222423329 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcda40cae10 con 0x7fcda409b780 2024-04-15T15:19:34.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.640+0000 7fcda99a8640 1 -- 172.21.15.105:0/4222423329 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcda4068a40 con 0x7fcda409b780 2024-04-15T15:19:34.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.640+0000 7fcd8bfff640 1 -- 172.21.15.105:0/4222423329 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcd9801c460 con 0x7fcda409b780 2024-04-15T15:19:34.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.640+0000 7fcd8bfff640 1 --2- 172.21.15.105:0/4222423329 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fcd9006c8f0 0x7fcd9006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:34.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.641+0000 7fcd8bfff640 1 -- 172.21.15.105:0/4222423329 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fcd9808b960 con 0x7fcda409b780 2024-04-15T15:19:34.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.641+0000 7fcda2ffd640 1 --2- 172.21.15.105:0/4222423329 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fcd9006c8f0 0x7fcd9006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:34.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.641+0000 7fcda2ffd640 1 --2- 172.21.15.105:0/4222423329 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fcd9006c8f0 0x7fcd9006edb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fcda409c6b0 tx=0x7fcd94008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:34.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.645+0000 7fcd8bfff640 1 -- 172.21.15.105:0/4222423329 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcd98059e20 con 0x7fcda409b780 2024-04-15T15:19:34.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.823+0000 7fcda99a8640 1 -- 172.21.15.105:0/4222423329 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcda40b1a00 con 0x7fcda409b780 2024-04-15T15:19:34.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.825+0000 7fcd8bfff640 1 -- 172.21.15.105:0/4222423329 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16653 (secure 0 0 0) 0x7fcd98059720 con 0x7fcda409b780 2024-04-15T15:19:34.827 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:19:34.827 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":20,"fsid":"09e4d2a4-a324-4793-af77-644b25e520e1","created":"2024-04-15T15:19:01.524543+0000","modified":"2024-04-15T15:19:27.414890+0000","last_up_change":"2024-04-15T15:19:12.825815+0000","last_in_change":"2024-04-15T15:19:03.788126+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:19:13.748585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:19:23.789287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:19:24.722250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6f6292b-2d7e-4aa0-9d1c-6ddd0f97287c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6801","nonce":2873040054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6803","nonce":2873040054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6807","nonce":2873040054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6805","nonce":2873040054}]},"public_addr":"172.21.15.105:6801/2873040054","cluster_addr":"172.21.15.105:6803/2873040054","heartbeat_back_addr":"172.21.15.105:6807/2873040054","heartbeat_front_addr":"172.21.15.105:6805/2873040054","state":["exists","up"]},{"osd":1,"uuid":"8900966a-b393-41e3-8f93-e5a43bd23622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6818","nonce":2494994479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6821","nonce":2494994479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6828","nonce":2494994479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6823","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6825","nonce":2494994479}]},"public_addr":"172.21.15.105:6818/2494994479","cluster_addr":"172.21.15.105:6821/2494994479","heartbeat_back_addr":"172.21.15.105:6828/2494994479","heartbeat_front_addr":"172.21.15.105:6825/2494994479","state":["exists","up"]},{"osd":2,"uuid":"81a5dc7a-9325-4ff5-941d-a2d5fba21f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6809","nonce":3798408699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6811","nonce":3798408699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6815","nonce":3798408699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6813","nonce":3798408699}]},"public_addr":"172.21.15.105:6809/3798408699","cluster_addr":"172.21.15.105:6811/3798408699","heartbeat_back_addr":"172.21.15.105:6815/3798408699","heartbeat_front_addr":"172.21.15.105:6813/3798408699","state":["exists","up"]},{"osd":3,"uuid":"df766ed2-5c71-4805-96f4-3d4970edc960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6819","nonce":1920720186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6824","nonce":1920720186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6831","nonce":1920720186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6827","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6829","nonce":1920720186}]},"public_addr":"172.21.15.105:6819/1920720186","cluster_addr":"172.21.15.105:6824/1920720186","heartbeat_back_addr":"172.21.15.105:6831/1920720186","heartbeat_front_addr":"172.21.15.105:6829/1920720186","state":["exists","up"]},{"osd":4,"uuid":"e1039e03-28b8-49df-b951-c91930b93c9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6810","nonce":3516494456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6813","nonce":3516494456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6820","nonce":3516494456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6817","nonce":3516494456}]},"public_addr":"172.21.15.154:6810/3516494456","cluster_addr":"172.21.15.154:6813/3516494456","heartbeat_back_addr":"172.21.15.154:6820/3516494456","heartbeat_front_addr":"172.21.15.154:6817/3516494456","state":["exists","up"]},{"osd":5,"uuid":"a9f5eaf1-66ff-405a-a9e4-ec3340812a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6801","nonce":540367751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6803","nonce":540367751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6807","nonce":540367751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6805","nonce":540367751}]},"public_addr":"172.21.15.154:6801/540367751","cluster_addr":"172.21.15.154:6803/540367751","heartbeat_back_addr":"172.21.15.154:6807/540367751","heartbeat_front_addr":"172.21.15.154:6805/540367751","state":["exists","up"]},{"osd":6,"uuid":"267d02ea-675f-428f-aae8-e0375e4d4feb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6825","nonce":1550829005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6827","nonce":1550829005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6831","nonce":1550829005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6829","nonce":1550829005}]},"public_addr":"172.21.15.154:6825/1550829005","cluster_addr":"172.21.15.154:6827/1550829005","heartbeat_back_addr":"172.21.15.154:6831/1550829005","heartbeat_front_addr":"172.21.15.154:6829/1550829005","state":["exists","up"]},{"osd":7,"uuid":"759ac6cc-896a-423c-885f-9fe90d8dcb14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6809","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6811","nonce":3920089903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6816","nonce":3920089903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6823","nonce":3920089903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6819","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6821","nonce":3920089903}]},"public_addr":"172.21.15.154:6811/3920089903","cluster_addr":"172.21.15.154:6816/3920089903","heartbeat_back_addr":"172.21.15.154:6823/3920089903","heartbeat_front_addr":"172.21.15.154:6821/3920089903","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.949509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.984072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.907278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.947391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.975447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.998896+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:09.028739+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.894119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:19:34.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.826+0000 7fcda99a8640 1 -- 172.21.15.105:0/4222423329 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fcd9006c8f0 msgr2=0x7fcd9006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:34.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.826+0000 7fcda99a8640 1 --2- 172.21.15.105:0/4222423329 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fcd9006c8f0 0x7fcd9006edb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fcda409c6b0 tx=0x7fcd94008040 comp rx=0 tx=0).stop 2024-04-15T15:19:34.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.826+0000 7fcda99a8640 1 -- 172.21.15.105:0/4222423329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcda409b780 msgr2=0x7fcda40b6850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:34.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.826+0000 7fcda99a8640 1 --2- 172.21.15.105:0/4222423329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcda409b780 0x7fcda40b6850 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcd9800c430 tx=0x7fcd9800c8f0 comp rx=0 tx=0).stop 2024-04-15T15:19:34.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.827+0000 7fcda99a8640 1 -- 172.21.15.105:0/4222423329 shutdown_connections 2024-04-15T15:19:34.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.827+0000 7fcda99a8640 1 --2- 172.21.15.105:0/4222423329 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fcd9006c8f0 0x7fcd9006edb0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:34.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.827+0000 7fcda99a8640 1 --2- 172.21.15.105:0/4222423329 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcda409d460 0x7fcda40b6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:34.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.827+0000 7fcda99a8640 1 --2- 172.21.15.105:0/4222423329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcda409b780 0x7fcda40b6850 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:34.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.827+0000 7fcda99a8640 1 --2- 172.21.15.105:0/4222423329 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcda409b150 0x7fcda415d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:34.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.827+0000 7fcda99a8640 1 -- 172.21.15.105:0/4222423329 >> 172.21.15.105:0/4222423329 conn(0x7fcda40d2570 msgr2=0x7fcda406dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:34.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.827+0000 7fcda99a8640 1 -- 172.21.15.105:0/4222423329 shutdown_connections 2024-04-15T15:19:34.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.827+0000 7fcda99a8640 1 -- 172.21.15.105:0/4222423329 wait complete. 2024-04-15T15:19:34.840 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:19:13.748585+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-15T15:19:23.789287+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-15T15:19:24.722250+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:19:34.840 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:19:34.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.945+0000 7f9e546e4640 1 Processor -- start 2024-04-15T15:19:34.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.945+0000 7f9e546e4640 1 -- start start 2024-04-15T15:19:34.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.946+0000 7f9e546e4640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9e4c0b6c70 0x7f9e4c0d3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:34.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.946+0000 7f9e546e4640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e4c162a10 con 0x7f9e4c15eaf0 2024-04-15T15:19:34.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.946+0000 7f9e546e4640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e4c162cf0 con 0x7f9e4c0d1c70 2024-04-15T15:19:34.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.946+0000 7f9e546e4640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f9e4c162fd0 con 0x7f9e4c0b6c70 2024-04-15T15:19:34.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.946+0000 7f9e52480640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f9e4c15eaf0 0x7f9e4c0d1560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41068/0 (socket says 172.21.15.105:41068) 2024-04-15T15:19:34.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.946+0000 7f9e52480640 1 -- 172.21.15.105:0/510439049 learned_addr learned my addr 172.21.15.105:0/510439049 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:19:34.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.946+0000 7f9e5147e640 1 -- 172.21.15.105:0/510439049 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 401903153 0 0) 0x7f9e4c162a10 con 0x7f9e4c15eaf0 2024-04-15T15:19:34.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.946+0000 7f9e5147e640 1 -- 172.21.15.105:0/510439049 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9e38003520 con 0x7f9e4c15eaf0 2024-04-15T15:19:34.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.946+0000 7f9e52c81640 1 --2- 172.21.15.105:0/510439049 >> v2:172.21.15.154:3300/0 conn(0x7f9e4c0b6c70 0x7f9e4c0d3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:34.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.946+0000 7f9e5147e640 1 -- 172.21.15.105:0/510439049 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3174748729 0 0) 0x7f9e38003520 con 0x7f9e4c15eaf0 2024-04-15T15:19:34.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.946+0000 7f9e5147e640 1 -- 172.21.15.105:0/510439049 >> v2:172.21.15.154:3300/0 conn(0x7f9e4c0b6c70 msgr2=0x7f9e4c0d3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:34.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.946+0000 7f9e5147e640 1 --2- 172.21.15.105:0/510439049 >> v2:172.21.15.154:3300/0 conn(0x7f9e4c0b6c70 0x7f9e4c0d3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:34.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.946+0000 7f9e5147e640 1 -- 172.21.15.105:0/510439049 >> v1:172.21.15.154:6789/0 conn(0x7f9e4c0d1c70 legacy=0x7f9e4c0d2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:34.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.946+0000 7f9e5147e640 1 -- 172.21.15.105:0/510439049 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e4c1632b0 con 0x7f9e4c15eaf0 2024-04-15T15:19:34.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.946+0000 7f9e5147e640 1 -- 172.21.15.105:0/510439049 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7f9e48001670 con 0x7f9e4c15eaf0 2024-04-15T15:19:34.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.946+0000 7f9e5147e640 1 -- 172.21.15.105:0/510439049 >> v1:172.21.15.105:6789/0 conn(0x7f9e4c15eaf0 legacy=0x7f9e4c0d1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:34.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.946+0000 7f9e5147e640 1 --2- 172.21.15.105:0/510439049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e380039b0 0x7f9e38003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:34.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.947+0000 7f9e5147e640 1 --2- 172.21.15.105:0/510439049 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e38004800 0x7f9e38004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:34.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.947+0000 7f9e5147e640 1 --2- 172.21.15.105:0/510439049 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e380051c0 0x7f9e3800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:34.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.947+0000 7f9e5147e640 1 -- 172.21.15.105:0/510439049 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4c162fd0 con 0x7f9e38004800 2024-04-15T15:19:34.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.947+0000 7f9e5147e640 1 -- 172.21.15.105:0/510439049 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9e3800c160 con 0x7f9e380039b0 2024-04-15T15:19:34.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.947+0000 7f9e5147e640 1 -- 172.21.15.105:0/510439049 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f9e3800c360 con 0x7f9e380051c0 2024-04-15T15:19:34.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.947+0000 7f9e51c7f640 1 --2- 172.21.15.105:0/510439049 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e38004800 0x7f9e38004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:34.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.947+0000 7f9e52480640 1 --2- 172.21.15.105:0/510439049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e380039b0 0x7f9e38003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:34.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.947+0000 7f9e52c81640 1 --2- 172.21.15.105:0/510439049 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e380051c0 0x7f9e3800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:34.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.947+0000 7f9e51c7f640 1 -- 172.21.15.105:0/510439049 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e380051c0 msgr2=0x7f9e3800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:34.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.947+0000 7f9e51c7f640 1 --2- 172.21.15.105:0/510439049 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e380051c0 0x7f9e3800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:34.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.947+0000 7f9e51c7f640 1 -- 172.21.15.105:0/510439049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e380039b0 msgr2=0x7f9e38003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:34.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.947+0000 7f9e51c7f640 1 --2- 172.21.15.105:0/510439049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e380039b0 0x7f9e38003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:34.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.947+0000 7f9e52c81640 1 -- 172.21.15.105:0/510439049 reap_dead start 2024-04-15T15:19:34.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.947+0000 7f9e51c7f640 1 -- 172.21.15.105:0/510439049 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e38003520 con 0x7f9e38004800 2024-04-15T15:19:34.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.948+0000 7f9e51c7f640 1 --2- 172.21.15.105:0/510439049 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e38004800 0x7f9e38004c80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c001750 tx=0x7f9e38005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d0d9aeffc384ae3 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:34.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.948+0000 7f9e5147e640 1 -- 172.21.15.105:0/510439049 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c001370 con 0x7f9e38004800 2024-04-15T15:19:34.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.948+0000 7f9e5147e640 1 -- 172.21.15.105:0/510439049 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e3c0052d0 con 0x7f9e38004800 2024-04-15T15:19:34.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.948+0000 7f9e5147e640 1 -- 172.21.15.105:0/510439049 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c003830 con 0x7f9e38004800 2024-04-15T15:19:34.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.948+0000 7f9e546e4640 1 -- 172.21.15.105:0/510439049 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e38004800 msgr2=0x7f9e38004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:34.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.948+0000 7f9e546e4640 1 --2- 172.21.15.105:0/510439049 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e38004800 0x7f9e38004c80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c001750 tx=0x7f9e38005830 comp rx=0 tx=0).stop 2024-04-15T15:19:34.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.948+0000 7f9e546e4640 1 -- 172.21.15.105:0/510439049 shutdown_connections 2024-04-15T15:19:34.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.948+0000 7f9e546e4640 1 --2- 172.21.15.105:0/510439049 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e38004800 0x7f9e38004c80 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:34.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.948+0000 7f9e546e4640 1 -- 172.21.15.105:0/510439049 >> 172.21.15.105:0/510439049 conn(0x7f9e4c06cf90 msgr2=0x7f9e4c06d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:34.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.948+0000 7f9e546e4640 1 -- 172.21.15.105:0/510439049 shutdown_connections 2024-04-15T15:19:34.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.948+0000 7f9e546e4640 1 -- 172.21.15.105:0/510439049 wait complete. 2024-04-15T15:19:34.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.949+0000 7f9e546e4640 1 Processor -- start 2024-04-15T15:19:34.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.950+0000 7f9e546e4640 1 -- start start 2024-04-15T15:19:34.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.950+0000 7f9e546e4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e38004800 0x7f9e4c14fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:34.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.950+0000 7f9e546e4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e4c1f7da0 0x7f9e4c1f8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:34.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.950+0000 7f9e546e4640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e4c1f8760 0x7f9e4c160220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:34.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.950+0000 7f9e546e4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4c0b3d60 con 0x7f9e4c1f7da0 2024-04-15T15:19:34.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.950+0000 7f9e546e4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4c0b70a0 con 0x7f9e38004800 2024-04-15T15:19:34.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.950+0000 7f9e546e4640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f9e4c162650 con 0x7f9e4c1f8760 2024-04-15T15:19:34.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.950+0000 7f9e51c7f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e4c1f7da0 0x7f9e4c1f8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:34.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.950+0000 7f9e51c7f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e4c1f7da0 0x7f9e4c1f8220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39896/0 (socket says 172.21.15.105:39896) 2024-04-15T15:19:34.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.950+0000 7f9e51c7f640 1 -- 172.21.15.105:0/3322027109 learned_addr learned my addr 172.21.15.105:0/3322027109 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:34.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.951+0000 7f9e51c7f640 1 -- 172.21.15.105:0/3322027109 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e4c1f8760 msgr2=0x7f9e4c160220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:34.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.951+0000 7f9e51c7f640 1 --2- 172.21.15.105:0/3322027109 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e4c1f8760 0x7f9e4c160220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:34.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.951+0000 7f9e51c7f640 1 -- 172.21.15.105:0/3322027109 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e38004800 msgr2=0x7f9e4c14fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:34.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.951+0000 7f9e51c7f640 1 --2- 172.21.15.105:0/3322027109 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e38004800 0x7f9e4c14fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:34.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.951+0000 7f9e51c7f640 1 -- 172.21.15.105:0/3322027109 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e4c1f8be0 con 0x7f9e4c1f7da0 2024-04-15T15:19:34.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.951+0000 7f9e51c7f640 1 --2- 172.21.15.105:0/3322027109 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e4c1f7da0 0x7f9e4c1f8220 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c005040 tx=0x7f9e3c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:34.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.951+0000 7f9e42ffd640 1 -- 172.21.15.105:0/3322027109 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c041020 con 0x7f9e4c1f7da0 2024-04-15T15:19:34.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.951+0000 7f9e42ffd640 1 -- 172.21.15.105:0/3322027109 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e3c005480 con 0x7f9e4c1f7da0 2024-04-15T15:19:34.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.951+0000 7f9e546e4640 1 -- 172.21.15.105:0/3322027109 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e4c150040 con 0x7f9e4c1f7da0 2024-04-15T15:19:34.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.951+0000 7f9e42ffd640 1 -- 172.21.15.105:0/3322027109 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c007a30 con 0x7f9e4c1f7da0 2024-04-15T15:19:34.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.951+0000 7f9e546e4640 1 -- 172.21.15.105:0/3322027109 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e4c15f460 con 0x7f9e4c1f7da0 2024-04-15T15:19:34.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.952+0000 7f9e546e4640 1 -- 172.21.15.105:0/3322027109 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e4c068ab0 con 0x7f9e4c1f7da0 2024-04-15T15:19:34.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.952+0000 7f9e42ffd640 1 -- 172.21.15.105:0/3322027109 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9e3c044020 con 0x7f9e4c1f7da0 2024-04-15T15:19:34.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.953+0000 7f9e42ffd640 1 --2- 172.21.15.105:0/3322027109 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f9e3006c8f0 0x7f9e3006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:34.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.953+0000 7f9e42ffd640 1 -- 172.21.15.105:0/3322027109 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9e3c0ad590 con 0x7f9e4c1f7da0 2024-04-15T15:19:34.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.953+0000 7f9e52480640 1 --2- 172.21.15.105:0/3322027109 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f9e3006c8f0 0x7f9e3006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:34.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.953+0000 7f9e52480640 1 --2- 172.21.15.105:0/3322027109 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f9e3006c8f0 0x7f9e3006edb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9e48001eb0 tx=0x7f9e48000ff0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:34.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:34.957+0000 7f9e42ffd640 1 -- 172.21.15.105:0/3322027109 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9e3c04b030 con 0x7f9e4c1f7da0 2024-04-15T15:19:35.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.132+0000 7f9e546e4640 1 -- 172.21.15.105:0/3322027109 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9e4c161f60 con 0x7f9e4c1f7da0 2024-04-15T15:19:35.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.133+0000 7f9e42ffd640 1 -- 172.21.15.105:0/3322027109 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f9e3c0303c0 con 0x7f9e4c1f7da0 2024-04-15T15:19:35.134 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-04-15T15:19:35.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.134+0000 7f9e546e4640 1 -- 172.21.15.105:0/3322027109 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f9e3006c8f0 msgr2=0x7f9e3006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:35.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.134+0000 7f9e546e4640 1 --2- 172.21.15.105:0/3322027109 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f9e3006c8f0 0x7f9e3006edb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9e48001eb0 tx=0x7f9e48000ff0 comp rx=0 tx=0).stop 2024-04-15T15:19:35.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.134+0000 7f9e546e4640 1 -- 172.21.15.105:0/3322027109 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e4c1f7da0 msgr2=0x7f9e4c1f8220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:35.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.134+0000 7f9e546e4640 1 --2- 172.21.15.105:0/3322027109 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e4c1f7da0 0x7f9e4c1f8220 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c005040 tx=0x7f9e3c001120 comp rx=0 tx=0).stop 2024-04-15T15:19:35.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.134+0000 7f9e546e4640 1 -- 172.21.15.105:0/3322027109 shutdown_connections 2024-04-15T15:19:35.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.134+0000 7f9e546e4640 1 --2- 172.21.15.105:0/3322027109 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e4c1f8760 0x7f9e4c160220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:35.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.134+0000 7f9e546e4640 1 --2- 172.21.15.105:0/3322027109 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f9e3006c8f0 0x7f9e3006edb0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:35.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.134+0000 7f9e546e4640 1 --2- 172.21.15.105:0/3322027109 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e4c1f7da0 0x7f9e4c1f8220 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:35.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.134+0000 7f9e546e4640 1 --2- 172.21.15.105:0/3322027109 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e38004800 0x7f9e4c14fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:35.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.134+0000 7f9e546e4640 1 -- 172.21.15.105:0/3322027109 >> 172.21.15.105:0/3322027109 conn(0x7f9e4c06cf90 msgr2=0x7f9e4c0b5ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:35.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.134+0000 7f9e546e4640 1 -- 172.21.15.105:0/3322027109 shutdown_connections 2024-04-15T15:19:35.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.134+0000 7f9e546e4640 1 -- 172.21.15.105:0/3322027109 wait complete. 2024-04-15T15:19:35.147 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-15T15:19:35.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.253+0000 7f9d41d75640 1 Processor -- start 2024-04-15T15:19:35.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.253+0000 7f9d41d75640 1 -- start start 2024-04-15T15:19:35.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.254+0000 7f9d41d75640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9d3c0cfab0 0x7f9d3c15c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:35.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.254+0000 7f9d41d75640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9d3c0cfe90 0x7f9d3c0cf110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:35.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.254+0000 7f9d41d75640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d3c0d23f0 con 0x7f9d3c0cbfc0 2024-04-15T15:19:35.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.254+0000 7f9d41d75640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f9d3c0d1be0 con 0x7f9d3c0cfab0 2024-04-15T15:19:35.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.254+0000 7f9d41d75640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f9d3c0d1e30 con 0x7f9d3c0cfe90 2024-04-15T15:19:35.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.254+0000 7f9d3b7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9d3c0cfab0 0x7f9d3c15c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:35.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.254+0000 7f9d3b7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9d3c0cfab0 0x7f9d3c15c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39912/0 (socket says 172.21.15.105:39912) 2024-04-15T15:19:35.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.254+0000 7f9d3b7fe640 1 -- 172.21.15.105:0/3932920690 learned_addr learned my addr 172.21.15.105:0/3932920690 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:35.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.254+0000 7f9d33fff640 1 --2- 172.21.15.105:0/3932920690 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9d3c0cfe90 0x7f9d3c0cf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:35.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.254+0000 7f9d3b7fe640 1 -- 172.21.15.105:0/3932920690 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9d3c0cfe90 msgr2=0x7f9d3c0cf110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:35.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.254+0000 7f9d3b7fe640 1 --2- 172.21.15.105:0/3932920690 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9d3c0cfe90 0x7f9d3c0cf110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:35.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.254+0000 7f9d3b7fe640 1 -- 172.21.15.105:0/3932920690 >> v1:172.21.15.154:6789/0 conn(0x7f9d3c0cbfc0 legacy=0x7f9d3c0cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:35.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.254+0000 7f9d3b7fe640 1 -- 172.21.15.105:0/3932920690 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d3c0d2110 con 0x7f9d3c0cfab0 2024-04-15T15:19:35.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.254+0000 7f9d33fff640 1 --2- 172.21.15.105:0/3932920690 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9d3c0cfe90 0x7f9d3c0cf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:35.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.255+0000 7f9d3b7fe640 1 --2- 172.21.15.105:0/3932920690 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d3c0cfab0 0x7f9d3c15c670 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c009870 tx=0x7f9d2c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f95151d02e6329a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:35.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.255+0000 7f9d3affd640 1 -- 172.21.15.105:0/3932920690 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c035020 con 0x7f9d3c0cfab0 2024-04-15T15:19:35.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.255+0000 7f9d3affd640 1 -- 172.21.15.105:0/3932920690 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d2c031640 con 0x7f9d3c0cfab0 2024-04-15T15:19:35.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.255+0000 7f9d3affd640 1 -- 172.21.15.105:0/3932920690 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c02d040 con 0x7f9d3c0cfab0 2024-04-15T15:19:35.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.255+0000 7f9d41d75640 1 -- 172.21.15.105:0/3932920690 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d3c0cfab0 msgr2=0x7f9d3c15c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:35.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.255+0000 7f9d41d75640 1 --2- 172.21.15.105:0/3932920690 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d3c0cfab0 0x7f9d3c15c670 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c009870 tx=0x7f9d2c02b8f0 comp rx=0 tx=0).stop 2024-04-15T15:19:35.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.256+0000 7f9d41d75640 1 -- 172.21.15.105:0/3932920690 shutdown_connections 2024-04-15T15:19:35.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.256+0000 7f9d41d75640 1 --2- 172.21.15.105:0/3932920690 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9d3c0cfe90 0x7f9d3c0cf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:35.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.256+0000 7f9d41d75640 1 --2- 172.21.15.105:0/3932920690 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d3c0cfab0 0x7f9d3c15c670 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:35.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.256+0000 7f9d41d75640 1 -- 172.21.15.105:0/3932920690 >> 172.21.15.105:0/3932920690 conn(0x7f9d3c09aa40 msgr2=0x7f9d3c0d14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:35.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.256+0000 7f9d41d75640 1 -- 172.21.15.105:0/3932920690 shutdown_connections 2024-04-15T15:19:35.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.256+0000 7f9d41d75640 1 -- 172.21.15.105:0/3932920690 wait complete. 2024-04-15T15:19:35.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.257+0000 7f9d41d75640 1 Processor -- start 2024-04-15T15:19:35.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.257+0000 7f9d41d75640 1 -- start start 2024-04-15T15:19:35.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.257+0000 7f9d41d75640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d3c0cbfc0 0x7f9d3c0ca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:35.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.257+0000 7f9d41d75640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9d3c0cfab0 0x7f9d3c1f6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:35.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.257+0000 7f9d41d75640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d3c0cfe90 0x7f9d3c1f6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:35.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.257+0000 7f9d41d75640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9d3c15bb80 con 0x7f9d3c0cfe90 2024-04-15T15:19:35.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.257+0000 7f9d41d75640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9d3c15c150 con 0x7f9d3c0cbfc0 2024-04-15T15:19:35.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.257+0000 7f9d41d75640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f9d3c0ccb60 con 0x7f9d3c0cfab0 2024-04-15T15:19:35.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.258+0000 7f9d3bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d3c0cfe90 0x7f9d3c1f6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:35.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.258+0000 7f9d3bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d3c0cfe90 0x7f9d3c1f6bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39920/0 (socket says 172.21.15.105:39920) 2024-04-15T15:19:35.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.258+0000 7f9d3bfff640 1 -- 172.21.15.105:0/1816180917 learned_addr learned my addr 172.21.15.105:0/1816180917 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:35.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.258+0000 7f9d33fff640 1 --2- 172.21.15.105:0/1816180917 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9d3c0cfab0 0x7f9d3c1f6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:35.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.258+0000 7f9d3b7fe640 1 --2- 172.21.15.105:0/1816180917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d3c0cbfc0 0x7f9d3c0ca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:35.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.258+0000 7f9d3bfff640 1 -- 172.21.15.105:0/1816180917 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9d3c0cfab0 msgr2=0x7f9d3c1f6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:35.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.258+0000 7f9d3bfff640 1 --2- 172.21.15.105:0/1816180917 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9d3c0cfab0 0x7f9d3c1f6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:35.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.258+0000 7f9d3bfff640 1 -- 172.21.15.105:0/1816180917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d3c0cbfc0 msgr2=0x7f9d3c0ca9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:35.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.258+0000 7f9d3bfff640 1 --2- 172.21.15.105:0/1816180917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d3c0cbfc0 0x7f9d3c0ca9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:35.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.258+0000 7f9d3bfff640 1 -- 172.21.15.105:0/1816180917 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d3c15e800 con 0x7f9d3c0cfe90 2024-04-15T15:19:35.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.258+0000 7f9d3b7fe640 1 --2- 172.21.15.105:0/1816180917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d3c0cbfc0 0x7f9d3c0ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:19:35.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.258+0000 7f9d3bfff640 1 --2- 172.21.15.105:0/1816180917 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d3c0cfe90 0x7f9d3c1f6bb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9d3400c780 tx=0x7f9d3400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:35.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.258+0000 7f9d38ff9640 1 -- 172.21.15.105:0/1816180917 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d34004b60 con 0x7f9d3c0cfe90 2024-04-15T15:19:35.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.258+0000 7f9d38ff9640 1 -- 172.21.15.105:0/1816180917 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d34004d00 con 0x7f9d3c0cfe90 2024-04-15T15:19:35.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.258+0000 7f9d41d75640 1 -- 172.21.15.105:0/1816180917 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d3c0b3890 con 0x7f9d3c0cfe90 2024-04-15T15:19:35.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.258+0000 7f9d38ff9640 1 -- 172.21.15.105:0/1816180917 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d3400aa90 con 0x7f9d3c0cfe90 2024-04-15T15:19:35.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.258+0000 7f9d41d75640 1 -- 172.21.15.105:0/1816180917 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d3c146840 con 0x7f9d3c0cfe90 2024-04-15T15:19:35.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.259+0000 7f9d41d75640 1 -- 172.21.15.105:0/1816180917 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d3c069150 con 0x7f9d3c0cfe90 2024-04-15T15:19:35.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.260+0000 7f9d38ff9640 1 -- 172.21.15.105:0/1816180917 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9d3400ac30 con 0x7f9d3c0cfe90 2024-04-15T15:19:35.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.260+0000 7f9d38ff9640 1 --2- 172.21.15.105:0/1816180917 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f9d1c06c8f0 0x7f9d1c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:35.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.260+0000 7f9d38ff9640 1 -- 172.21.15.105:0/1816180917 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9d3408bd60 con 0x7f9d3c0cfe90 2024-04-15T15:19:35.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.260+0000 7f9d3b7fe640 1 --2- 172.21.15.105:0/1816180917 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f9d1c06c8f0 0x7f9d1c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:35.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.261+0000 7f9d3b7fe640 1 --2- 172.21.15.105:0/1816180917 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f9d1c06c8f0 0x7f9d1c06edb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c036000 tx=0x7f9d2c03c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:35.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.264+0000 7f9d38ff9640 1 -- 172.21.15.105:0/1816180917 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9d34020070 con 0x7f9d3c0cfe90 2024-04-15T15:19:35.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.440+0000 7f9d41d75640 1 -- 172.21.15.105:0/1816180917 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9d3c0cec30 con 0x7f9d3c0cfe90 2024-04-15T15:19:35.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.441+0000 7f9d38ff9640 1 -- 172.21.15.105:0/1816180917 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f9d3405a190 con 0x7f9d3c0cfe90 2024-04-15T15:19:35.442 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-04-15T15:19:35.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.442+0000 7f9d41d75640 1 -- 172.21.15.105:0/1816180917 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f9d1c06c8f0 msgr2=0x7f9d1c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:35.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.442+0000 7f9d41d75640 1 --2- 172.21.15.105:0/1816180917 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f9d1c06c8f0 0x7f9d1c06edb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c036000 tx=0x7f9d2c03c040 comp rx=0 tx=0).stop 2024-04-15T15:19:35.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.442+0000 7f9d41d75640 1 -- 172.21.15.105:0/1816180917 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d3c0cfe90 msgr2=0x7f9d3c1f6bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:35.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.442+0000 7f9d41d75640 1 --2- 172.21.15.105:0/1816180917 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d3c0cfe90 0x7f9d3c1f6bb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9d3400c780 tx=0x7f9d3400cc40 comp rx=0 tx=0).stop 2024-04-15T15:19:35.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.442+0000 7f9d41d75640 1 -- 172.21.15.105:0/1816180917 shutdown_connections 2024-04-15T15:19:35.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.442+0000 7f9d41d75640 1 --2- 172.21.15.105:0/1816180917 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9d3c0cfab0 0x7f9d3c1f6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:35.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.442+0000 7f9d41d75640 1 --2- 172.21.15.105:0/1816180917 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f9d1c06c8f0 0x7f9d1c06edb0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:35.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.442+0000 7f9d41d75640 1 --2- 172.21.15.105:0/1816180917 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d3c0cfe90 0x7f9d3c1f6bb0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:35.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.442+0000 7f9d41d75640 1 --2- 172.21.15.105:0/1816180917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d3c0cbfc0 0x7f9d3c0ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:35.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.443+0000 7f9d41d75640 1 -- 172.21.15.105:0/1816180917 >> 172.21.15.105:0/1816180917 conn(0x7f9d3c09aa40 msgr2=0x7f9d3c0aed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:35.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.443+0000 7f9d41d75640 1 -- 172.21.15.105:0/1816180917 shutdown_connections 2024-04-15T15:19:35.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.443+0000 7f9d41d75640 1 -- 172.21.15.105:0/1816180917 wait complete. 2024-04-15T15:19:35.455 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-15T15:19:35.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.559+0000 7fea181d4640 1 Processor -- start 2024-04-15T15:19:35.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.559+0000 7fea181d4640 1 -- start start 2024-04-15T15:19:35.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.560+0000 7fea181d4640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea1015a3f0 0x7fea1015d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:35.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.560+0000 7fea181d4640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea100ade60 con 0x7fea100cd290 2024-04-15T15:19:35.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.560+0000 7fea181d4640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea100ae140 con 0x7fea100af820 2024-04-15T15:19:35.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.560+0000 7fea181d4640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fea100aef00 con 0x7fea1015a3f0 2024-04-15T15:19:35.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.560+0000 7fea1576f640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fea100cd290 0x7fea100af110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41082/0 (socket says 172.21.15.105:41082) 2024-04-15T15:19:35.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.560+0000 7fea1576f640 1 -- 172.21.15.105:0/4238682098 learned_addr learned my addr 172.21.15.105:0/4238682098 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:19:35.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.560+0000 7fea15f70640 1 --2- 172.21.15.105:0/4238682098 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea1015a3f0 0x7fea1015d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:35.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.560+0000 7fea14f6e640 1 -- 172.21.15.105:0/4238682098 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2888913754 0 0) 0x7fea100ade60 con 0x7fea100cd290 2024-04-15T15:19:35.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.560+0000 7fea14f6e640 1 -- 172.21.15.105:0/4238682098 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe9fc003520 con 0x7fea100cd290 2024-04-15T15:19:35.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.561+0000 7fea14f6e640 1 -- 172.21.15.105:0/4238682098 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2851629526 0 0) 0x7fe9fc003520 con 0x7fea100cd290 2024-04-15T15:19:35.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.561+0000 7fea14f6e640 1 -- 172.21.15.105:0/4238682098 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea1015a3f0 msgr2=0x7fea1015d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:35.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.561+0000 7fea14f6e640 1 --2- 172.21.15.105:0/4238682098 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea1015a3f0 0x7fea1015d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:35.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.561+0000 7fea14f6e640 1 -- 172.21.15.105:0/4238682098 >> v1:172.21.15.154:6789/0 conn(0x7fea100af820 legacy=0x7fea1006e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:35.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.561+0000 7fea14f6e640 1 -- 172.21.15.105:0/4238682098 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea100ae420 con 0x7fea100cd290 2024-04-15T15:19:35.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.561+0000 7fea14f6e640 1 -- 172.21.15.105:0/4238682098 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7fea0c001670 con 0x7fea100cd290 2024-04-15T15:19:35.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.561+0000 7fea14f6e640 1 -- 172.21.15.105:0/4238682098 >> v1:172.21.15.105:6789/0 conn(0x7fea100cd290 legacy=0x7fea100af110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:35.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.561+0000 7fea14f6e640 1 --2- 172.21.15.105:0/4238682098 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea1015a3f0 0x7fe9fc0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:35.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.561+0000 7fea14f6e640 1 --2- 172.21.15.105:0/4238682098 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9fc003f20 0x7fe9fc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:35.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.561+0000 7fea14f6e640 1 --2- 172.21.15.105:0/4238682098 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9fc004d40 0x7fe9fc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:35.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.561+0000 7fea14f6e640 1 -- 172.21.15.105:0/4238682098 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe9fc00b9f0 con 0x7fe9fc004d40 2024-04-15T15:19:35.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.561+0000 7fea14f6e640 1 -- 172.21.15.105:0/4238682098 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe9fc00bba0 con 0x7fe9fc003f20 2024-04-15T15:19:35.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.561+0000 7fea14f6e640 1 -- 172.21.15.105:0/4238682098 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fe9fc00bda0 con 0x7fea1015a3f0 2024-04-15T15:19:35.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.561+0000 7fea16771640 1 --2- 172.21.15.105:0/4238682098 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9fc004d40 0x7fe9fc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:35.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.561+0000 7fea15f70640 1 --2- 172.21.15.105:0/4238682098 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea1015a3f0 0x7fe9fc0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:35.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.561+0000 7fea1576f640 1 --2- 172.21.15.105:0/4238682098 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9fc003f20 0x7fe9fc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:35.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.561+0000 7fea16771640 1 -- 172.21.15.105:0/4238682098 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea1015a3f0 msgr2=0x7fe9fc0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:35.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.561+0000 7fea16771640 1 --2- 172.21.15.105:0/4238682098 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea1015a3f0 0x7fe9fc0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:35.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.561+0000 7fea16771640 1 -- 172.21.15.105:0/4238682098 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9fc003f20 msgr2=0x7fe9fc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:35.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.561+0000 7fea16771640 1 --2- 172.21.15.105:0/4238682098 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9fc003f20 0x7fe9fc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:35.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.561+0000 7fea16771640 1 -- 172.21.15.105:0/4238682098 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9fc003520 con 0x7fe9fc004d40 2024-04-15T15:19:35.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.561+0000 7fea1576f640 1 --2- 172.21.15.105:0/4238682098 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9fc003f20 0x7fe9fc004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:19:35.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.562+0000 7fea16771640 1 --2- 172.21.15.105:0/4238682098 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9fc004d40 0x7fe9fc0051c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fea100cd050 tx=0x7fe9fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85bf7fb7a7b82647 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:35.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.562+0000 7fea14f6e640 1 -- 172.21.15.105:0/4238682098 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea08038070 con 0x7fe9fc004d40 2024-04-15T15:19:35.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.562+0000 7fea14f6e640 1 -- 172.21.15.105:0/4238682098 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea08033630 con 0x7fe9fc004d40 2024-04-15T15:19:35.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.562+0000 7fea14f6e640 1 -- 172.21.15.105:0/4238682098 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0802f050 con 0x7fe9fc004d40 2024-04-15T15:19:35.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.562+0000 7fea181d4640 1 -- 172.21.15.105:0/4238682098 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9fc004d40 msgr2=0x7fe9fc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:35.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.562+0000 7fea181d4640 1 --2- 172.21.15.105:0/4238682098 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9fc004d40 0x7fe9fc0051c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fea100cd050 tx=0x7fe9fc005830 comp rx=0 tx=0).stop 2024-04-15T15:19:35.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.562+0000 7fea16771640 1 -- 172.21.15.105:0/4238682098 reap_dead start 2024-04-15T15:19:35.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.562+0000 7fea181d4640 1 -- 172.21.15.105:0/4238682098 shutdown_connections 2024-04-15T15:19:35.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.562+0000 7fea181d4640 1 -- 172.21.15.105:0/4238682098 >> 172.21.15.105:0/4238682098 conn(0x7fea10073570 msgr2=0x7fea1006b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:35.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.563+0000 7fea181d4640 1 -- 172.21.15.105:0/4238682098 shutdown_connections 2024-04-15T15:19:35.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.563+0000 7fea181d4640 1 -- 172.21.15.105:0/4238682098 wait complete. 2024-04-15T15:19:35.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.564+0000 7fea181d4640 1 Processor -- start 2024-04-15T15:19:35.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.564+0000 7fea181d4640 1 -- start start 2024-04-15T15:19:35.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.564+0000 7fea181d4640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea10153af0 0x7fea10155190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:35.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.564+0000 7fea181d4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea1006b9c0 0x7fea1006be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:35.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.564+0000 7fea181d4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea101f1e00 0x7fea101f2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:35.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.564+0000 7fea181d4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fea100704c0 con 0x7fea101f1e00 2024-04-15T15:19:35.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.564+0000 7fea181d4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fea100ae9e0 con 0x7fea1006b9c0 2024-04-15T15:19:35.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.564+0000 7fea181d4640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fea1015d720 con 0x7fea10153af0 2024-04-15T15:19:35.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.565+0000 7fea16771640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea101f1e00 0x7fea101f2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:35.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.565+0000 7fea16771640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea101f1e00 0x7fea101f2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39926/0 (socket says 172.21.15.105:39926) 2024-04-15T15:19:35.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.565+0000 7fea16771640 1 -- 172.21.15.105:0/2211802637 learned_addr learned my addr 172.21.15.105:0/2211802637 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:35.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.565+0000 7fea15f70640 1 --2- 172.21.15.105:0/2211802637 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea10153af0 0x7fea10155190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:35.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.565+0000 7fea1576f640 1 --2- 172.21.15.105:0/2211802637 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea1006b9c0 0x7fea1006be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:35.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.565+0000 7fea16771640 1 -- 172.21.15.105:0/2211802637 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea10153af0 msgr2=0x7fea10155190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:35.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.565+0000 7fea16771640 1 --2- 172.21.15.105:0/2211802637 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea10153af0 0x7fea10155190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:35.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.565+0000 7fea16771640 1 -- 172.21.15.105:0/2211802637 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea1006b9c0 msgr2=0x7fea1006be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:35.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.565+0000 7fea16771640 1 --2- 172.21.15.105:0/2211802637 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea1006b9c0 0x7fea1006be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:35.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.565+0000 7fea16771640 1 -- 172.21.15.105:0/2211802637 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea101f27c0 con 0x7fea101f1e00 2024-04-15T15:19:35.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.565+0000 7fea1576f640 1 --2- 172.21.15.105:0/2211802637 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea1006b9c0 0x7fea1006be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:35.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.565+0000 7fea16771640 1 --2- 172.21.15.105:0/2211802637 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea101f1e00 0x7fea101f2280 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fea080019d0 tx=0x7fea080012b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:35.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.565+0000 7fea027fc640 1 -- 172.21.15.105:0/2211802637 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0802f050 con 0x7fea101f1e00 2024-04-15T15:19:35.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.565+0000 7fea027fc640 1 -- 172.21.15.105:0/2211802637 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea08033430 con 0x7fea101f1e00 2024-04-15T15:19:35.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.565+0000 7fea027fc640 1 -- 172.21.15.105:0/2211802637 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea08038040 con 0x7fea101f1e00 2024-04-15T15:19:35.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.565+0000 7fea181d4640 1 -- 172.21.15.105:0/2211802637 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea101556d0 con 0x7fea101f1e00 2024-04-15T15:19:35.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.565+0000 7fea181d4640 1 -- 172.21.15.105:0/2211802637 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea101f2a30 con 0x7fea101f1e00 2024-04-15T15:19:35.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.566+0000 7fea181d4640 1 -- 172.21.15.105:0/2211802637 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea1006c380 con 0x7fea101f1e00 2024-04-15T15:19:35.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.567+0000 7fea027fc640 1 -- 172.21.15.105:0/2211802637 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fea08044020 con 0x7fea101f1e00 2024-04-15T15:19:35.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.567+0000 7fea027fc640 1 --2- 172.21.15.105:0/2211802637 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fe9f406c920 0x7fe9f406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:35.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.567+0000 7fea027fc640 1 -- 172.21.15.105:0/2211802637 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fea080ac2a0 con 0x7fea101f1e00 2024-04-15T15:19:35.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.567+0000 7fea15f70640 1 --2- 172.21.15.105:0/2211802637 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fe9f406c920 0x7fe9f406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:35.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.567+0000 7fea15f70640 1 --2- 172.21.15.105:0/2211802637 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fe9f406c920 0x7fe9f406ede0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fea04007520 tx=0x7fea0400b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:35.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.571+0000 7fea027fc640 1 -- 172.21.15.105:0/2211802637 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fea0807a6d0 con 0x7fea101f1e00 2024-04-15T15:19:35.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.748+0000 7fea181d4640 1 -- 172.21.15.105:0/2211802637 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fea1006c620 con 0x7fea101f1e00 2024-04-15T15:19:35.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.749+0000 7fea027fc640 1 -- 172.21.15.105:0/2211802637 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fea08079fd0 con 0x7fea101f1e00 2024-04-15T15:19:35.751 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-04-15T15:19:35.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.751+0000 7fea181d4640 1 -- 172.21.15.105:0/2211802637 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fe9f406c920 msgr2=0x7fe9f406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:35.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.751+0000 7fea181d4640 1 --2- 172.21.15.105:0/2211802637 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fe9f406c920 0x7fe9f406ede0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fea04007520 tx=0x7fea0400b040 comp rx=0 tx=0).stop 2024-04-15T15:19:35.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.751+0000 7fea181d4640 1 -- 172.21.15.105:0/2211802637 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea101f1e00 msgr2=0x7fea101f2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:35.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.751+0000 7fea181d4640 1 --2- 172.21.15.105:0/2211802637 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea101f1e00 0x7fea101f2280 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fea080019d0 tx=0x7fea080012b0 comp rx=0 tx=0).stop 2024-04-15T15:19:35.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.751+0000 7fea181d4640 1 -- 172.21.15.105:0/2211802637 shutdown_connections 2024-04-15T15:19:35.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.751+0000 7fea181d4640 1 --2- 172.21.15.105:0/2211802637 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fe9f406c920 0x7fe9f406ede0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:35.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.751+0000 7fea181d4640 1 --2- 172.21.15.105:0/2211802637 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea101f1e00 0x7fea101f2280 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:35.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.751+0000 7fea181d4640 1 --2- 172.21.15.105:0/2211802637 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea1006b9c0 0x7fea1006be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:35.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.751+0000 7fea181d4640 1 --2- 172.21.15.105:0/2211802637 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea10153af0 0x7fea10155190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:35.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.751+0000 7fea181d4640 1 -- 172.21.15.105:0/2211802637 >> 172.21.15.105:0/2211802637 conn(0x7fea10073570 msgr2=0x7fea100d0340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:35.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.751+0000 7fea181d4640 1 -- 172.21.15.105:0/2211802637 shutdown_connections 2024-04-15T15:19:35.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.751+0000 7fea181d4640 1 -- 172.21.15.105:0/2211802637 wait complete. 2024-04-15T15:19:35.763 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-15T15:19:35.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.864+0000 7f35c9996640 1 Processor -- start 2024-04-15T15:19:35.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.864+0000 7f35c9996640 1 -- start start 2024-04-15T15:19:35.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.864+0000 7f35c9996640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f35c40ba480 0x7f35c40d1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:35.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.864+0000 7f35c9996640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f35c40d1650 0x7f35c40b7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:35.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.864+0000 7f35c9996640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35c40d3400 con 0x7f35c415e8d0 2024-04-15T15:19:35.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.864+0000 7f35c9996640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f35c40d31c0 con 0x7f35c40ba480 2024-04-15T15:19:35.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.864+0000 7f35c9996640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f35c40d1aa0 con 0x7f35c40d1650 2024-04-15T15:19:35.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.865+0000 7f35c27fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f35c40ba480 0x7f35c40d1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:35.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.865+0000 7f35c2ffd640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f35c415e8d0 0x7f35c40d2ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41096/0 (socket says 172.21.15.105:41096) 2024-04-15T15:19:35.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.865+0000 7f35c2ffd640 1 -- 172.21.15.105:0/1752602444 learned_addr learned my addr 172.21.15.105:0/1752602444 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:19:35.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.865+0000 7f35c27fc640 1 --2- 172.21.15.105:0/1752602444 >> v2:172.21.15.105:3300/0 conn(0x7f35c40ba480 0x7f35c40d1110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39932/0 (socket says 172.21.15.105:39932) 2024-04-15T15:19:35.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.865+0000 7f35c27fc640 1 -- 172.21.15.105:0/1752602444 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f35c40d1650 msgr2=0x7f35c40b7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:35.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.865+0000 7f35c37fe640 1 --2- 172.21.15.105:0/1752602444 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f35c40d1650 0x7f35c40b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:35.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.865+0000 7f35c27fc640 1 --2- 172.21.15.105:0/1752602444 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f35c40d1650 0x7f35c40b7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:35.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.865+0000 7f35c27fc640 1 -- 172.21.15.105:0/1752602444 >> v1:172.21.15.105:6789/0 conn(0x7f35c415e8d0 legacy=0x7f35c40d2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:35.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.865+0000 7f35c1ffb640 1 -- 172.21.15.105:0/1752602444 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 169982238 0 0) 0x7f35c40d3400 con 0x7f35c415e8d0 2024-04-15T15:19:35.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.865+0000 7f35c27fc640 1 -- 172.21.15.105:0/1752602444 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35c40d36e0 con 0x7f35c40ba480 2024-04-15T15:19:35.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.865+0000 7f35c37fe640 1 --2- 172.21.15.105:0/1752602444 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f35c40d1650 0x7f35c40b7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:35.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.866+0000 7f35c27fc640 1 --2- 172.21.15.105:0/1752602444 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35c40ba480 0x7f35c40d1110 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f35b8009800 tx=0x7f35b802b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3d2b068cb2f7eff server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:35.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.866+0000 7f35c1ffb640 1 -- 172.21.15.105:0/1752602444 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b8035020 con 0x7f35c40ba480 2024-04-15T15:19:35.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.866+0000 7f35c1ffb640 1 -- 172.21.15.105:0/1752602444 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35b8031890 con 0x7f35c40ba480 2024-04-15T15:19:35.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.866+0000 7f35c1ffb640 1 -- 172.21.15.105:0/1752602444 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b8031c60 con 0x7f35c40ba480 2024-04-15T15:19:35.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.866+0000 7f35c9996640 1 -- 172.21.15.105:0/1752602444 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35c40ba480 msgr2=0x7f35c40d1110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:35.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.866+0000 7f35c9996640 1 --2- 172.21.15.105:0/1752602444 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35c40ba480 0x7f35c40d1110 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f35b8009800 tx=0x7f35b802b930 comp rx=0 tx=0).stop 2024-04-15T15:19:35.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.866+0000 7f35c9996640 1 -- 172.21.15.105:0/1752602444 shutdown_connections 2024-04-15T15:19:35.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.866+0000 7f35c9996640 1 --2- 172.21.15.105:0/1752602444 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f35c40d1650 0x7f35c40b7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:35.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.866+0000 7f35c9996640 1 --2- 172.21.15.105:0/1752602444 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35c40ba480 0x7f35c40d1110 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:35.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.866+0000 7f35c9996640 1 -- 172.21.15.105:0/1752602444 >> 172.21.15.105:0/1752602444 conn(0x7f35c406d460 msgr2=0x7f35c4067720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:35.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.866+0000 7f35c9996640 1 -- 172.21.15.105:0/1752602444 shutdown_connections 2024-04-15T15:19:35.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.866+0000 7f35c9996640 1 -- 172.21.15.105:0/1752602444 wait complete. 2024-04-15T15:19:35.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.867+0000 7f35c9996640 1 Processor -- start 2024-04-15T15:19:35.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.868+0000 7f35c9996640 1 -- start start 2024-04-15T15:19:35.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.868+0000 7f35c9996640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35c40ba480 0x7f35c415afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:35.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.868+0000 7f35c9996640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35c40d1650 0x7f35c415b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:35.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.868+0000 7f35c9996640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f35c415e8d0 0x7f35c4148c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:35.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.868+0000 7f35c9996640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f35c40d3f80 con 0x7f35c40ba480 2024-04-15T15:19:35.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.868+0000 7f35c9996640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f35c40d3a80 con 0x7f35c40d1650 2024-04-15T15:19:35.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.868+0000 7f35c9996640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f35c40d44a0 con 0x7f35c415e8d0 2024-04-15T15:19:35.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.868+0000 7f35c2ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35c40ba480 0x7f35c415afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:35.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.868+0000 7f35c2ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35c40ba480 0x7f35c415afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39944/0 (socket says 172.21.15.105:39944) 2024-04-15T15:19:35.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.868+0000 7f35c2ffd640 1 -- 172.21.15.105:0/559336130 learned_addr learned my addr 172.21.15.105:0/559336130 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:35.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.869+0000 7f35c37fe640 1 --2- 172.21.15.105:0/559336130 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f35c415e8d0 0x7f35c4148c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:35.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.869+0000 7f35c2ffd640 1 -- 172.21.15.105:0/559336130 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f35c415e8d0 msgr2=0x7f35c4148c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:35.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.869+0000 7f35c2ffd640 1 --2- 172.21.15.105:0/559336130 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f35c415e8d0 0x7f35c4148c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:35.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.869+0000 7f35c2ffd640 1 -- 172.21.15.105:0/559336130 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35c40d1650 msgr2=0x7f35c415b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:35.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.869+0000 7f35c27fc640 1 --2- 172.21.15.105:0/559336130 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35c40d1650 0x7f35c415b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:35.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.869+0000 7f35c2ffd640 1 --2- 172.21.15.105:0/559336130 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35c40d1650 0x7f35c415b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:35.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.869+0000 7f35c2ffd640 1 -- 172.21.15.105:0/559336130 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35c4146c20 con 0x7f35c40ba480 2024-04-15T15:19:35.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.869+0000 7f35c37fe640 1 --2- 172.21.15.105:0/559336130 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f35c415e8d0 0x7f35c4148c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:35.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.869+0000 7f35c2ffd640 1 --2- 172.21.15.105:0/559336130 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35c40ba480 0x7f35c415afa0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f35b400c430 tx=0x7f35b400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:35.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.869+0000 7f35abfff640 1 -- 172.21.15.105:0/559336130 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b40074e0 con 0x7f35c40ba480 2024-04-15T15:19:35.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.869+0000 7f35abfff640 1 -- 172.21.15.105:0/559336130 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35b40047e0 con 0x7f35c40ba480 2024-04-15T15:19:35.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.869+0000 7f35c9996640 1 -- 172.21.15.105:0/559336130 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35c4160770 con 0x7f35c40ba480 2024-04-15T15:19:35.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.869+0000 7f35abfff640 1 -- 172.21.15.105:0/559336130 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b4004bd0 con 0x7f35c40ba480 2024-04-15T15:19:35.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.869+0000 7f35c9996640 1 -- 172.21.15.105:0/559336130 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35c4160a50 con 0x7f35c40ba480 2024-04-15T15:19:35.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.870+0000 7f35c9996640 1 -- 172.21.15.105:0/559336130 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35c40d0650 con 0x7f35c40ba480 2024-04-15T15:19:35.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.871+0000 7f35abfff640 1 -- 172.21.15.105:0/559336130 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f35b4011070 con 0x7f35c40ba480 2024-04-15T15:19:35.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.871+0000 7f35abfff640 1 --2- 172.21.15.105:0/559336130 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f35b006c8f0 0x7f35b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:35.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.871+0000 7f35c27fc640 1 --2- 172.21.15.105:0/559336130 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f35b006c8f0 0x7f35b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:35.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.871+0000 7f35abfff640 1 -- 172.21.15.105:0/559336130 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f35b408b780 con 0x7f35c40ba480 2024-04-15T15:19:35.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.871+0000 7f35c27fc640 1 --2- 172.21.15.105:0/559336130 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f35b006c8f0 0x7f35b006edb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f35b8036000 tx=0x7f35b8008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:35.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:35.875+0000 7f35abfff640 1 -- 172.21.15.105:0/559336130 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f35c40d0650 con 0x7f35c40ba480 2024-04-15T15:19:36.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.089+0000 7f35c9996640 1 -- 172.21.15.105:0/559336130 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f35c4068a40 con 0x7f35c40ba480 2024-04-15T15:19:36.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.089+0000 7f35abfff640 1 -- 172.21.15.105:0/559336130 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5187 (secure 0 0 0) 0x7f35b400e280 con 0x7f35c40ba480 2024-04-15T15:19:36.091 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:19:36.091 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4259,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/3319657830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":3319657830},{"type":"v1","addr":"172.21.15.154:6835","nonce":3319657830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2677199890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2677199890},{"type":"v1","addr":"172.21.15.105:6835","nonce":2677199890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2867739743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2867739743},{"type":"v1","addr":"172.21.15.105:6837","nonce":2867739743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-15T15:19:25.760362+0000","modified":"2024-04-15T15:19:27.857599+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.154:6834/1529675504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1529675504},{"type":"v1","addr":"172.21.15.154:6834","nonce":1529675504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4291,"qdb_cluster":[4291]},"id":1}]} 2024-04-15T15:19:36.092 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 6 2024-04-15T15:19:36.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.091+0000 7f35c9996640 1 -- 172.21.15.105:0/559336130 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f35b006c8f0 msgr2=0x7f35b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:36.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.091+0000 7f35c9996640 1 --2- 172.21.15.105:0/559336130 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f35b006c8f0 0x7f35b006edb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f35b8036000 tx=0x7f35b8008030 comp rx=0 tx=0).stop 2024-04-15T15:19:36.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.091+0000 7f35c9996640 1 -- 172.21.15.105:0/559336130 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35c40ba480 msgr2=0x7f35c415afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:36.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.091+0000 7f35c9996640 1 --2- 172.21.15.105:0/559336130 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35c40ba480 0x7f35c415afa0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f35b400c430 tx=0x7f35b400c8f0 comp rx=0 tx=0).stop 2024-04-15T15:19:36.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.091+0000 7f35c9996640 1 -- 172.21.15.105:0/559336130 shutdown_connections 2024-04-15T15:19:36.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.091+0000 7f35c9996640 1 --2- 172.21.15.105:0/559336130 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f35c415e8d0 0x7f35c4148c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:36.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.091+0000 7f35c9996640 1 --2- 172.21.15.105:0/559336130 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f35b006c8f0 0x7f35b006edb0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:36.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.091+0000 7f35c9996640 1 --2- 172.21.15.105:0/559336130 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35c40d1650 0x7f35c415b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:36.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.091+0000 7f35c9996640 1 --2- 172.21.15.105:0/559336130 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35c40ba480 0x7f35c415afa0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:36.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.091+0000 7f35c9996640 1 -- 172.21.15.105:0/559336130 >> 172.21.15.105:0/559336130 conn(0x7f35c406d460 msgr2=0x7f35c40c8e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:36.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.091+0000 7f35c9996640 1 -- 172.21.15.105:0/559336130 shutdown_connections 2024-04-15T15:19:36.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.091+0000 7f35c9996640 1 -- 172.21.15.105:0/559336130 wait complete. 2024-04-15T15:19:36.104 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:19:36.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.205+0000 7f6150ed9640 1 Processor -- start 2024-04-15T15:19:36.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.205+0000 7f6150ed9640 1 -- start start 2024-04-15T15:19:36.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.206+0000 7f6150ed9640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f614c1554d0 0x7f614c0afaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:36.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.206+0000 7f6150ed9640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f614c0726f0 0x7f614c072af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:36.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.206+0000 7f6150ed9640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f614c0b5a30 con 0x7f614c0b5580 2024-04-15T15:19:36.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.206+0000 7f6150ed9640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f614c0cf280 con 0x7f614c0726f0 2024-04-15T15:19:36.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.206+0000 7f6150ed9640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f614c073030 con 0x7f614c1554d0 2024-04-15T15:19:36.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.206+0000 7f614ad9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f614c0726f0 0x7f614c072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:36.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.206+0000 7f614ad9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f614c0726f0 0x7f614c072af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39950/0 (socket says 172.21.15.105:39950) 2024-04-15T15:19:36.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.206+0000 7f614ad9d640 1 -- 172.21.15.105:0/629369305 learned_addr learned my addr 172.21.15.105:0/629369305 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:36.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.206+0000 7f6141d9b640 1 --1- 172.21.15.105:0/629369305 >> v1:172.21.15.105:6789/0 conn(0x7f614c0b5580 0x7f614c0cf710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41102/0 (socket says 172.21.15.105:41102) 2024-04-15T15:19:36.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.206+0000 7f614a59c640 1 --2- 172.21.15.105:0/629369305 >> v2:172.21.15.154:3300/0 conn(0x7f614c1554d0 0x7f614c0afaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:36.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.206+0000 7f614ad9d640 1 -- 172.21.15.105:0/629369305 >> v2:172.21.15.154:3300/0 conn(0x7f614c1554d0 msgr2=0x7f614c0afaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:36.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.206+0000 7f614ad9d640 1 --2- 172.21.15.105:0/629369305 >> v2:172.21.15.154:3300/0 conn(0x7f614c1554d0 0x7f614c0afaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:36.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.206+0000 7f614ad9d640 1 -- 172.21.15.105:0/629369305 >> v1:172.21.15.105:6789/0 conn(0x7f614c0b5580 legacy=0x7f614c0cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:36.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.206+0000 7f614ad9d640 1 -- 172.21.15.105:0/629369305 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f614c15bf70 con 0x7f614c0726f0 2024-04-15T15:19:36.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.206+0000 7f6149d9b640 1 -- 172.21.15.105:0/629369305 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2222855731 0 0) 0x7f614c0b5a30 con 0x7f614c0b5580 2024-04-15T15:19:36.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.207+0000 7f614ad9d640 1 --2- 172.21.15.105:0/629369305 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f614c0726f0 0x7f614c072af0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f614c073300 tx=0x7f614402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=280bddb625e2e4fc server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:36.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.207+0000 7f6149d9b640 1 -- 172.21.15.105:0/629369305 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f614402cb20 con 0x7f614c0726f0 2024-04-15T15:19:36.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.207+0000 7f6149d9b640 1 -- 172.21.15.105:0/629369305 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f614402ccc0 con 0x7f614c0726f0 2024-04-15T15:19:36.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.207+0000 7f6149d9b640 1 -- 172.21.15.105:0/629369305 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6144034610 con 0x7f614c0726f0 2024-04-15T15:19:36.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.207+0000 7f6150ed9640 1 -- 172.21.15.105:0/629369305 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f614c0726f0 msgr2=0x7f614c072af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:36.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.207+0000 7f6150ed9640 1 --2- 172.21.15.105:0/629369305 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f614c0726f0 0x7f614c072af0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f614c073300 tx=0x7f614402bf80 comp rx=0 tx=0).stop 2024-04-15T15:19:36.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.207+0000 7f6150ed9640 1 -- 172.21.15.105:0/629369305 shutdown_connections 2024-04-15T15:19:36.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.207+0000 7f6150ed9640 1 --2- 172.21.15.105:0/629369305 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f614c0726f0 0x7f614c072af0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:36.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.207+0000 7f6150ed9640 1 --2- 172.21.15.105:0/629369305 >> v2:172.21.15.154:3300/0 conn(0x7f614c1554d0 0x7f614c0afaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:36.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.207+0000 7f6150ed9640 1 -- 172.21.15.105:0/629369305 >> 172.21.15.105:0/629369305 conn(0x7f614c067d50 msgr2=0x7f614c06b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:36.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.207+0000 7f6150ed9640 1 -- 172.21.15.105:0/629369305 shutdown_connections 2024-04-15T15:19:36.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.207+0000 7f6150ed9640 1 -- 172.21.15.105:0/629369305 wait complete. 2024-04-15T15:19:36.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.209+0000 7f6150ed9640 1 Processor -- start 2024-04-15T15:19:36.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.209+0000 7f6150ed9640 1 -- start start 2024-04-15T15:19:36.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.210+0000 7f6150ed9640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f614c0726f0 0x7f614c0a9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:36.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.210+0000 7f6150ed9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f614c0b5580 0x7f614c148f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:36.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.210+0000 7f6150ed9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f614c1554d0 0x7f614c0ab310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:36.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.210+0000 7f6150ed9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f614c0cfe20 con 0x7f614c0b5580 2024-04-15T15:19:36.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.210+0000 7f6150ed9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f614c0cef10 con 0x7f614c1554d0 2024-04-15T15:19:36.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.210+0000 7f6150ed9640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f614c15c250 con 0x7f614c0726f0 2024-04-15T15:19:36.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.210+0000 7f6141d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f614c0b5580 0x7f614c148f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:36.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.210+0000 7f6141d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f614c0b5580 0x7f614c148f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39954/0 (socket says 172.21.15.105:39954) 2024-04-15T15:19:36.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.210+0000 7f6141d9b640 1 -- 172.21.15.105:0/1993156701 learned_addr learned my addr 172.21.15.105:0/1993156701 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:36.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.210+0000 7f614a59c640 1 --2- 172.21.15.105:0/1993156701 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f614c0726f0 0x7f614c0a9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:36.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.210+0000 7f614ad9d640 1 --2- 172.21.15.105:0/1993156701 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f614c1554d0 0x7f614c0ab310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:36.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.210+0000 7f6141d9b640 1 -- 172.21.15.105:0/1993156701 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f614c0726f0 msgr2=0x7f614c0a9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:36.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.210+0000 7f6141d9b640 1 --2- 172.21.15.105:0/1993156701 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f614c0726f0 0x7f614c0a9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:36.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.210+0000 7f6141d9b640 1 -- 172.21.15.105:0/1993156701 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f614c1554d0 msgr2=0x7f614c0ab310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:36.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.210+0000 7f6141d9b640 1 --2- 172.21.15.105:0/1993156701 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f614c1554d0 0x7f614c0ab310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:36.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.210+0000 7f6141d9b640 1 -- 172.21.15.105:0/1993156701 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f614c0ad320 con 0x7f614c0b5580 2024-04-15T15:19:36.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.210+0000 7f614a59c640 1 --2- 172.21.15.105:0/1993156701 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f614c0726f0 0x7f614c0a9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:36.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.210+0000 7f6141d9b640 1 --2- 172.21.15.105:0/1993156701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f614c0b5580 0x7f614c148f30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f613400c4d0 tx=0x7f613400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:36.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.210+0000 7f61437fe640 1 -- 172.21.15.105:0/1993156701 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6134018070 con 0x7f614c0b5580 2024-04-15T15:19:36.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.210+0000 7f61437fe640 1 -- 172.21.15.105:0/1993156701 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6134007500 con 0x7f614c0b5580 2024-04-15T15:19:36.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.210+0000 7f61437fe640 1 -- 172.21.15.105:0/1993156701 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6134004a20 con 0x7f614c0b5580 2024-04-15T15:19:36.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.210+0000 7f6150ed9640 1 -- 172.21.15.105:0/1993156701 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f614c0aa3f0 con 0x7f614c0b5580 2024-04-15T15:19:36.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.210+0000 7f6150ed9640 1 -- 172.21.15.105:0/1993156701 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f614c0aa9a0 con 0x7f614c0b5580 2024-04-15T15:19:36.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.211+0000 7f6150ed9640 1 -- 172.21.15.105:0/1993156701 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f614c068a40 con 0x7f614c0b5580 2024-04-15T15:19:36.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.212+0000 7f61437fe640 1 -- 172.21.15.105:0/1993156701 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6134004bc0 con 0x7f614c0b5580 2024-04-15T15:19:36.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.212+0000 7f61437fe640 1 --2- 172.21.15.105:0/1993156701 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f612c06c8f0 0x7f612c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:36.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.212+0000 7f61437fe640 1 -- 172.21.15.105:0/1993156701 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f613408adf0 con 0x7f614c0b5580 2024-04-15T15:19:36.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.212+0000 7f614a59c640 1 --2- 172.21.15.105:0/1993156701 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f612c06c8f0 0x7f612c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:36.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.213+0000 7f614a59c640 1 --2- 172.21.15.105:0/1993156701 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f612c06c8f0 0x7f612c06edb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f613c006fd0 tx=0x7f613c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:36.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.217+0000 7f61437fe640 1 -- 172.21.15.105:0/1993156701 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6134059220 con 0x7f614c0b5580 2024-04-15T15:19:36.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.391+0000 7f6150ed9640 1 -- 172.21.15.105:0/1993156701 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f614c149470 con 0x7f614c0b5580 2024-04-15T15:19:36.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.392+0000 7f61437fe640 1 -- 172.21.15.105:0/1993156701 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16653 (secure 0 0 0) 0x7f6134058b20 con 0x7f614c0b5580 2024-04-15T15:19:36.394 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:19:36.394 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":20,"fsid":"09e4d2a4-a324-4793-af77-644b25e520e1","created":"2024-04-15T15:19:01.524543+0000","modified":"2024-04-15T15:19:27.414890+0000","last_up_change":"2024-04-15T15:19:12.825815+0000","last_in_change":"2024-04-15T15:19:03.788126+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:19:13.748585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:19:23.789287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:19:24.722250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6f6292b-2d7e-4aa0-9d1c-6ddd0f97287c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6801","nonce":2873040054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6803","nonce":2873040054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6807","nonce":2873040054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6805","nonce":2873040054}]},"public_addr":"172.21.15.105:6801/2873040054","cluster_addr":"172.21.15.105:6803/2873040054","heartbeat_back_addr":"172.21.15.105:6807/2873040054","heartbeat_front_addr":"172.21.15.105:6805/2873040054","state":["exists","up"]},{"osd":1,"uuid":"8900966a-b393-41e3-8f93-e5a43bd23622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6818","nonce":2494994479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6821","nonce":2494994479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6828","nonce":2494994479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6823","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6825","nonce":2494994479}]},"public_addr":"172.21.15.105:6818/2494994479","cluster_addr":"172.21.15.105:6821/2494994479","heartbeat_back_addr":"172.21.15.105:6828/2494994479","heartbeat_front_addr":"172.21.15.105:6825/2494994479","state":["exists","up"]},{"osd":2,"uuid":"81a5dc7a-9325-4ff5-941d-a2d5fba21f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6809","nonce":3798408699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6811","nonce":3798408699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6815","nonce":3798408699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6813","nonce":3798408699}]},"public_addr":"172.21.15.105:6809/3798408699","cluster_addr":"172.21.15.105:6811/3798408699","heartbeat_back_addr":"172.21.15.105:6815/3798408699","heartbeat_front_addr":"172.21.15.105:6813/3798408699","state":["exists","up"]},{"osd":3,"uuid":"df766ed2-5c71-4805-96f4-3d4970edc960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6819","nonce":1920720186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6824","nonce":1920720186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6831","nonce":1920720186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6827","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6829","nonce":1920720186}]},"public_addr":"172.21.15.105:6819/1920720186","cluster_addr":"172.21.15.105:6824/1920720186","heartbeat_back_addr":"172.21.15.105:6831/1920720186","heartbeat_front_addr":"172.21.15.105:6829/1920720186","state":["exists","up"]},{"osd":4,"uuid":"e1039e03-28b8-49df-b951-c91930b93c9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6810","nonce":3516494456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6813","nonce":3516494456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6820","nonce":3516494456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6817","nonce":3516494456}]},"public_addr":"172.21.15.154:6810/3516494456","cluster_addr":"172.21.15.154:6813/3516494456","heartbeat_back_addr":"172.21.15.154:6820/3516494456","heartbeat_front_addr":"172.21.15.154:6817/3516494456","state":["exists","up"]},{"osd":5,"uuid":"a9f5eaf1-66ff-405a-a9e4-ec3340812a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6801","nonce":540367751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6803","nonce":540367751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6807","nonce":540367751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6805","nonce":540367751}]},"public_addr":"172.21.15.154:6801/540367751","cluster_addr":"172.21.15.154:6803/540367751","heartbeat_back_addr":"172.21.15.154:6807/540367751","heartbeat_front_addr":"172.21.15.154:6805/540367751","state":["exists","up"]},{"osd":6,"uuid":"267d02ea-675f-428f-aae8-e0375e4d4feb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6825","nonce":1550829005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6827","nonce":1550829005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6831","nonce":1550829005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6829","nonce":1550829005}]},"public_addr":"172.21.15.154:6825/1550829005","cluster_addr":"172.21.15.154:6827/1550829005","heartbeat_back_addr":"172.21.15.154:6831/1550829005","heartbeat_front_addr":"172.21.15.154:6829/1550829005","state":["exists","up"]},{"osd":7,"uuid":"759ac6cc-896a-423c-885f-9fe90d8dcb14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6809","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6811","nonce":3920089903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6816","nonce":3920089903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6823","nonce":3920089903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6819","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6821","nonce":3920089903}]},"public_addr":"172.21.15.154:6811/3920089903","cluster_addr":"172.21.15.154:6816/3920089903","heartbeat_back_addr":"172.21.15.154:6823/3920089903","heartbeat_front_addr":"172.21.15.154:6821/3920089903","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.949509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.984072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.907278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.947391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.975447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.998896+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:09.028739+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.894119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:19:36.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.394+0000 7f6150ed9640 1 -- 172.21.15.105:0/1993156701 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f612c06c8f0 msgr2=0x7f612c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:36.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.394+0000 7f6150ed9640 1 --2- 172.21.15.105:0/1993156701 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f612c06c8f0 0x7f612c06edb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f613c006fd0 tx=0x7f613c008040 comp rx=0 tx=0).stop 2024-04-15T15:19:36.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.394+0000 7f6150ed9640 1 -- 172.21.15.105:0/1993156701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f614c0b5580 msgr2=0x7f614c148f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:36.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.394+0000 7f6150ed9640 1 --2- 172.21.15.105:0/1993156701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f614c0b5580 0x7f614c148f30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f613400c4d0 tx=0x7f613400c990 comp rx=0 tx=0).stop 2024-04-15T15:19:36.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.394+0000 7f6150ed9640 1 -- 172.21.15.105:0/1993156701 shutdown_connections 2024-04-15T15:19:36.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.394+0000 7f6150ed9640 1 --2- 172.21.15.105:0/1993156701 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f612c06c8f0 0x7f612c06edb0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:36.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.394+0000 7f6150ed9640 1 --2- 172.21.15.105:0/1993156701 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f614c1554d0 0x7f614c0ab310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:36.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.394+0000 7f6150ed9640 1 --2- 172.21.15.105:0/1993156701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f614c0b5580 0x7f614c148f30 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:36.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.394+0000 7f6150ed9640 1 --2- 172.21.15.105:0/1993156701 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f614c0726f0 0x7f614c0a9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:36.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.394+0000 7f6150ed9640 1 -- 172.21.15.105:0/1993156701 >> 172.21.15.105:0/1993156701 conn(0x7f614c067d50 msgr2=0x7f614c0b1980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:36.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.394+0000 7f6150ed9640 1 -- 172.21.15.105:0/1993156701 shutdown_connections 2024-04-15T15:19:36.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.394+0000 7f6150ed9640 1 -- 172.21.15.105:0/1993156701 wait complete. 2024-04-15T15:19:36.407 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:19:13.748585+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-15T15:19:23.789287+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-15T15:19:24.722250+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:19:36.407 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:19:36.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.511+0000 7fcc3f3d4640 1 Processor -- start 2024-04-15T15:19:36.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.511+0000 7fcc3f3d4640 1 -- start start 2024-04-15T15:19:36.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.512+0000 7fcc3f3d4640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fcc38144c90 0x7fcc380b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:36.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.512+0000 7fcc3f3d4640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcc380cd8e0 0x7fcc3806f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:36.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.512+0000 7fcc3f3d4640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc380b40a0 con 0x7fcc3806ef90 2024-04-15T15:19:36.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.512+0000 7fcc3f3d4640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fcc3806f940 con 0x7fcc38144c90 2024-04-15T15:19:36.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.512+0000 7fcc3f3d4640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fcc380cdcf0 con 0x7fcc380cd8e0 2024-04-15T15:19:36.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.512+0000 7fcc3d971640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcc380cd8e0 0x7fcc3806f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:36.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.512+0000 7fcc3d170640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fcc38144c90 0x7fcc380b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:36.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.512+0000 7fcc3c96f640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fcc3806ef90 0x7fcc380cd1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.105:36960/0 (socket says 172.21.15.105:36960) 2024-04-15T15:19:36.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.512+0000 7fcc3d971640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcc380cd8e0 0x7fcc3806f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.105:57352/0 (socket says 172.21.15.105:57352) 2024-04-15T15:19:36.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.512+0000 7fcc3c96f640 1 -- 172.21.15.105:0/2242121278 learned_addr learned my addr 172.21.15.105:0/2242121278 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:19:36.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.513+0000 7fcc3d971640 1 -- 172.21.15.105:0/2242121278 >> v2:172.21.15.154:3300/0 conn(0x7fcc38144c90 msgr2=0x7fcc380b4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:36.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.513+0000 7fcc3d971640 1 --2- 172.21.15.105:0/2242121278 >> v2:172.21.15.154:3300/0 conn(0x7fcc38144c90 0x7fcc380b4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:36.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.513+0000 7fcc3d971640 1 -- 172.21.15.105:0/2242121278 >> v1:172.21.15.154:6789/0 conn(0x7fcc3806ef90 legacy=0x7fcc380cd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:36.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.513+0000 7fcc3d971640 1 -- 172.21.15.105:0/2242121278 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc380b4380 con 0x7fcc380cd8e0 2024-04-15T15:19:36.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.513+0000 7fcc3d170640 1 --2- 172.21.15.105:0/2242121278 >> v2:172.21.15.154:3300/0 conn(0x7fcc38144c90 0x7fcc380b4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:19:36.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.514+0000 7fcc3d971640 1 --2- 172.21.15.105:0/2242121278 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcc380cd8e0 0x7fcc3806f400 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fcc30009d90 tx=0x7fcc3002be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2e77522a7dc4ec4 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:36.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.514+0000 7fcc2ffff640 1 -- 172.21.15.105:0/2242121278 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc30039070 con 0x7fcc380cd8e0 2024-04-15T15:19:36.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.514+0000 7fcc2ffff640 1 -- 172.21.15.105:0/2242121278 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc3002cbc0 con 0x7fcc380cd8e0 2024-04-15T15:19:36.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.514+0000 7fcc2ffff640 1 -- 172.21.15.105:0/2242121278 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc30034440 con 0x7fcc380cd8e0 2024-04-15T15:19:36.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.514+0000 7fcc3f3d4640 1 -- 172.21.15.105:0/2242121278 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcc380cd8e0 msgr2=0x7fcc3806f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:36.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.514+0000 7fcc3f3d4640 1 --2- 172.21.15.105:0/2242121278 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcc380cd8e0 0x7fcc3806f400 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fcc30009d90 tx=0x7fcc3002be10 comp rx=0 tx=0).stop 2024-04-15T15:19:36.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.514+0000 7fcc3f3d4640 1 -- 172.21.15.105:0/2242121278 shutdown_connections 2024-04-15T15:19:36.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.514+0000 7fcc3f3d4640 1 --2- 172.21.15.105:0/2242121278 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcc380cd8e0 0x7fcc3806f400 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:36.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.514+0000 7fcc3f3d4640 1 --2- 172.21.15.105:0/2242121278 >> v2:172.21.15.154:3300/0 conn(0x7fcc38144c90 0x7fcc380b4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:36.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.514+0000 7fcc3f3d4640 1 -- 172.21.15.105:0/2242121278 >> 172.21.15.105:0/2242121278 conn(0x7fcc38067d50 msgr2=0x7fcc3806b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:36.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.514+0000 7fcc3f3d4640 1 -- 172.21.15.105:0/2242121278 shutdown_connections 2024-04-15T15:19:36.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.514+0000 7fcc3f3d4640 1 -- 172.21.15.105:0/2242121278 wait complete. 2024-04-15T15:19:36.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.515+0000 7fcc3f3d4640 1 Processor -- start 2024-04-15T15:19:36.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.516+0000 7fcc3f3d4640 1 -- start start 2024-04-15T15:19:36.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.516+0000 7fcc3f3d4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc3806ef90 0x7fcc38138660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:36.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.516+0000 7fcc3f3d4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc380cd8e0 0x7fcc3813ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:36.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.516+0000 7fcc3f3d4640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcc38144c90 0x7fcc381efdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:36.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.516+0000 7fcc3f3d4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcc380b4940 con 0x7fcc3806ef90 2024-04-15T15:19:36.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.516+0000 7fcc3f3d4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fcc380b3e50 con 0x7fcc380cd8e0 2024-04-15T15:19:36.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.516+0000 7fcc3f3d4640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fcc38145070 con 0x7fcc38144c90 2024-04-15T15:19:36.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.516+0000 7fcc3d170640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc3806ef90 0x7fcc38138660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:36.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.516+0000 7fcc3d170640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc3806ef90 0x7fcc38138660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39958/0 (socket says 172.21.15.105:39958) 2024-04-15T15:19:36.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.516+0000 7fcc3d170640 1 -- 172.21.15.105:0/755637491 learned_addr learned my addr 172.21.15.105:0/755637491 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:36.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.517+0000 7fcc3c96f640 1 --2- 172.21.15.105:0/755637491 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc380cd8e0 0x7fcc3813ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:36.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.517+0000 7fcc3d971640 1 --2- 172.21.15.105:0/755637491 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcc38144c90 0x7fcc381efdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:36.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.517+0000 7fcc3d170640 1 -- 172.21.15.105:0/755637491 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcc38144c90 msgr2=0x7fcc381efdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:36.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.517+0000 7fcc3d170640 1 --2- 172.21.15.105:0/755637491 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcc38144c90 0x7fcc381efdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:36.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.517+0000 7fcc3d170640 1 -- 172.21.15.105:0/755637491 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc380cd8e0 msgr2=0x7fcc3813ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:36.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.517+0000 7fcc3d170640 1 --2- 172.21.15.105:0/755637491 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc380cd8e0 0x7fcc3813ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:36.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.517+0000 7fcc3d170640 1 -- 172.21.15.105:0/755637491 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc38138ba0 con 0x7fcc3806ef90 2024-04-15T15:19:36.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.517+0000 7fcc3d170640 1 --2- 172.21.15.105:0/755637491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc3806ef90 0x7fcc38138660 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcc3400e960 tx=0x7fcc3400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:36.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.517+0000 7fcc2dffb640 1 -- 172.21.15.105:0/755637491 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc34017070 con 0x7fcc3806ef90 2024-04-15T15:19:36.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.517+0000 7fcc2dffb640 1 -- 172.21.15.105:0/755637491 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc3400ca00 con 0x7fcc3806ef90 2024-04-15T15:19:36.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.517+0000 7fcc3f3d4640 1 -- 172.21.15.105:0/755637491 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc38138e40 con 0x7fcc3806ef90 2024-04-15T15:19:36.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.517+0000 7fcc2dffb640 1 -- 172.21.15.105:0/755637491 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc3400cdd0 con 0x7fcc3806ef90 2024-04-15T15:19:36.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.517+0000 7fcc3f3d4640 1 -- 172.21.15.105:0/755637491 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc38139400 con 0x7fcc3806ef90 2024-04-15T15:19:36.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.518+0000 7fcc3f3d4640 1 -- 172.21.15.105:0/755637491 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc38068a90 con 0x7fcc3806ef90 2024-04-15T15:19:36.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.519+0000 7fcc2dffb640 1 -- 172.21.15.105:0/755637491 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcc3401a070 con 0x7fcc3806ef90 2024-04-15T15:19:36.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.519+0000 7fcc2dffb640 1 --2- 172.21.15.105:0/755637491 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fcc1c06c8f0 0x7fcc1c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:36.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.519+0000 7fcc2dffb640 1 -- 172.21.15.105:0/755637491 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fcc3408bd30 con 0x7fcc3806ef90 2024-04-15T15:19:36.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.519+0000 7fcc3c96f640 1 --2- 172.21.15.105:0/755637491 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fcc1c06c8f0 0x7fcc1c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:36.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.519+0000 7fcc3c96f640 1 --2- 172.21.15.105:0/755637491 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fcc1c06c8f0 0x7fcc1c06edb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcc38142710 tx=0x7fcc28009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:36.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.523+0000 7fcc2dffb640 1 -- 172.21.15.105:0/755637491 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcc3405a1f0 con 0x7fcc3806ef90 2024-04-15T15:19:36.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.699+0000 7fcc3f3d4640 1 -- 172.21.15.105:0/755637491 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcc3813b1c0 con 0x7fcc3806ef90 2024-04-15T15:19:36.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.699+0000 7fcc2dffb640 1 -- 172.21.15.105:0/755637491 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fcc34059af0 con 0x7fcc3806ef90 2024-04-15T15:19:36.701 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-04-15T15:19:36.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.701+0000 7fcc3f3d4640 1 -- 172.21.15.105:0/755637491 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fcc1c06c8f0 msgr2=0x7fcc1c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:36.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.701+0000 7fcc3f3d4640 1 --2- 172.21.15.105:0/755637491 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fcc1c06c8f0 0x7fcc1c06edb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcc38142710 tx=0x7fcc28009040 comp rx=0 tx=0).stop 2024-04-15T15:19:36.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.701+0000 7fcc3f3d4640 1 -- 172.21.15.105:0/755637491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc3806ef90 msgr2=0x7fcc38138660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:36.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.701+0000 7fcc3f3d4640 1 --2- 172.21.15.105:0/755637491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc3806ef90 0x7fcc38138660 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcc3400e960 tx=0x7fcc3400ee20 comp rx=0 tx=0).stop 2024-04-15T15:19:36.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.701+0000 7fcc3f3d4640 1 -- 172.21.15.105:0/755637491 shutdown_connections 2024-04-15T15:19:36.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.701+0000 7fcc3f3d4640 1 --2- 172.21.15.105:0/755637491 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcc38144c90 0x7fcc381efdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:36.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.701+0000 7fcc3f3d4640 1 --2- 172.21.15.105:0/755637491 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fcc1c06c8f0 0x7fcc1c06edb0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:36.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.701+0000 7fcc3f3d4640 1 --2- 172.21.15.105:0/755637491 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc380cd8e0 0x7fcc3813ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:36.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.701+0000 7fcc3f3d4640 1 --2- 172.21.15.105:0/755637491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc3806ef90 0x7fcc38138660 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:36.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.701+0000 7fcc3f3d4640 1 -- 172.21.15.105:0/755637491 >> 172.21.15.105:0/755637491 conn(0x7fcc38067d50 msgr2=0x7fcc380b21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:36.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.701+0000 7fcc3f3d4640 1 -- 172.21.15.105:0/755637491 shutdown_connections 2024-04-15T15:19:36.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.701+0000 7fcc3f3d4640 1 -- 172.21.15.105:0/755637491 wait complete. 2024-04-15T15:19:36.714 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-15T15:19:36.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.818+0000 7ff6a2262640 1 Processor -- start 2024-04-15T15:19:36.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.818+0000 7ff6a2262640 1 -- start start 2024-04-15T15:19:36.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.819+0000 7ff6a2262640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff69c0ce330 0x7ff69c15d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:36.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.819+0000 7ff6a2262640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff69c15dbd0 con 0x7ff69c15e320 2024-04-15T15:19:36.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.819+0000 7ff6a2262640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff69c1607a0 con 0x7ff69c160230 2024-04-15T15:19:36.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.819+0000 7ff6a2262640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7ff69c15de90 con 0x7ff69c0ce330 2024-04-15T15:19:36.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.819+0000 7ff69bfff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7ff69c15e320 0x7ff69c15e7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41114/0 (socket says 172.21.15.105:41114) 2024-04-15T15:19:36.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.819+0000 7ff69bfff640 1 -- 172.21.15.105:0/394222094 learned_addr learned my addr 172.21.15.105:0/394222094 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:19:36.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.819+0000 7ff69b7fe640 1 --2- 172.21.15.105:0/394222094 >> v2:172.21.15.154:3300/0 conn(0x7ff69c0ce330 0x7ff69c15d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:36.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.819+0000 7ff69a7fc640 1 -- 172.21.15.105:0/394222094 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3855831141 0 0) 0x7ff69c15dbd0 con 0x7ff69c15e320 2024-04-15T15:19:36.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.819+0000 7ff69a7fc640 1 -- 172.21.15.105:0/394222094 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff688003520 con 0x7ff69c15e320 2024-04-15T15:19:36.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.819+0000 7ff69b7fe640 1 -- 172.21.15.105:0/394222094 >> v1:172.21.15.154:6789/0 conn(0x7ff69c160230 legacy=0x7ff69c0ce710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:36.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.819+0000 7ff69b7fe640 1 -- 172.21.15.105:0/394222094 >> v1:172.21.15.105:6789/0 conn(0x7ff69c15e320 legacy=0x7ff69c15e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:36.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.819+0000 7ff69b7fe640 1 -- 172.21.15.105:0/394222094 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff69c160a80 con 0x7ff69c0ce330 2024-04-15T15:19:36.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.820+0000 7ff69b7fe640 1 --2- 172.21.15.105:0/394222094 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff69c0ce330 0x7ff69c15d690 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff68c0097a0 tx=0x7ff68c02b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f0aa2e6ca71aa0b server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:36.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.820+0000 7ff69a7fc640 1 -- 172.21.15.105:0/394222094 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff68c034070 con 0x7ff69c0ce330 2024-04-15T15:19:36.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.820+0000 7ff69a7fc640 1 -- 172.21.15.105:0/394222094 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff68c030610 con 0x7ff69c0ce330 2024-04-15T15:19:36.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.820+0000 7ff69a7fc640 1 -- 172.21.15.105:0/394222094 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff68c0309c0 con 0x7ff69c0ce330 2024-04-15T15:19:36.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.821+0000 7ff6a2262640 1 -- 172.21.15.105:0/394222094 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff69c0ce330 msgr2=0x7ff69c15d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:36.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.821+0000 7ff6a2262640 1 --2- 172.21.15.105:0/394222094 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff69c0ce330 0x7ff69c15d690 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff68c0097a0 tx=0x7ff68c02b960 comp rx=0 tx=0).stop 2024-04-15T15:19:36.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.821+0000 7ff6a2262640 1 -- 172.21.15.105:0/394222094 shutdown_connections 2024-04-15T15:19:36.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.821+0000 7ff6a2262640 1 --2- 172.21.15.105:0/394222094 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff69c0ce330 0x7ff69c15d690 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:36.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.821+0000 7ff6a2262640 1 -- 172.21.15.105:0/394222094 >> 172.21.15.105:0/394222094 conn(0x7ff69c06b9c0 msgr2=0x7ff69c06bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:36.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.821+0000 7ff6a2262640 1 -- 172.21.15.105:0/394222094 shutdown_connections 2024-04-15T15:19:36.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.821+0000 7ff6a2262640 1 -- 172.21.15.105:0/394222094 wait complete. 2024-04-15T15:19:36.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.822+0000 7ff6a2262640 1 Processor -- start 2024-04-15T15:19:36.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.822+0000 7ff6a2262640 1 -- start start 2024-04-15T15:19:36.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.823+0000 7ff6a2262640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff69c0ce330 0x7ff69c155f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:36.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.823+0000 7ff6a2262640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff69c15e320 0x7ff69c1318b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:36.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.823+0000 7ff6a2262640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff69c160230 0x7ff69c134060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:36.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.823+0000 7ff6a2262640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff69c156f90 con 0x7ff69c0ce330 2024-04-15T15:19:36.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.823+0000 7ff6a2262640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff69c161040 con 0x7ff69c15e320 2024-04-15T15:19:36.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.823+0000 7ff6a2262640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7ff69c157110 con 0x7ff69c160230 2024-04-15T15:19:36.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.823+0000 7ff69b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff69c0ce330 0x7ff69c155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:36.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.823+0000 7ff69b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff69c0ce330 0x7ff69c155f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39964/0 (socket says 172.21.15.105:39964) 2024-04-15T15:19:36.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.823+0000 7ff69b7fe640 1 -- 172.21.15.105:0/1760160801 learned_addr learned my addr 172.21.15.105:0/1760160801 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:36.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.823+0000 7ff69bfff640 1 --2- 172.21.15.105:0/1760160801 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff69c160230 0x7ff69c134060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:36.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.823+0000 7ff69affd640 1 --2- 172.21.15.105:0/1760160801 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff69c15e320 0x7ff69c1318b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:36.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.823+0000 7ff69b7fe640 1 -- 172.21.15.105:0/1760160801 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff69c160230 msgr2=0x7ff69c134060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:36.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.823+0000 7ff69b7fe640 1 --2- 172.21.15.105:0/1760160801 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff69c160230 0x7ff69c134060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:36.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.823+0000 7ff69b7fe640 1 -- 172.21.15.105:0/1760160801 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff69c15e320 msgr2=0x7ff69c1318b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:36.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.823+0000 7ff69b7fe640 1 --2- 172.21.15.105:0/1760160801 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff69c15e320 0x7ff69c1318b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:36.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.823+0000 7ff69b7fe640 1 -- 172.21.15.105:0/1760160801 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff69c1564b0 con 0x7ff69c0ce330 2024-04-15T15:19:36.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.823+0000 7ff69bfff640 1 --2- 172.21.15.105:0/1760160801 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff69c160230 0x7ff69c134060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:19:36.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.823+0000 7ff69b7fe640 1 --2- 172.21.15.105:0/1760160801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff69c0ce330 0x7ff69c155f70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff68c000c00 tx=0x7ff68c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:36.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.824+0000 7ff693fff640 1 -- 172.21.15.105:0/1760160801 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff68c034040 con 0x7ff69c0ce330 2024-04-15T15:19:36.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.824+0000 7ff693fff640 1 -- 172.21.15.105:0/1760160801 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff68c030410 con 0x7ff69c0ce330 2024-04-15T15:19:36.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.824+0000 7ff693fff640 1 -- 172.21.15.105:0/1760160801 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff68c004230 con 0x7ff69c0ce330 2024-04-15T15:19:36.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.824+0000 7ff6a2262640 1 -- 172.21.15.105:0/1760160801 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff69c157330 con 0x7ff69c0ce330 2024-04-15T15:19:36.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.824+0000 7ff6a2262640 1 -- 172.21.15.105:0/1760160801 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff69c1578f0 con 0x7ff69c0ce330 2024-04-15T15:19:36.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.825+0000 7ff6a2262640 1 -- 172.21.15.105:0/1760160801 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff69c068a40 con 0x7ff69c0ce330 2024-04-15T15:19:36.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.825+0000 7ff693fff640 1 -- 172.21.15.105:0/1760160801 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff68c04c060 con 0x7ff69c0ce330 2024-04-15T15:19:36.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.826+0000 7ff693fff640 1 --2- 172.21.15.105:0/1760160801 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff67c06c8f0 0x7ff67c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:36.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.826+0000 7ff693fff640 1 -- 172.21.15.105:0/1760160801 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff68c0ad550 con 0x7ff69c0ce330 2024-04-15T15:19:36.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.826+0000 7ff69affd640 1 --2- 172.21.15.105:0/1760160801 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff67c06c8f0 0x7ff67c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:36.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.826+0000 7ff69affd640 1 --2- 172.21.15.105:0/1760160801 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff67c06c8f0 0x7ff67c06edb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff684007fd0 tx=0x7ff684009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:36.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:36.830+0000 7ff693fff640 1 -- 172.21.15.105:0/1760160801 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff68c07ba10 con 0x7ff69c0ce330 2024-04-15T15:19:37.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.005+0000 7ff6a2262640 1 -- 172.21.15.105:0/1760160801 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff69c15d7c0 con 0x7ff69c0ce330 2024-04-15T15:19:37.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.005+0000 7ff693fff640 1 -- 172.21.15.105:0/1760160801 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7ff68c0305e0 con 0x7ff69c0ce330 2024-04-15T15:19:37.007 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-04-15T15:19:37.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.007+0000 7ff6a2262640 1 -- 172.21.15.105:0/1760160801 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff67c06c8f0 msgr2=0x7ff67c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:37.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.007+0000 7ff6a2262640 1 --2- 172.21.15.105:0/1760160801 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff67c06c8f0 0x7ff67c06edb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff684007fd0 tx=0x7ff684009040 comp rx=0 tx=0).stop 2024-04-15T15:19:37.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.007+0000 7ff6a2262640 1 -- 172.21.15.105:0/1760160801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff69c0ce330 msgr2=0x7ff69c155f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:37.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.007+0000 7ff6a2262640 1 --2- 172.21.15.105:0/1760160801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff69c0ce330 0x7ff69c155f70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff68c000c00 tx=0x7ff68c0046c0 comp rx=0 tx=0).stop 2024-04-15T15:19:37.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.007+0000 7ff6a2262640 1 -- 172.21.15.105:0/1760160801 shutdown_connections 2024-04-15T15:19:37.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.007+0000 7ff6a2262640 1 --2- 172.21.15.105:0/1760160801 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff69c160230 0x7ff69c134060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:37.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.007+0000 7ff6a2262640 1 --2- 172.21.15.105:0/1760160801 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff67c06c8f0 0x7ff67c06edb0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:37.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.007+0000 7ff6a2262640 1 --2- 172.21.15.105:0/1760160801 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff69c15e320 0x7ff69c1318b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:37.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.007+0000 7ff6a2262640 1 --2- 172.21.15.105:0/1760160801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff69c0ce330 0x7ff69c155f70 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:37.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.007+0000 7ff6a2262640 1 -- 172.21.15.105:0/1760160801 >> 172.21.15.105:0/1760160801 conn(0x7ff69c06b9c0 msgr2=0x7ff69c15cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:37.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.007+0000 7ff6a2262640 1 -- 172.21.15.105:0/1760160801 shutdown_connections 2024-04-15T15:19:37.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.007+0000 7ff6a2262640 1 -- 172.21.15.105:0/1760160801 wait complete. 2024-04-15T15:19:37.020 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-15T15:19:37.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.125+0000 7ffa3e7a8640 1 Processor -- start 2024-04-15T15:19:37.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.125+0000 7ffa3e7a8640 1 -- start start 2024-04-15T15:19:37.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.126+0000 7ffa3e7a8640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ffa3815e8d0 0x7ffa380d2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:37.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.126+0000 7ffa3e7a8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffa380ba480 0x7ffa380d1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:37.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.126+0000 7ffa3e7a8640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ffa380d1650 0x7ffa380b61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:37.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.126+0000 7ffa3e7a8640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7ffa380d1ac0 con 0x7ffa380d1650 2024-04-15T15:19:37.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.126+0000 7ffa3e7a8640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7ffa380d3400 con 0x7ffa3815e8d0 2024-04-15T15:19:37.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.126+0000 7ffa3e7a8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7ffa380d36e0 con 0x7ffa380ba480 2024-04-15T15:19:37.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.126+0000 7ffa3cd45640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ffa380d1650 0x7ffa380b61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:37.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.126+0000 7ffa3cd45640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ffa380d1650 0x7ffa380b61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39966/0 (socket says 172.21.15.105:39966) 2024-04-15T15:19:37.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.126+0000 7ffa3cd45640 1 -- 172.21.15.105:0/3316838480 learned_addr learned my addr 172.21.15.105:0/3316838480 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:37.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.126+0000 7ffa2ffff640 1 --2- 172.21.15.105:0/3316838480 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffa380ba480 0x7ffa380d1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:37.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.126+0000 7ffa37fff640 1 --2- 172.21.15.105:0/3316838480 >> v2:172.21.15.154:3300/0 conn(0x7ffa3815e8d0 0x7ffa380d2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:37.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.126+0000 7ffa3cd45640 1 -- 172.21.15.105:0/3316838480 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffa380ba480 msgr2=0x7ffa380d1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:37.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.126+0000 7ffa3cd45640 1 --2- 172.21.15.105:0/3316838480 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffa380ba480 0x7ffa380d1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:37.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.126+0000 7ffa3cd45640 1 -- 172.21.15.105:0/3316838480 >> v2:172.21.15.154:3300/0 conn(0x7ffa3815e8d0 msgr2=0x7ffa380d2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:37.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.126+0000 7ffa3cd45640 1 --2- 172.21.15.105:0/3316838480 >> v2:172.21.15.154:3300/0 conn(0x7ffa3815e8d0 0x7ffa380d2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:37.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.126+0000 7ffa3cd45640 1 -- 172.21.15.105:0/3316838480 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa380d39c0 con 0x7ffa380d1650 2024-04-15T15:19:37.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.126+0000 7ffa2ffff640 1 --2- 172.21.15.105:0/3316838480 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffa380ba480 0x7ffa380d1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:37.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.127+0000 7ffa3cd45640 1 --2- 172.21.15.105:0/3316838480 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa380d1650 0x7ffa380b61d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ffa380b7310 tx=0x7ffa3002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6057de64d035619e server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:37.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.127+0000 7ffa377fe640 1 -- 172.21.15.105:0/3316838480 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3002cb00 con 0x7ffa380d1650 2024-04-15T15:19:37.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.127+0000 7ffa377fe640 1 -- 172.21.15.105:0/3316838480 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa3002cca0 con 0x7ffa380d1650 2024-04-15T15:19:37.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.127+0000 7ffa377fe640 1 -- 172.21.15.105:0/3316838480 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa30034610 con 0x7ffa380d1650 2024-04-15T15:19:37.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.127+0000 7ffa3e7a8640 1 -- 172.21.15.105:0/3316838480 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa380d1650 msgr2=0x7ffa380b61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:37.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.127+0000 7ffa3e7a8640 1 --2- 172.21.15.105:0/3316838480 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa380d1650 0x7ffa380b61d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ffa380b7310 tx=0x7ffa3002bf60 comp rx=0 tx=0).stop 2024-04-15T15:19:37.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.128+0000 7ffa3e7a8640 1 -- 172.21.15.105:0/3316838480 shutdown_connections 2024-04-15T15:19:37.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.128+0000 7ffa3e7a8640 1 --2- 172.21.15.105:0/3316838480 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa380d1650 0x7ffa380b61d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:37.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.128+0000 7ffa3e7a8640 1 --2- 172.21.15.105:0/3316838480 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffa380ba480 0x7ffa380d1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:37.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.128+0000 7ffa3e7a8640 1 --2- 172.21.15.105:0/3316838480 >> v2:172.21.15.154:3300/0 conn(0x7ffa3815e8d0 0x7ffa380d2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:37.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.128+0000 7ffa3e7a8640 1 -- 172.21.15.105:0/3316838480 >> 172.21.15.105:0/3316838480 conn(0x7ffa3806d460 msgr2=0x7ffa38067720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:37.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.128+0000 7ffa3e7a8640 1 -- 172.21.15.105:0/3316838480 shutdown_connections 2024-04-15T15:19:37.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.128+0000 7ffa3e7a8640 1 -- 172.21.15.105:0/3316838480 wait complete. 2024-04-15T15:19:37.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.129+0000 7ffa3e7a8640 1 Processor -- start 2024-04-15T15:19:37.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.129+0000 7ffa3e7a8640 1 -- start start 2024-04-15T15:19:37.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.129+0000 7ffa3e7a8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffa380ba480 0x7ffa381f3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:37.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.129+0000 7ffa3e7a8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa380d1650 0x7ffa38152b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:37.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.129+0000 7ffa3e7a8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffa3815e8d0 0x7ffa38160f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:37.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.129+0000 7ffa3e7a8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ffa380d44a0 con 0x7ffa380d1650 2024-04-15T15:19:37.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.129+0000 7ffa3e7a8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ffa380d3f80 con 0x7ffa380ba480 2024-04-15T15:19:37.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.129+0000 7ffa3e7a8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7ffa380d2ff0 con 0x7ffa3815e8d0 2024-04-15T15:19:37.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.130+0000 7ffa2ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa380d1650 0x7ffa38152b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:37.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.130+0000 7ffa2ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa380d1650 0x7ffa38152b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39980/0 (socket says 172.21.15.105:39980) 2024-04-15T15:19:37.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.130+0000 7ffa2ffff640 1 -- 172.21.15.105:0/2849416571 learned_addr learned my addr 172.21.15.105:0/2849416571 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:37.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.130+0000 7ffa37fff640 1 --2- 172.21.15.105:0/2849416571 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffa380ba480 0x7ffa381f3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:37.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.130+0000 7ffa3cd45640 1 --2- 172.21.15.105:0/2849416571 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffa3815e8d0 0x7ffa38160f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:37.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.130+0000 7ffa2ffff640 1 -- 172.21.15.105:0/2849416571 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffa3815e8d0 msgr2=0x7ffa38160f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:37.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.130+0000 7ffa2ffff640 1 --2- 172.21.15.105:0/2849416571 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffa3815e8d0 0x7ffa38160f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:37.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.130+0000 7ffa2ffff640 1 -- 172.21.15.105:0/2849416571 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffa380ba480 msgr2=0x7ffa381f3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:37.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.130+0000 7ffa2ffff640 1 --2- 172.21.15.105:0/2849416571 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffa380ba480 0x7ffa381f3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:37.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.130+0000 7ffa2ffff640 1 -- 172.21.15.105:0/2849416571 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa380cc180 con 0x7ffa380d1650 2024-04-15T15:19:37.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.130+0000 7ffa3cd45640 1 --2- 172.21.15.105:0/2849416571 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffa3815e8d0 0x7ffa38160f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:19:37.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.130+0000 7ffa37fff640 1 --2- 172.21.15.105:0/2849416571 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffa380ba480 0x7ffa381f3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:19:37.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.130+0000 7ffa2ffff640 1 --2- 172.21.15.105:0/2849416571 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa380d1650 0x7ffa38152b40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ffa2000c7f0 tx=0x7ffa2000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:37.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.130+0000 7ffa357fa640 1 -- 172.21.15.105:0/2849416571 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa20004230 con 0x7ffa380d1650 2024-04-15T15:19:37.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.130+0000 7ffa357fa640 1 -- 172.21.15.105:0/2849416571 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa200043f0 con 0x7ffa380d1650 2024-04-15T15:19:37.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.130+0000 7ffa3e7a8640 1 -- 172.21.15.105:0/2849416571 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa380cc460 con 0x7ffa380d1650 2024-04-15T15:19:37.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.130+0000 7ffa357fa640 1 -- 172.21.15.105:0/2849416571 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa20007860 con 0x7ffa380d1650 2024-04-15T15:19:37.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.130+0000 7ffa3e7a8640 1 -- 172.21.15.105:0/2849416571 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa380cca20 con 0x7ffa380d1650 2024-04-15T15:19:37.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.131+0000 7ffa3e7a8640 1 -- 172.21.15.105:0/2849416571 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa38068ab0 con 0x7ffa380d1650 2024-04-15T15:19:37.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.132+0000 7ffa357fa640 1 -- 172.21.15.105:0/2849416571 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffa20007c10 con 0x7ffa380d1650 2024-04-15T15:19:37.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.132+0000 7ffa357fa640 1 --2- 172.21.15.105:0/2849416571 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ffa1806c8f0 0x7ffa1806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:37.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.132+0000 7ffa37fff640 1 --2- 172.21.15.105:0/2849416571 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ffa1806c8f0 0x7ffa1806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:37.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.132+0000 7ffa357fa640 1 -- 172.21.15.105:0/2849416571 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ffa2008be30 con 0x7ffa380d1650 2024-04-15T15:19:37.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.132+0000 7ffa37fff640 1 --2- 172.21.15.105:0/2849416571 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ffa1806c8f0 0x7ffa1806edb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ffa28006fd0 tx=0x7ffa28008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:37.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.136+0000 7ffa357fa640 1 -- 172.21.15.105:0/2849416571 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ffa20015020 con 0x7ffa380d1650 2024-04-15T15:19:37.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.313+0000 7ffa3e7a8640 1 -- 172.21.15.105:0/2849416571 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ffa380ccd60 con 0x7ffa380d1650 2024-04-15T15:19:37.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.314+0000 7ffa357fa640 1 -- 172.21.15.105:0/2849416571 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7ffa2005a1d0 con 0x7ffa380d1650 2024-04-15T15:19:37.316 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-04-15T15:19:37.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.316+0000 7ffa3e7a8640 1 -- 172.21.15.105:0/2849416571 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ffa1806c8f0 msgr2=0x7ffa1806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:37.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.316+0000 7ffa3e7a8640 1 --2- 172.21.15.105:0/2849416571 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ffa1806c8f0 0x7ffa1806edb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ffa28006fd0 tx=0x7ffa28008040 comp rx=0 tx=0).stop 2024-04-15T15:19:37.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.316+0000 7ffa3e7a8640 1 -- 172.21.15.105:0/2849416571 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa380d1650 msgr2=0x7ffa38152b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:37.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.316+0000 7ffa3e7a8640 1 --2- 172.21.15.105:0/2849416571 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa380d1650 0x7ffa38152b40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ffa2000c7f0 tx=0x7ffa2000ccb0 comp rx=0 tx=0).stop 2024-04-15T15:19:37.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.316+0000 7ffa3e7a8640 1 -- 172.21.15.105:0/2849416571 shutdown_connections 2024-04-15T15:19:37.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.316+0000 7ffa3e7a8640 1 --2- 172.21.15.105:0/2849416571 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffa3815e8d0 0x7ffa38160f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:37.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.316+0000 7ffa3e7a8640 1 --2- 172.21.15.105:0/2849416571 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ffa1806c8f0 0x7ffa1806edb0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:37.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.316+0000 7ffa3e7a8640 1 --2- 172.21.15.105:0/2849416571 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa380d1650 0x7ffa38152b40 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:37.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.316+0000 7ffa3e7a8640 1 --2- 172.21.15.105:0/2849416571 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffa380ba480 0x7ffa381f3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:37.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.316+0000 7ffa3e7a8640 1 -- 172.21.15.105:0/2849416571 >> 172.21.15.105:0/2849416571 conn(0x7ffa3806d460 msgr2=0x7ffa380c9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:37.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.316+0000 7ffa3e7a8640 1 -- 172.21.15.105:0/2849416571 shutdown_connections 2024-04-15T15:19:37.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.316+0000 7ffa3e7a8640 1 -- 172.21.15.105:0/2849416571 wait complete. 2024-04-15T15:19:37.329 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-15T15:19:37.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.429+0000 7f4d10263640 1 Processor -- start 2024-04-15T15:19:37.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.430+0000 7f4d10263640 1 -- start start 2024-04-15T15:19:37.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.430+0000 7f4d10263640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4d0813e680 0x7f4d080cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:37.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.430+0000 7f4d10263640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4d0813ec60 0x7f4d0806ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:37.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.430+0000 7f4d10263640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d0815bc90 con 0x7f4d0806f4d0 2024-04-15T15:19:37.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.430+0000 7f4d10263640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f4d0815bf70 con 0x7f4d0813e680 2024-04-15T15:19:37.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.430+0000 7f4d10263640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f4d0815c250 con 0x7f4d0813ec60 2024-04-15T15:19:37.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.431+0000 7f4d0dfff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4d0813e680 0x7f4d080cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:37.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.431+0000 7f4d0dfff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4d0813e680 0x7f4d080cfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39982/0 (socket says 172.21.15.105:39982) 2024-04-15T15:19:37.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.431+0000 7f4d0dfff640 1 -- 172.21.15.105:0/3289020771 learned_addr learned my addr 172.21.15.105:0/3289020771 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:37.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.431+0000 7f4d0d7fe640 1 --2- 172.21.15.105:0/3289020771 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4d0813ec60 0x7f4d0806ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:37.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.431+0000 7f4d0dfff640 1 -- 172.21.15.105:0/3289020771 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4d0813ec60 msgr2=0x7f4d0806ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:37.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.431+0000 7f4d0dfff640 1 --2- 172.21.15.105:0/3289020771 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4d0813ec60 0x7f4d0806ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:37.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.431+0000 7f4d0dfff640 1 -- 172.21.15.105:0/3289020771 >> v1:172.21.15.154:6789/0 conn(0x7f4d0806f4d0 legacy=0x7f4d08071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:37.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.431+0000 7f4d0dfff640 1 -- 172.21.15.105:0/3289020771 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d0815c530 con 0x7f4d0813e680 2024-04-15T15:19:37.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.432+0000 7f4d0dfff640 1 --2- 172.21.15.105:0/3289020771 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d0813e680 0x7f4d080cfb70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4d04009e50 tx=0x7f4d0402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65e710cfb266bb40 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:37.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.432+0000 7f4d0cffd640 1 -- 172.21.15.105:0/3289020771 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d0402cb00 con 0x7f4d0813e680 2024-04-15T15:19:37.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.432+0000 7f4d0cffd640 1 -- 172.21.15.105:0/3289020771 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d0402cca0 con 0x7f4d0813e680 2024-04-15T15:19:37.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.432+0000 7f4d0cffd640 1 -- 172.21.15.105:0/3289020771 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d04034610 con 0x7f4d0813e680 2024-04-15T15:19:37.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.432+0000 7f4d10263640 1 -- 172.21.15.105:0/3289020771 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d0813e680 msgr2=0x7f4d080cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:37.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.432+0000 7f4d10263640 1 --2- 172.21.15.105:0/3289020771 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d0813e680 0x7f4d080cfb70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4d04009e50 tx=0x7f4d0402bf60 comp rx=0 tx=0).stop 2024-04-15T15:19:37.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.432+0000 7f4d10263640 1 -- 172.21.15.105:0/3289020771 shutdown_connections 2024-04-15T15:19:37.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.432+0000 7f4d10263640 1 --2- 172.21.15.105:0/3289020771 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4d0813ec60 0x7f4d0806ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:37.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.432+0000 7f4d10263640 1 --2- 172.21.15.105:0/3289020771 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d0813e680 0x7f4d080cfb70 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:37.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.432+0000 7f4d10263640 1 -- 172.21.15.105:0/3289020771 >> 172.21.15.105:0/3289020771 conn(0x7f4d08067d50 msgr2=0x7f4d0806b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:37.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.433+0000 7f4d10263640 1 -- 172.21.15.105:0/3289020771 shutdown_connections 2024-04-15T15:19:37.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.433+0000 7f4d10263640 1 -- 172.21.15.105:0/3289020771 wait complete. 2024-04-15T15:19:37.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.434+0000 7f4d10263640 1 Processor -- start 2024-04-15T15:19:37.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.434+0000 7f4d10263640 1 -- start start 2024-04-15T15:19:37.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.434+0000 7f4d10263640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4d0806f4d0 0x7f4d081e4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:37.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.434+0000 7f4d10263640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d0813e680 0x7f4d081f10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:37.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.434+0000 7f4d10263640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d0813ec60 0x7f4d081e5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:37.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.434+0000 7f4d10263640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4d080d0230 con 0x7f4d0813ec60 2024-04-15T15:19:37.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.434+0000 7f4d10263640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4d080d00b0 con 0x7f4d0813e680 2024-04-15T15:19:37.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.434+0000 7f4d10263640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f4d080cf590 con 0x7f4d0806f4d0 2024-04-15T15:19:37.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.434+0000 7f4d0e800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d0813ec60 0x7f4d081e5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:37.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.435+0000 7f4d0e800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d0813ec60 0x7f4d081e5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39998/0 (socket says 172.21.15.105:39998) 2024-04-15T15:19:37.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.435+0000 7f4d0e800640 1 -- 172.21.15.105:0/4220673595 learned_addr learned my addr 172.21.15.105:0/4220673595 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:37.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.435+0000 7f4d0d7fe640 1 --2- 172.21.15.105:0/4220673595 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d0813e680 0x7f4d081f10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:37.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.435+0000 7f4d0dfff640 1 --2- 172.21.15.105:0/4220673595 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4d0806f4d0 0x7f4d081e4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:37.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.435+0000 7f4d0e800640 1 -- 172.21.15.105:0/4220673595 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4d0806f4d0 msgr2=0x7f4d081e4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:37.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.435+0000 7f4d0e800640 1 --2- 172.21.15.105:0/4220673595 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4d0806f4d0 0x7f4d081e4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:37.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.435+0000 7f4d0e800640 1 -- 172.21.15.105:0/4220673595 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d0813e680 msgr2=0x7f4d081f10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:37.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.435+0000 7f4d0e800640 1 --2- 172.21.15.105:0/4220673595 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d0813e680 0x7f4d081f10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:37.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.435+0000 7f4d0e800640 1 -- 172.21.15.105:0/4220673595 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d081e50f0 con 0x7f4d0813ec60 2024-04-15T15:19:37.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.435+0000 7f4d0e800640 1 --2- 172.21.15.105:0/4220673595 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d0813ec60 0x7f4d081e5540 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4d00001eb0 tx=0x7f4d0000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:37.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.435+0000 7f4cfe7fc640 1 -- 172.21.15.105:0/4220673595 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d00017070 con 0x7f4d0813ec60 2024-04-15T15:19:37.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.435+0000 7f4cfe7fc640 1 -- 172.21.15.105:0/4220673595 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d000052d0 con 0x7f4d0813ec60 2024-04-15T15:19:37.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.435+0000 7f4d10263640 1 -- 172.21.15.105:0/4220673595 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d08143840 con 0x7f4d0813ec60 2024-04-15T15:19:37.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.435+0000 7f4cfe7fc640 1 -- 172.21.15.105:0/4220673595 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d00003810 con 0x7f4d0813ec60 2024-04-15T15:19:37.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.435+0000 7f4d10263640 1 -- 172.21.15.105:0/4220673595 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d08143e00 con 0x7f4d0813ec60 2024-04-15T15:19:37.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.436+0000 7f4d10263640 1 -- 172.21.15.105:0/4220673595 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d08068a40 con 0x7f4d0813ec60 2024-04-15T15:19:37.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.437+0000 7f4cfe7fc640 1 -- 172.21.15.105:0/4220673595 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4d0001a070 con 0x7f4d0813ec60 2024-04-15T15:19:37.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.437+0000 7f4cfe7fc640 1 --2- 172.21.15.105:0/4220673595 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f4cec06c8f0 0x7f4cec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:37.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.437+0000 7f4d0dfff640 1 --2- 172.21.15.105:0/4220673595 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f4cec06c8f0 0x7f4cec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:37.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.437+0000 7f4cfe7fc640 1 -- 172.21.15.105:0/4220673595 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4d0008ae20 con 0x7f4d0813ec60 2024-04-15T15:19:37.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.437+0000 7f4d0dfff640 1 --2- 172.21.15.105:0/4220673595 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f4cec06c8f0 0x7f4cec06edb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4d04035000 tx=0x7f4d04030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:37.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.441+0000 7f4cfe7fc640 1 -- 172.21.15.105:0/4220673595 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4d00058c00 con 0x7f4d0813ec60 2024-04-15T15:19:37.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.652+0000 7f4d10263640 1 -- 172.21.15.105:0/4220673595 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4d081417b0 con 0x7f4d0813ec60 2024-04-15T15:19:37.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.653+0000 7f4cfe7fc640 1 -- 172.21.15.105:0/4220673595 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5187 (secure 0 0 0) 0x7f4d0005cf00 con 0x7f4d0813ec60 2024-04-15T15:19:37.655 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:19:37.655 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4259,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/3319657830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":3319657830},{"type":"v1","addr":"172.21.15.154:6835","nonce":3319657830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2677199890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2677199890},{"type":"v1","addr":"172.21.15.105:6835","nonce":2677199890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2867739743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2867739743},{"type":"v1","addr":"172.21.15.105:6837","nonce":2867739743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-15T15:19:25.760362+0000","modified":"2024-04-15T15:19:27.857599+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.154:6834/1529675504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1529675504},{"type":"v1","addr":"172.21.15.154:6834","nonce":1529675504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4291,"qdb_cluster":[4291]},"id":1}]} 2024-04-15T15:19:37.655 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 6 2024-04-15T15:19:37.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.655+0000 7f4d10263640 1 -- 172.21.15.105:0/4220673595 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f4cec06c8f0 msgr2=0x7f4cec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:37.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.655+0000 7f4d10263640 1 --2- 172.21.15.105:0/4220673595 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f4cec06c8f0 0x7f4cec06edb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4d04035000 tx=0x7f4d04030000 comp rx=0 tx=0).stop 2024-04-15T15:19:37.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.655+0000 7f4d10263640 1 -- 172.21.15.105:0/4220673595 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d0813ec60 msgr2=0x7f4d081e5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:37.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.655+0000 7f4d10263640 1 --2- 172.21.15.105:0/4220673595 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d0813ec60 0x7f4d081e5540 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4d00001eb0 tx=0x7f4d0000c890 comp rx=0 tx=0).stop 2024-04-15T15:19:37.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.655+0000 7f4d10263640 1 -- 172.21.15.105:0/4220673595 shutdown_connections 2024-04-15T15:19:37.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.655+0000 7f4d10263640 1 --2- 172.21.15.105:0/4220673595 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f4cec06c8f0 0x7f4cec06edb0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:37.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.655+0000 7f4d10263640 1 --2- 172.21.15.105:0/4220673595 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d0813ec60 0x7f4d081e5540 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:37.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.655+0000 7f4d10263640 1 --2- 172.21.15.105:0/4220673595 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d0813e680 0x7f4d081f10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:37.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.655+0000 7f4d10263640 1 --2- 172.21.15.105:0/4220673595 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4d0806f4d0 0x7f4d081e4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:37.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.655+0000 7f4d10263640 1 -- 172.21.15.105:0/4220673595 >> 172.21.15.105:0/4220673595 conn(0x7f4d08067d50 msgr2=0x7f4d0815e750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:37.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.655+0000 7f4d10263640 1 -- 172.21.15.105:0/4220673595 shutdown_connections 2024-04-15T15:19:37.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.655+0000 7f4d10263640 1 -- 172.21.15.105:0/4220673595 wait complete. 2024-04-15T15:19:37.668 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:19:37.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.769+0000 7f026fc74640 1 Processor -- start 2024-04-15T15:19:37.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.769+0000 7f026fc74640 1 -- start start 2024-04-15T15:19:37.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.770+0000 7f026fc74640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f026815c7b0 0x7f02680d09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:37.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.770+0000 7f026fc74640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f02680d0f30 0x7f02680b25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:37.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.770+0000 7f026fc74640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02680ccf40 con 0x7f02680b2ae0 2024-04-15T15:19:37.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.770+0000 7f026fc74640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f02680b32d0 con 0x7f02680d0f30 2024-04-15T15:19:37.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.770+0000 7f026fc74640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f02680b35b0 con 0x7f026815c7b0 2024-04-15T15:19:37.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.770+0000 7f026d20f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f02680d0f30 0x7f02680b25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:37.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.770+0000 7f026d20f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f02680d0f30 0x7f02680b25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:40014/0 (socket says 172.21.15.105:40014) 2024-04-15T15:19:37.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.770+0000 7f026d20f640 1 -- 172.21.15.105:0/3458387468 learned_addr learned my addr 172.21.15.105:0/3458387468 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:37.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.771+0000 7f026da10640 1 --2- 172.21.15.105:0/3458387468 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f026815c7b0 0x7f02680d09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:37.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.771+0000 7f026d20f640 1 -- 172.21.15.105:0/3458387468 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f026815c7b0 msgr2=0x7f02680d09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:37.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.771+0000 7f026d20f640 1 --2- 172.21.15.105:0/3458387468 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f026815c7b0 0x7f02680d09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:37.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.771+0000 7f026d20f640 1 -- 172.21.15.105:0/3458387468 >> v1:172.21.15.154:6789/0 conn(0x7f02680b2ae0 legacy=0x7f0268160410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:37.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.771+0000 7f026d20f640 1 -- 172.21.15.105:0/3458387468 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02680b3890 con 0x7f02680d0f30 2024-04-15T15:19:37.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.771+0000 7f026d20f640 1 --2- 172.21.15.105:0/3458387468 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02680d0f30 0x7f02680b25a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f025c009870 tx=0x7f025c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19e9d86ca758b2be server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:37.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.772+0000 7f026ca0e640 1 -- 172.21.15.105:0/3458387468 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f025c034070 con 0x7f02680d0f30 2024-04-15T15:19:37.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.772+0000 7f026ca0e640 1 -- 172.21.15.105:0/3458387468 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f025c030840 con 0x7f02680d0f30 2024-04-15T15:19:37.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.772+0000 7f026ca0e640 1 -- 172.21.15.105:0/3458387468 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f025c030c10 con 0x7f02680d0f30 2024-04-15T15:19:37.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.772+0000 7f026fc74640 1 -- 172.21.15.105:0/3458387468 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02680d0f30 msgr2=0x7f02680b25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:37.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.772+0000 7f026fc74640 1 --2- 172.21.15.105:0/3458387468 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02680d0f30 0x7f02680b25a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f025c009870 tx=0x7f025c02b980 comp rx=0 tx=0).stop 2024-04-15T15:19:37.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.772+0000 7f026fc74640 1 -- 172.21.15.105:0/3458387468 shutdown_connections 2024-04-15T15:19:37.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.772+0000 7f026fc74640 1 --2- 172.21.15.105:0/3458387468 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02680d0f30 0x7f02680b25a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:37.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.772+0000 7f026fc74640 1 --2- 172.21.15.105:0/3458387468 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f026815c7b0 0x7f02680d09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:37.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.772+0000 7f026fc74640 1 -- 172.21.15.105:0/3458387468 >> 172.21.15.105:0/3458387468 conn(0x7f02680d25a0 msgr2=0x7f02680d29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:37.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.772+0000 7f026fc74640 1 -- 172.21.15.105:0/3458387468 shutdown_connections 2024-04-15T15:19:37.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.772+0000 7f026fc74640 1 -- 172.21.15.105:0/3458387468 wait complete. 2024-04-15T15:19:37.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.773+0000 7f026fc74640 1 Processor -- start 2024-04-15T15:19:37.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.773+0000 7f026fc74640 1 -- start start 2024-04-15T15:19:37.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.774+0000 7f026fc74640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f02680b2ae0 0x7f0268153c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:37.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.774+0000 7f026fc74640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02680d0f30 0x7f026815d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:37.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.774+0000 7f026fc74640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f026815c7b0 0x7f026815db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:37.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.774+0000 7f026fc74640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f026815cb90 con 0x7f02680d0f30 2024-04-15T15:19:37.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.774+0000 7f026fc74640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f02680b2f70 con 0x7f026815c7b0 2024-04-15T15:19:37.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.774+0000 7f026fc74640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f026815cd10 con 0x7f02680b2ae0 2024-04-15T15:19:37.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.774+0000 7f026d20f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02680d0f30 0x7f026815d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:37.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.774+0000 7f026d20f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02680d0f30 0x7f026815d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:40022/0 (socket says 172.21.15.105:40022) 2024-04-15T15:19:37.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.774+0000 7f026d20f640 1 -- 172.21.15.105:0/1073254849 learned_addr learned my addr 172.21.15.105:0/1073254849 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:37.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.774+0000 7f026da10640 1 --2- 172.21.15.105:0/1073254849 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f02680b2ae0 0x7f0268153c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:37.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.774+0000 7f026e211640 1 --2- 172.21.15.105:0/1073254849 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f026815c7b0 0x7f026815db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:37.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.774+0000 7f026d20f640 1 -- 172.21.15.105:0/1073254849 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f02680b2ae0 msgr2=0x7f0268153c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:37.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.774+0000 7f026d20f640 1 --2- 172.21.15.105:0/1073254849 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f02680b2ae0 0x7f0268153c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:37.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.774+0000 7f026d20f640 1 -- 172.21.15.105:0/1073254849 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f026815c7b0 msgr2=0x7f026815db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:37.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.774+0000 7f026d20f640 1 --2- 172.21.15.105:0/1073254849 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f026815c7b0 0x7f026815db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:37.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.774+0000 7f026d20f640 1 -- 172.21.15.105:0/1073254849 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0268159250 con 0x7f02680d0f30 2024-04-15T15:19:37.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.775+0000 7f026d20f640 1 --2- 172.21.15.105:0/1073254849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02680d0f30 0x7f026815d650 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f025c000c00 tx=0x7f025c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:37.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.775+0000 7f02667fc640 1 -- 172.21.15.105:0/1073254849 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f025c048070 con 0x7f02680d0f30 2024-04-15T15:19:37.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.775+0000 7f02667fc640 1 -- 172.21.15.105:0/1073254849 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f025c030640 con 0x7f02680d0f30 2024-04-15T15:19:37.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.775+0000 7f026fc74640 1 -- 172.21.15.105:0/1073254849 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0268159530 con 0x7f02680d0f30 2024-04-15T15:19:37.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.775+0000 7f02667fc640 1 -- 172.21.15.105:0/1073254849 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f025c034040 con 0x7f02680d0f30 2024-04-15T15:19:37.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.775+0000 7f026fc74640 1 -- 172.21.15.105:0/1073254849 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0268159af0 con 0x7f02680d0f30 2024-04-15T15:19:37.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.776+0000 7f026fc74640 1 -- 172.21.15.105:0/1073254849 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0268069150 con 0x7f02680d0f30 2024-04-15T15:19:37.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.776+0000 7f02667fc640 1 -- 172.21.15.105:0/1073254849 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f025c04b070 con 0x7f02680d0f30 2024-04-15T15:19:37.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.777+0000 7f02667fc640 1 --2- 172.21.15.105:0/1073254849 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f025406c920 0x7f025406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:37.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.777+0000 7f026da10640 1 --2- 172.21.15.105:0/1073254849 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f025406c920 0x7f025406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:37.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.777+0000 7f02667fc640 1 -- 172.21.15.105:0/1073254849 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f025c0ada50 con 0x7f02680d0f30 2024-04-15T15:19:37.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.777+0000 7f026da10640 1 --2- 172.21.15.105:0/1073254849 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f025406c920 0x7f025406ede0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0258009f60 tx=0x7f025800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:37.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.781+0000 7f02667fc640 1 -- 172.21.15.105:0/1073254849 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f025c07be80 con 0x7f02680d0f30 2024-04-15T15:19:37.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.957+0000 7f026fc74640 1 -- 172.21.15.105:0/1073254849 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f02680d1ba0 con 0x7f02680d0f30 2024-04-15T15:19:37.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.958+0000 7f02667fc640 1 -- 172.21.15.105:0/1073254849 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16653 (secure 0 0 0) 0x7f025c07b780 con 0x7f02680d0f30 2024-04-15T15:19:37.960 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:19:37.960 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":20,"fsid":"09e4d2a4-a324-4793-af77-644b25e520e1","created":"2024-04-15T15:19:01.524543+0000","modified":"2024-04-15T15:19:27.414890+0000","last_up_change":"2024-04-15T15:19:12.825815+0000","last_in_change":"2024-04-15T15:19:03.788126+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:19:13.748585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:19:23.789287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:19:24.722250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6f6292b-2d7e-4aa0-9d1c-6ddd0f97287c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6801","nonce":2873040054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6803","nonce":2873040054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6807","nonce":2873040054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6805","nonce":2873040054}]},"public_addr":"172.21.15.105:6801/2873040054","cluster_addr":"172.21.15.105:6803/2873040054","heartbeat_back_addr":"172.21.15.105:6807/2873040054","heartbeat_front_addr":"172.21.15.105:6805/2873040054","state":["exists","up"]},{"osd":1,"uuid":"8900966a-b393-41e3-8f93-e5a43bd23622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6818","nonce":2494994479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6821","nonce":2494994479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6828","nonce":2494994479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6823","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6825","nonce":2494994479}]},"public_addr":"172.21.15.105:6818/2494994479","cluster_addr":"172.21.15.105:6821/2494994479","heartbeat_back_addr":"172.21.15.105:6828/2494994479","heartbeat_front_addr":"172.21.15.105:6825/2494994479","state":["exists","up"]},{"osd":2,"uuid":"81a5dc7a-9325-4ff5-941d-a2d5fba21f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6809","nonce":3798408699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6811","nonce":3798408699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6815","nonce":3798408699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6813","nonce":3798408699}]},"public_addr":"172.21.15.105:6809/3798408699","cluster_addr":"172.21.15.105:6811/3798408699","heartbeat_back_addr":"172.21.15.105:6815/3798408699","heartbeat_front_addr":"172.21.15.105:6813/3798408699","state":["exists","up"]},{"osd":3,"uuid":"df766ed2-5c71-4805-96f4-3d4970edc960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6819","nonce":1920720186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6824","nonce":1920720186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6831","nonce":1920720186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6827","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6829","nonce":1920720186}]},"public_addr":"172.21.15.105:6819/1920720186","cluster_addr":"172.21.15.105:6824/1920720186","heartbeat_back_addr":"172.21.15.105:6831/1920720186","heartbeat_front_addr":"172.21.15.105:6829/1920720186","state":["exists","up"]},{"osd":4,"uuid":"e1039e03-28b8-49df-b951-c91930b93c9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6810","nonce":3516494456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6813","nonce":3516494456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6820","nonce":3516494456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6817","nonce":3516494456}]},"public_addr":"172.21.15.154:6810/3516494456","cluster_addr":"172.21.15.154:6813/3516494456","heartbeat_back_addr":"172.21.15.154:6820/3516494456","heartbeat_front_addr":"172.21.15.154:6817/3516494456","state":["exists","up"]},{"osd":5,"uuid":"a9f5eaf1-66ff-405a-a9e4-ec3340812a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6801","nonce":540367751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6803","nonce":540367751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6807","nonce":540367751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6805","nonce":540367751}]},"public_addr":"172.21.15.154:6801/540367751","cluster_addr":"172.21.15.154:6803/540367751","heartbeat_back_addr":"172.21.15.154:6807/540367751","heartbeat_front_addr":"172.21.15.154:6805/540367751","state":["exists","up"]},{"osd":6,"uuid":"267d02ea-675f-428f-aae8-e0375e4d4feb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6825","nonce":1550829005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6827","nonce":1550829005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6831","nonce":1550829005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6829","nonce":1550829005}]},"public_addr":"172.21.15.154:6825/1550829005","cluster_addr":"172.21.15.154:6827/1550829005","heartbeat_back_addr":"172.21.15.154:6831/1550829005","heartbeat_front_addr":"172.21.15.154:6829/1550829005","state":["exists","up"]},{"osd":7,"uuid":"759ac6cc-896a-423c-885f-9fe90d8dcb14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6809","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6811","nonce":3920089903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6816","nonce":3920089903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6823","nonce":3920089903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6819","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6821","nonce":3920089903}]},"public_addr":"172.21.15.154:6811/3920089903","cluster_addr":"172.21.15.154:6816/3920089903","heartbeat_back_addr":"172.21.15.154:6823/3920089903","heartbeat_front_addr":"172.21.15.154:6821/3920089903","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.949509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.984072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.907278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.947391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.975447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.998896+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:09.028739+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.894119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:19:37.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.960+0000 7f026fc74640 1 -- 172.21.15.105:0/1073254849 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f025406c920 msgr2=0x7f025406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:37.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.960+0000 7f026fc74640 1 --2- 172.21.15.105:0/1073254849 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f025406c920 0x7f025406ede0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0258009f60 tx=0x7f025800b040 comp rx=0 tx=0).stop 2024-04-15T15:19:37.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.960+0000 7f026fc74640 1 -- 172.21.15.105:0/1073254849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02680d0f30 msgr2=0x7f026815d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:37.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.960+0000 7f026fc74640 1 --2- 172.21.15.105:0/1073254849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02680d0f30 0x7f026815d650 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f025c000c00 tx=0x7f025c004730 comp rx=0 tx=0).stop 2024-04-15T15:19:37.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.960+0000 7f026fc74640 1 -- 172.21.15.105:0/1073254849 shutdown_connections 2024-04-15T15:19:37.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.960+0000 7f026fc74640 1 --2- 172.21.15.105:0/1073254849 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f025406c920 0x7f025406ede0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:37.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.960+0000 7f026fc74640 1 --2- 172.21.15.105:0/1073254849 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f026815c7b0 0x7f026815db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:37.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.960+0000 7f026fc74640 1 --2- 172.21.15.105:0/1073254849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02680d0f30 0x7f026815d650 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:37.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.960+0000 7f026fc74640 1 --2- 172.21.15.105:0/1073254849 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f02680b2ae0 0x7f0268153c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:37.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.960+0000 7f026fc74640 1 -- 172.21.15.105:0/1073254849 >> 172.21.15.105:0/1073254849 conn(0x7f02680d25a0 msgr2=0x7f02680af330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:37.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.960+0000 7f026fc74640 1 -- 172.21.15.105:0/1073254849 shutdown_connections 2024-04-15T15:19:37.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:37.960+0000 7f026fc74640 1 -- 172.21.15.105:0/1073254849 wait complete. 2024-04-15T15:19:37.973 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-15T15:19:38.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.074+0000 7f74d3cba640 1 Processor -- start 2024-04-15T15:19:38.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.075+0000 7f74d3cba640 1 -- start start 2024-04-15T15:19:38.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.075+0000 7f74d3cba640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f74cc15c7b0 0x7f74cc0d09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:38.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.075+0000 7f74d3cba640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f74cc0d0f30 0x7f74cc0b25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:38.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.075+0000 7f74d3cba640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74cc0ccf40 con 0x7f74cc0b2ae0 2024-04-15T15:19:38.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.075+0000 7f74d3cba640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f74cc0b32d0 con 0x7f74cc0d0f30 2024-04-15T15:19:38.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.075+0000 7f74d3cba640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f74cc0b35b0 con 0x7f74cc15c7b0 2024-04-15T15:19:38.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.076+0000 7f74d1255640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f74cc0d0f30 0x7f74cc0b25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:38.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.076+0000 7f74d1255640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f74cc0d0f30 0x7f74cc0b25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:40036/0 (socket says 172.21.15.105:40036) 2024-04-15T15:19:38.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.076+0000 7f74d1255640 1 -- 172.21.15.105:0/4121968655 learned_addr learned my addr 172.21.15.105:0/4121968655 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:38.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.076+0000 7f74d1a56640 1 --2- 172.21.15.105:0/4121968655 >> v2:172.21.15.154:3300/0 conn(0x7f74cc15c7b0 0x7f74cc0d09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:38.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.076+0000 7f74d1255640 1 -- 172.21.15.105:0/4121968655 >> v2:172.21.15.154:3300/0 conn(0x7f74cc15c7b0 msgr2=0x7f74cc0d09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:38.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.076+0000 7f74d1255640 1 --2- 172.21.15.105:0/4121968655 >> v2:172.21.15.154:3300/0 conn(0x7f74cc15c7b0 0x7f74cc0d09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.076+0000 7f74d1255640 1 -- 172.21.15.105:0/4121968655 >> v1:172.21.15.154:6789/0 conn(0x7f74cc0b2ae0 legacy=0x7f74cc160410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:38.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.076+0000 7f74d1255640 1 -- 172.21.15.105:0/4121968655 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74cc0b3890 con 0x7f74cc0d0f30 2024-04-15T15:19:38.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.077+0000 7f74d1255640 1 --2- 172.21.15.105:0/4121968655 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74cc0d0f30 0x7f74cc0b25a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f74cc0cfdf0 tx=0x7f74c802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9084194ddf249a54 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:38.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.077+0000 7f74d0a54640 1 -- 172.21.15.105:0/4121968655 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c8034070 con 0x7f74cc0d0f30 2024-04-15T15:19:38.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.077+0000 7f74d0a54640 1 -- 172.21.15.105:0/4121968655 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74c8030860 con 0x7f74cc0d0f30 2024-04-15T15:19:38.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.077+0000 7f74d0a54640 1 -- 172.21.15.105:0/4121968655 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c8030c30 con 0x7f74cc0d0f30 2024-04-15T15:19:38.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.077+0000 7f74d3cba640 1 -- 172.21.15.105:0/4121968655 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74cc0d0f30 msgr2=0x7f74cc0b25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:38.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.077+0000 7f74d3cba640 1 --2- 172.21.15.105:0/4121968655 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74cc0d0f30 0x7f74cc0b25a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f74cc0cfdf0 tx=0x7f74c802b9a0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.077+0000 7f74d3cba640 1 -- 172.21.15.105:0/4121968655 shutdown_connections 2024-04-15T15:19:38.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.077+0000 7f74d3cba640 1 --2- 172.21.15.105:0/4121968655 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74cc0d0f30 0x7f74cc0b25a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.077+0000 7f74d3cba640 1 --2- 172.21.15.105:0/4121968655 >> v2:172.21.15.154:3300/0 conn(0x7f74cc15c7b0 0x7f74cc0d09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.077+0000 7f74d3cba640 1 -- 172.21.15.105:0/4121968655 >> 172.21.15.105:0/4121968655 conn(0x7f74cc0d25a0 msgr2=0x7f74cc0d29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:38.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.077+0000 7f74d3cba640 1 -- 172.21.15.105:0/4121968655 shutdown_connections 2024-04-15T15:19:38.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.077+0000 7f74d3cba640 1 -- 172.21.15.105:0/4121968655 wait complete. 2024-04-15T15:19:38.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.079+0000 7f74d3cba640 1 Processor -- start 2024-04-15T15:19:38.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.079+0000 7f74d3cba640 1 -- start start 2024-04-15T15:19:38.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.079+0000 7f74d3cba640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74cc0b2ae0 0x7f74cc15ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:38.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.079+0000 7f74d3cba640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74cc0d0f30 0x7f74cc0b53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:38.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.079+0000 7f74d3cba640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f74cc15c7b0 0x7f74cc0cb840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:38.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.079+0000 7f74d3cba640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f74cc15cb90 con 0x7f74cc0b2ae0 2024-04-15T15:19:38.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.079+0000 7f74d3cba640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f74cc0b2f70 con 0x7f74cc15c7b0 2024-04-15T15:19:38.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.079+0000 7f74d3cba640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f74cc15cd10 con 0x7f74cc0d0f30 2024-04-15T15:19:38.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.079+0000 7f74d1a56640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74cc0b2ae0 0x7f74cc15ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:38.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.079+0000 7f74d1a56640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74cc0b2ae0 0x7f74cc15ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:40052/0 (socket says 172.21.15.105:40052) 2024-04-15T15:19:38.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.079+0000 7f74d1a56640 1 -- 172.21.15.105:0/1179865335 learned_addr learned my addr 172.21.15.105:0/1179865335 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:38.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.080+0000 7f74d1a56640 1 -- 172.21.15.105:0/1179865335 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74cc0d0f30 msgr2=0x7f74cc0b53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:38.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.080+0000 7f74d1a56640 1 --2- 172.21.15.105:0/1179865335 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74cc0d0f30 0x7f74cc0b53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.080+0000 7f74d1a56640 1 -- 172.21.15.105:0/1179865335 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f74cc15c7b0 msgr2=0x7f74cc0cb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:38.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.080+0000 7f74d1a56640 1 --2- 172.21.15.105:0/1179865335 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f74cc15c7b0 0x7f74cc0cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.080+0000 7f74d1a56640 1 -- 172.21.15.105:0/1179865335 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74cc0b5c20 con 0x7f74cc0b2ae0 2024-04-15T15:19:38.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.080+0000 7f74d1a56640 1 --2- 172.21.15.105:0/1179865335 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74cc0b2ae0 0x7f74cc15ace0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f74c000d5f0 tx=0x7f74c000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:38.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.080+0000 7f74be7fc640 1 -- 172.21.15.105:0/1179865335 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c00125f0 con 0x7f74cc0b2ae0 2024-04-15T15:19:38.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.080+0000 7f74be7fc640 1 -- 172.21.15.105:0/1179865335 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74c0019070 con 0x7f74cc0b2ae0 2024-04-15T15:19:38.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.080+0000 7f74be7fc640 1 -- 172.21.15.105:0/1179865335 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c00129d0 con 0x7f74cc0b2ae0 2024-04-15T15:19:38.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.080+0000 7f74d3cba640 1 -- 172.21.15.105:0/1179865335 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74cc0b5f00 con 0x7f74cc0b2ae0 2024-04-15T15:19:38.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.080+0000 7f74d3cba640 1 -- 172.21.15.105:0/1179865335 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74cc0b64c0 con 0x7f74cc0b2ae0 2024-04-15T15:19:38.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.081+0000 7f74d3cba640 1 -- 172.21.15.105:0/1179865335 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74cc0b6700 con 0x7f74cc0b2ae0 2024-04-15T15:19:38.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.081+0000 7f74be7fc640 1 -- 172.21.15.105:0/1179865335 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f74c0012b70 con 0x7f74cc0b2ae0 2024-04-15T15:19:38.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.082+0000 7f74be7fc640 1 --2- 172.21.15.105:0/1179865335 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f74b006c8f0 0x7f74b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:38.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.082+0000 7f74be7fc640 1 -- 172.21.15.105:0/1179865335 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f74c008bec0 con 0x7f74cc0b2ae0 2024-04-15T15:19:38.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.082+0000 7f74d1255640 1 --2- 172.21.15.105:0/1179865335 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f74b006c8f0 0x7f74b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:38.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.082+0000 7f74d1255640 1 --2- 172.21.15.105:0/1179865335 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f74b006c8f0 0x7f74b006edb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f74c8036000 tx=0x7f74c803b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:38.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.086+0000 7f74be7fc640 1 -- 172.21.15.105:0/1179865335 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f74cc0b6700 con 0x7f74cc0b2ae0 2024-04-15T15:19:38.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.299+0000 7f74d3cba640 1 -- 172.21.15.105:0/1179865335 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f74cc0b6dd0 con 0x7f74cc0b2ae0 2024-04-15T15:19:38.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.299+0000 7f74be7fc640 1 -- 172.21.15.105:0/1179865335 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5187 (secure 0 0 0) 0x7f74c005a380 con 0x7f74cc0b2ae0 2024-04-15T15:19:38.301 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:19:38.301 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4259,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/3319657830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":3319657830},{"type":"v1","addr":"172.21.15.154:6835","nonce":3319657830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2677199890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2677199890},{"type":"v1","addr":"172.21.15.105:6835","nonce":2677199890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2867739743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2867739743},{"type":"v1","addr":"172.21.15.105:6837","nonce":2867739743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-15T15:19:25.760362+0000","modified":"2024-04-15T15:19:27.857599+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.154:6834/1529675504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1529675504},{"type":"v1","addr":"172.21.15.154:6834","nonce":1529675504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4291,"qdb_cluster":[4291]},"id":1}]} 2024-04-15T15:19:38.301 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 6 2024-04-15T15:19:38.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.301+0000 7f74d3cba640 1 -- 172.21.15.105:0/1179865335 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f74b006c8f0 msgr2=0x7f74b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:38.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.301+0000 7f74d3cba640 1 --2- 172.21.15.105:0/1179865335 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f74b006c8f0 0x7f74b006edb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f74c8036000 tx=0x7f74c803b040 comp rx=0 tx=0).stop 2024-04-15T15:19:38.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.301+0000 7f74d3cba640 1 -- 172.21.15.105:0/1179865335 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74cc0b2ae0 msgr2=0x7f74cc15ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:38.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.301+0000 7f74d3cba640 1 --2- 172.21.15.105:0/1179865335 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74cc0b2ae0 0x7f74cc15ace0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f74c000d5f0 tx=0x7f74c000dab0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.301+0000 7f74d3cba640 1 -- 172.21.15.105:0/1179865335 shutdown_connections 2024-04-15T15:19:38.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.301+0000 7f74d3cba640 1 --2- 172.21.15.105:0/1179865335 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74cc0d0f30 0x7f74cc0b53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.301+0000 7f74d3cba640 1 --2- 172.21.15.105:0/1179865335 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f74b006c8f0 0x7f74b006edb0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.301+0000 7f74d3cba640 1 --2- 172.21.15.105:0/1179865335 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f74cc15c7b0 0x7f74cc0cb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.301+0000 7f74d3cba640 1 --2- 172.21.15.105:0/1179865335 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74cc0b2ae0 0x7f74cc15ace0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.301+0000 7f74d3cba640 1 -- 172.21.15.105:0/1179865335 >> 172.21.15.105:0/1179865335 conn(0x7f74cc0d25a0 msgr2=0x7f74cc0b1340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:38.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.301+0000 7f74d3cba640 1 -- 172.21.15.105:0/1179865335 shutdown_connections 2024-04-15T15:19:38.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.301+0000 7f74d3cba640 1 -- 172.21.15.105:0/1179865335 wait complete. 2024-04-15T15:19:38.716 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-15T15:19:25.760362+0000', 'modified': '2024-04-15T15:19:27.857599+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4291}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4291': {'gid': 4291, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.154:6834/1529675504', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6832', 'nonce': 1529675504}, {'type': 'v1', 'addr': '172.21.15.154:6834', 'nonce': 1529675504}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4291, 'qdb_cluster': [4291]} 2024-04-15T15:19:38.716 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-15T15:19:38.716 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-15T15:19:38.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.820+0000 7f7ecf1b5640 1 Processor -- start 2024-04-15T15:19:38.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.820+0000 7f7ecf1b5640 1 -- start start 2024-04-15T15:19:38.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.821+0000 7f7ecf1b5640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7ec809b150 0x7f7ec80d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:38.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.821+0000 7f7ecf1b5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ec809b780 0x7f7ec80cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:38.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.821+0000 7f7ecf1b5640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ec80714f0 con 0x7f7ec815ec60 2024-04-15T15:19:38.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.821+0000 7f7ecf1b5640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f7ec806cad0 con 0x7f7ec809b150 2024-04-15T15:19:38.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.821+0000 7f7ecf1b5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f7ec80cd3b0 con 0x7f7ec809b780 2024-04-15T15:19:38.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.821+0000 7f7ecd752640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ec809b780 0x7f7ec80cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:38.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.821+0000 7f7ec7fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7ec809b150 0x7f7ec80d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:38.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.821+0000 7f7ecd752640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ec809b780 0x7f7ec80cd840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.105:57442/0 (socket says 172.21.15.105:57442) 2024-04-15T15:19:38.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.821+0000 7f7ec7fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7ec809b150 0x7f7ec80d1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.105:45444/0 (socket says 172.21.15.105:45444) 2024-04-15T15:19:38.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.821+0000 7f7ecd752640 1 -- 172.21.15.105:0/1011743215 learned_addr learned my addr 172.21.15.105:0/1011743215 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:38.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.821+0000 7f7eccf51640 1 --1- 172.21.15.105:0/1011743215 >> v1:172.21.15.154:6789/0 conn(0x7f7ec815ec60 0x7f7ec809aa40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.105:36996/0 (socket says 172.21.15.105:36996) 2024-04-15T15:19:38.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.822+0000 7f7ecd752640 1 -- 172.21.15.105:0/1011743215 >> v2:172.21.15.154:3300/0 conn(0x7f7ec809b150 msgr2=0x7f7ec80d1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:38.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.822+0000 7f7ecd752640 1 --2- 172.21.15.105:0/1011743215 >> v2:172.21.15.154:3300/0 conn(0x7f7ec809b150 0x7f7ec80d1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.822+0000 7f7ecd752640 1 -- 172.21.15.105:0/1011743215 >> v1:172.21.15.154:6789/0 conn(0x7f7ec815ec60 legacy=0x7f7ec809aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:38.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.822+0000 7f7ecd752640 1 -- 172.21.15.105:0/1011743215 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ec80d1940 con 0x7f7ec809b780 2024-04-15T15:19:38.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.822+0000 7f7ec77fe640 1 -- 172.21.15.105:0/1011743215 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 207625587 0 0) 0x7f7ec80714f0 con 0x7f7ec815ec60 2024-04-15T15:19:38.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.822+0000 7f7ec7fff640 1 --2- 172.21.15.105:0/1011743215 >> v2:172.21.15.154:3300/0 conn(0x7f7ec809b150 0x7f7ec80d1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:19:38.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.822+0000 7f7ecd752640 1 --2- 172.21.15.105:0/1011743215 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ec809b780 0x7f7ec80cd840 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0009e50 tx=0x7f7ec002bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ae03e4d24b41889 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:38.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.823+0000 7f7ec77fe640 1 -- 172.21.15.105:0/1011743215 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec002cb00 con 0x7f7ec809b780 2024-04-15T15:19:38.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.823+0000 7f7ec77fe640 1 -- 172.21.15.105:0/1011743215 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ec002cca0 con 0x7f7ec809b780 2024-04-15T15:19:38.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.823+0000 7f7ec77fe640 1 -- 172.21.15.105:0/1011743215 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec0034610 con 0x7f7ec809b780 2024-04-15T15:19:38.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.823+0000 7f7ecf1b5640 1 -- 172.21.15.105:0/1011743215 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ec809b780 msgr2=0x7f7ec80cd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:38.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.823+0000 7f7ecf1b5640 1 --2- 172.21.15.105:0/1011743215 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ec809b780 0x7f7ec80cd840 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0009e50 tx=0x7f7ec002bf60 comp rx=0 tx=0).stop 2024-04-15T15:19:38.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.823+0000 7f7ecf1b5640 1 -- 172.21.15.105:0/1011743215 shutdown_connections 2024-04-15T15:19:38.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.823+0000 7f7ecf1b5640 1 --2- 172.21.15.105:0/1011743215 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ec809b780 0x7f7ec80cd840 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.823+0000 7f7ecf1b5640 1 --2- 172.21.15.105:0/1011743215 >> v2:172.21.15.154:3300/0 conn(0x7f7ec809b150 0x7f7ec80d1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.823+0000 7f7ecf1b5640 1 -- 172.21.15.105:0/1011743215 >> 172.21.15.105:0/1011743215 conn(0x7f7ec80d2570 msgr2=0x7f7ec80d2980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:38.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.823+0000 7f7ecf1b5640 1 -- 172.21.15.105:0/1011743215 shutdown_connections 2024-04-15T15:19:38.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.823+0000 7f7ecf1b5640 1 -- 172.21.15.105:0/1011743215 wait complete. 2024-04-15T15:19:38.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.824+0000 7f7ecf1b5640 1 Processor -- start 2024-04-15T15:19:38.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.824+0000 7f7ecf1b5640 1 -- start start 2024-04-15T15:19:38.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.825+0000 7f7ecf1b5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ec809b150 0x7f7ec80cc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:38.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.825+0000 7f7ecf1b5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7ec809b780 0x7f7ec80ca250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:38.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.825+0000 7f7ecf1b5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ec80d10f0 0x7f7ec80b7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:38.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.825+0000 7f7ecf1b5640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec80cdd80 con 0x7f7ec809b780 2024-04-15T15:19:38.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.825+0000 7f7ecf1b5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec80cd040 con 0x7f7ec809b150 2024-04-15T15:19:38.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.825+0000 7f7ecf1b5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f7ec80cdf00 con 0x7f7ec80d10f0 2024-04-15T15:19:38.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.825+0000 7f7ec7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7ec809b780 0x7f7ec80ca250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:38.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.825+0000 7f7ec7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7ec809b780 0x7f7ec80ca250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:40066/0 (socket says 172.21.15.105:40066) 2024-04-15T15:19:38.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.825+0000 7f7ec7fff640 1 -- 172.21.15.105:0/3038865120 learned_addr learned my addr 172.21.15.105:0/3038865120 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:38.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.826+0000 7f7eccf51640 1 --2- 172.21.15.105:0/3038865120 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ec809b150 0x7f7ec80cc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:38.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.826+0000 7f7ecd752640 1 --2- 172.21.15.105:0/3038865120 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ec80d10f0 0x7f7ec80b7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:38.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.826+0000 7f7ec7fff640 1 -- 172.21.15.105:0/3038865120 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ec80d10f0 msgr2=0x7f7ec80b7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:38.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.826+0000 7f7ec7fff640 1 --2- 172.21.15.105:0/3038865120 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ec80d10f0 0x7f7ec80b7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.826+0000 7f7ec7fff640 1 -- 172.21.15.105:0/3038865120 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ec809b150 msgr2=0x7f7ec80cc0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:38.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.826+0000 7f7ec7fff640 1 --2- 172.21.15.105:0/3038865120 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ec809b150 0x7f7ec80cc0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.826+0000 7f7ec7fff640 1 -- 172.21.15.105:0/3038865120 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ec80b6de0 con 0x7f7ec809b780 2024-04-15T15:19:38.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.826+0000 7f7ec7fff640 1 --2- 172.21.15.105:0/3038865120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7ec809b780 0x7f7ec80ca250 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc00ebe0 tx=0x7f7ebc00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:38.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.826+0000 7f7ec57fa640 1 -- 172.21.15.105:0/3038865120 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc017070 con 0x7f7ec809b780 2024-04-15T15:19:38.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.826+0000 7f7ec57fa640 1 -- 172.21.15.105:0/3038865120 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ebc00cbd0 con 0x7f7ec809b780 2024-04-15T15:19:38.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.826+0000 7f7ecf1b5640 1 -- 172.21.15.105:0/3038865120 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ec80d1580 con 0x7f7ec809b780 2024-04-15T15:19:38.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.826+0000 7f7ec57fa640 1 -- 172.21.15.105:0/3038865120 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc013470 con 0x7f7ec809b780 2024-04-15T15:19:38.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.826+0000 7f7ecf1b5640 1 -- 172.21.15.105:0/3038865120 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ec80b7640 con 0x7f7ec809b780 2024-04-15T15:19:38.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.828+0000 7f7ec57fa640 1 -- 172.21.15.105:0/3038865120 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7ebc01a070 con 0x7f7ec809b780 2024-04-15T15:19:38.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.828+0000 7f7ec57fa640 1 --2- 172.21.15.105:0/3038865120 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f7eb406c920 0x7f7eb406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:38.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.828+0000 7f7eccf51640 1 --2- 172.21.15.105:0/3038865120 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f7eb406c920 0x7f7eb406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:38.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.828+0000 7f7ec57fa640 1 -- 172.21.15.105:0/3038865120 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7ebc08ba40 con 0x7f7ec809b780 2024-04-15T15:19:38.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.829+0000 7f7eccf51640 1 --2- 172.21.15.105:0/3038865120 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f7eb406c920 0x7f7eb406ede0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8007fd0 tx=0x7f7eb8007de0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:38.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.829+0000 7f7ed1d95740 1 Processor -- start 2024-04-15T15:19:38.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.829+0000 7f7ed1d95740 1 -- start start 2024-04-15T15:19:38.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.830+0000 7f7ed1d95740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5599753a6e10 0x559975379db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:38.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.830+0000 7f7ed1d95740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599753a71f0 0x5599753a9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:38.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.830+0000 7f7ed1d95740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55997537a2f0 0x5599753aa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:38.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.830+0000 7f7ed1d95740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5599752e49c0 con 0x5599753a71f0 2024-04-15T15:19:38.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.830+0000 7f7ed1d95740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55997528b650 con 0x55997537a2f0 2024-04-15T15:19:38.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.830+0000 7f7ed1d95740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x5599752e0980 con 0x5599753a6e10 2024-04-15T15:19:38.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.830+0000 7f7ec7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599753a71f0 0x5599753a9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:38.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.830+0000 7f7ec7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599753a71f0 0x5599753a9ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:40076/0 (socket says 172.21.15.105:40076) 2024-04-15T15:19:38.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.830+0000 7f7ec7fff640 1 -- 172.21.15.105:0/3535761620 learned_addr learned my addr 172.21.15.105:0/3535761620 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:38.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.830+0000 7f7ec7fff640 1 -- 172.21.15.105:0/3535761620 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5599753a6e10 msgr2=0x559975379db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:38.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.830+0000 7f7ec7fff640 1 --2- 172.21.15.105:0/3535761620 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5599753a6e10 0x559975379db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.830+0000 7f7ec7fff640 1 -- 172.21.15.105:0/3535761620 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55997537a2f0 msgr2=0x5599753aa0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:38.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.830+0000 7f7ec7fff640 1 --2- 172.21.15.105:0/3535761620 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55997537a2f0 0x5599753aa0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.830+0000 7f7ec7fff640 1 -- 172.21.15.105:0/3535761620 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559974ff72d0 con 0x5599753a71f0 2024-04-15T15:19:38.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.831+0000 7f7ec7fff640 1 --2- 172.21.15.105:0/3535761620 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599753a71f0 0x5599753a9ba0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc055bf0 tx=0x7f7ebc0590f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2e8ec60c84c69b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:38.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.831+0000 7f7eae7fc640 1 -- 172.21.15.105:0/3535761620 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc021050 con 0x5599753a71f0 2024-04-15T15:19:38.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.831+0000 7f7eae7fc640 1 -- 172.21.15.105:0/3535761620 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ebc0622c0 con 0x5599753a71f0 2024-04-15T15:19:38.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.831+0000 7f7eae7fc640 1 -- 172.21.15.105:0/3535761620 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc055e00 con 0x5599753a71f0 2024-04-15T15:19:38.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.831+0000 7f7ed1d95740 1 -- 172.21.15.105:0/3535761620 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599753a71f0 msgr2=0x5599753a9ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:38.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.831+0000 7f7ed1d95740 1 --2- 172.21.15.105:0/3535761620 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599753a71f0 0x5599753a9ba0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc055bf0 tx=0x7f7ebc0590f0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.831+0000 7f7ed1d95740 1 -- 172.21.15.105:0/3535761620 shutdown_connections 2024-04-15T15:19:38.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.831+0000 7f7ed1d95740 1 --2- 172.21.15.105:0/3535761620 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55997537a2f0 0x5599753aa0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.831+0000 7f7ed1d95740 1 --2- 172.21.15.105:0/3535761620 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599753a71f0 0x5599753a9ba0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.831+0000 7f7ed1d95740 1 --2- 172.21.15.105:0/3535761620 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5599753a6e10 0x559975379db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.831+0000 7f7ed1d95740 1 -- 172.21.15.105:0/3535761620 >> 172.21.15.105:0/3535761620 conn(0x55997537c020 msgr2=0x5599753c4ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:38.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.831+0000 7f7ed1d95740 1 -- 172.21.15.105:0/3535761620 shutdown_connections 2024-04-15T15:19:38.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.831+0000 7f7ed1d95740 1 -- 172.21.15.105:0/3535761620 wait complete. 2024-04-15T15:19:38.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.832+0000 7f7ed1d95740 1 -- start start 2024-04-15T15:19:38.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.832+0000 7f7ed1d95740 1 Processor -- start 2024-04-15T15:19:38.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.833+0000 7f7ed1d95740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5599753a6e10 0x5599753d5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:38.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.833+0000 7f7ed1d95740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5599753a71f0 0x5599753c4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:38.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.833+0000 7f7ed1d95740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55997539f960 0x55997539fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:38.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.833+0000 7f7ed1d95740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55997537e5c0 con 0x55997539f960 2024-04-15T15:19:38.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.833+0000 7f7ed1d95740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55997529ea10 con 0x5599753a6e10 2024-04-15T15:19:38.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.833+0000 7f7ed1d95740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x55997537d9b0 con 0x5599753a71f0 2024-04-15T15:19:38.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.833+0000 7f7ed1d95740 10 client.0 _ll_register_callbacks cb 0x5599752da250 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-15T15:19:38.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.833+0000 7f7ecd752640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55997539f960 0x55997539fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:38.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.833+0000 7f7ecd752640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55997539f960 0x55997539fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:40082/0 (socket says 172.21.15.105:40082) 2024-04-15T15:19:38.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.833+0000 7f7ecd752640 1 -- 172.21.15.105:0/1756892186 learned_addr learned my addr 172.21.15.105:0/1756892186 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:38.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.833+0000 7f7ec7fff640 1 --2- 172.21.15.105:0/1756892186 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5599753a71f0 0x5599753c4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:38.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.833+0000 7f7eccf51640 1 --2- 172.21.15.105:0/1756892186 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5599753a6e10 0x5599753d5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:38.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.833+0000 7f7ecd752640 1 -- 172.21.15.105:0/1756892186 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5599753a71f0 msgr2=0x5599753c4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:38.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.833+0000 7f7ecd752640 1 --2- 172.21.15.105:0/1756892186 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5599753a71f0 0x5599753c4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.833+0000 7f7ecd752640 1 -- 172.21.15.105:0/1756892186 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5599753a6e10 msgr2=0x5599753d5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:38.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.833+0000 7f7ecd752640 1 --2- 172.21.15.105:0/1756892186 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5599753a6e10 0x5599753d5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.833+0000 7f7ecd752640 1 -- 172.21.15.105:0/1756892186 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599750c16b0 con 0x55997539f960 2024-04-15T15:19:38.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.833+0000 7f7ecd752640 1 --2- 172.21.15.105:0/1756892186 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55997539f960 0x55997539fde0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0009f70 tx=0x7f7ec0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8fa6f5e3f64a9d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:38.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.833+0000 7f7ea77fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-04-15T15:19:38.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.834+0000 7f7ea77fe640 1 -- 172.21.15.105:0/1756892186 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec0045030 con 0x55997539f960 2024-04-15T15:19:38.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.834+0000 7f7ea77fe640 1 -- 172.21.15.105:0/1756892186 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7ec003a040 con 0x55997539f960 2024-04-15T15:19:38.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.834+0000 7f7ed1d95740 1 -- 172.21.15.105:0/1756892186 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ebc0622c0 con 0x55997539f960 2024-04-15T15:19:38.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.834+0000 7f7ea77fe640 1 -- 172.21.15.105:0/1756892186 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) ==== 24+0+0 (secure 0 0 0) 0x7f7ec004e050 con 0x55997539f960 2024-04-15T15:19:38.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.834+0000 7f7ed1d95740 10 client.4650 fetch_fsmap learned FSMap version 6 2024-04-15T15:19:38.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.834+0000 7f7ed1d95740 1 -- 172.21.15.105:0/1756892186 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) -- 0x5599753a20c0 con 0x55997539f960 2024-04-15T15:19:38.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.834+0000 7f7ea77fe640 1 -- 172.21.15.105:0/1756892186 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ebc0622c0 con 0x55997539f960 2024-04-15T15:19:38.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.834+0000 7f7ea77fe640 1 -- 172.21.15.105:0/1756892186 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 6) ==== 3494+0+0 (secure 0 0 0) 0x7f7ec0051020 con 0x55997539f960 2024-04-15T15:19:38.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.834+0000 7f7ed1d95740 10 client.4650 fetch_fsmap finished waiting for FSMap version 6 2024-04-15T15:19:38.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.834+0000 7f7ed1d95740 10 client.4650 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-15T15:19:38.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.834+0000 7f7ed1d95740 20 client.4650 populate_metadata read hostname 'smithi105' 2024-04-15T15:19:38.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.834+0000 7f7ed1d95740 1 --2- 172.21.15.105:0/1756892186 >> [v2:172.21.15.154:6832/1529675504,v1:172.21.15.154:6834/1529675504] conn(0x5599753abec0 0x5599753ac2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:38.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.834+0000 7f7ed1d95740 4 client.4650 mds_command: new command op to 4291 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-15T15:19:38.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.834+0000 7f7ed1d95740 1 -- 172.21.15.105:0/1756892186 --> [v2:172.21.15.154:6832/1529675504,v1:172.21.15.154:6834/1529675504] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599752f23e0 con 0x5599753abec0 2024-04-15T15:19:38.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.834+0000 7f7eccf51640 1 --2- 172.21.15.105:0/1756892186 >> [v2:172.21.15.154:6832/1529675504,v1:172.21.15.154:6834/1529675504] conn(0x5599753abec0 0x5599753ac2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:38.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.835+0000 7f7eccf51640 1 --2- 172.21.15.105:0/1756892186 >> [v2:172.21.15.154:6832/1529675504,v1:172.21.15.154:6834/1529675504] conn(0x5599753abec0 0x5599753ac2a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=339fd1d4b4935ed3 server_cookie=64bbae95289865a4 in_seq=0 out_seq=0 2024-04-15T15:19:38.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.835+0000 7f7ea77fe640 10 client.4650 ms_handle_connect on v2:172.21.15.154:6832/1529675504 2024-04-15T15:19:38.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.835+0000 7f7ea77fe640 1 -- 172.21.15.105:0/1756892186 <== mds.0 v2:172.21.15.154:6832/1529675504 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5599752f23e0 con 0x5599753abec0 2024-04-15T15:19:38.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.835+0000 7f7ea77fe640 10 client.4650 handle_command_reply: tid=0 2024-04-15T15:19:38.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.836+0000 7f7ed1d95740 1 client.4650 shutdown 2024-04-15T15:19:38.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.836+0000 7f7ed1d95740 1 -- 172.21.15.105:0/1756892186 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55997539f960 msgr2=0x55997539fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:38.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.836+0000 7f7ed1d95740 1 --2- 172.21.15.105:0/1756892186 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55997539f960 0x55997539fde0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0009f70 tx=0x7f7ec0004a40 comp rx=0 tx=0).stop 2024-04-15T15:19:38.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.836+0000 7f7ed1d95740 1 -- 172.21.15.105:0/1756892186 shutdown_connections 2024-04-15T15:19:38.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.836+0000 7f7ed1d95740 1 --2- 172.21.15.105:0/1756892186 >> [v2:172.21.15.154:6832/1529675504,v1:172.21.15.154:6834/1529675504] conn(0x5599753abec0 0x5599753ac2a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.836+0000 7f7ed1d95740 1 --2- 172.21.15.105:0/1756892186 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5599753a71f0 0x5599753c4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.836+0000 7f7ed1d95740 1 --2- 172.21.15.105:0/1756892186 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55997539f960 0x55997539fde0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.836+0000 7f7ed1d95740 1 --2- 172.21.15.105:0/1756892186 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5599753a6e10 0x5599753d5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.836+0000 7f7ea77fe640 0 client.4650 ms_handle_reset on v2:172.21.15.154:6832/1529675504 2024-04-15T15:19:38.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.836+0000 7f7ed1d95740 1 -- 172.21.15.105:0/1756892186 >> 172.21.15.105:0/1756892186 conn(0x55997537a2f0 msgr2=0x5599753c80b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:38.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.836+0000 7f7ed1d95740 1 -- 172.21.15.105:0/1756892186 shutdown_connections 2024-04-15T15:19:38.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.836+0000 7f7ed1d95740 1 -- 172.21.15.105:0/1756892186 wait complete. 2024-04-15T15:19:38.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.836+0000 7f7ed1d95740 20 client.4650 trim_cache size 0 max 16384 2024-04-15T15:19:38.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.854+0000 7f7ed1d95740 1 Processor -- start 2024-04-15T15:19:38.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.854+0000 7f7ed1d95740 1 -- start start 2024-04-15T15:19:38.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.855+0000 7f7ed1d95740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55997539f960 0x559975378ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:38.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.855+0000 7f7ed1d95740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5599753a6e10 0x55997539ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:38.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.855+0000 7f7ed1d95740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5599753a71f0 0x5599753d52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:38.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.855+0000 7f7ed1d95740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec8200c30 con 0x55997539f960 2024-04-15T15:19:38.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.855+0000 7f7ed1d95740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec80b6220 con 0x5599753a71f0 2024-04-15T15:19:38.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.855+0000 7f7ed1d95740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f7ec80b5580 con 0x5599753a6e10 2024-04-15T15:19:38.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.855+0000 7f7eccf51640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55997539f960 0x559975378ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:38.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.855+0000 7f7eccf51640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55997539f960 0x559975378ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:40096/0 (socket says 172.21.15.105:40096) 2024-04-15T15:19:38.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.855+0000 7f7eccf51640 1 -- 172.21.15.105:0/2230321965 learned_addr learned my addr 172.21.15.105:0/2230321965 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:38.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.855+0000 7f7eccf51640 1 -- 172.21.15.105:0/2230321965 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5599753a6e10 msgr2=0x55997539ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:38.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.855+0000 7f7eccf51640 1 --2- 172.21.15.105:0/2230321965 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5599753a6e10 0x55997539ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.855+0000 7f7eccf51640 1 -- 172.21.15.105:0/2230321965 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5599753a71f0 msgr2=0x5599753d52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:38.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.855+0000 7f7eccf51640 1 --2- 172.21.15.105:0/2230321965 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5599753a71f0 0x5599753d52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.855+0000 7f7eccf51640 1 -- 172.21.15.105:0/2230321965 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599753ada50 con 0x55997539f960 2024-04-15T15:19:38.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.855+0000 7f7eccf51640 1 --2- 172.21.15.105:0/2230321965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55997539f960 0x559975378ed0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8007450 tx=0x7f7eb8017040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bdde4cefa3b6541 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:38.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.855+0000 7f7eadffb640 1 -- 172.21.15.105:0/2230321965 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb8023440 con 0x55997539f960 2024-04-15T15:19:38.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.855+0000 7f7eadffb640 1 -- 172.21.15.105:0/2230321965 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7eb8028070 con 0x55997539f960 2024-04-15T15:19:38.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.855+0000 7f7eadffb640 1 -- 172.21.15.105:0/2230321965 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb80237e0 con 0x55997539f960 2024-04-15T15:19:38.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.856+0000 7f7ed1d95740 1 -- 172.21.15.105:0/2230321965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55997539f960 msgr2=0x559975378ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:38.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.856+0000 7f7ed1d95740 1 --2- 172.21.15.105:0/2230321965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55997539f960 0x559975378ed0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8007450 tx=0x7f7eb8017040 comp rx=0 tx=0).stop 2024-04-15T15:19:38.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.856+0000 7f7ed1d95740 1 -- 172.21.15.105:0/2230321965 shutdown_connections 2024-04-15T15:19:38.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.856+0000 7f7ed1d95740 1 --2- 172.21.15.105:0/2230321965 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5599753a6e10 0x55997539ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.856+0000 7f7ed1d95740 1 --2- 172.21.15.105:0/2230321965 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5599753a71f0 0x5599753d52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.856+0000 7f7ed1d95740 1 --2- 172.21.15.105:0/2230321965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55997539f960 0x559975378ed0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.856+0000 7f7ed1d95740 1 -- 172.21.15.105:0/2230321965 >> 172.21.15.105:0/2230321965 conn(0x55997537a2f0 msgr2=0x5599753c4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:38.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.856+0000 7f7ed1d95740 1 -- 172.21.15.105:0/2230321965 shutdown_connections 2024-04-15T15:19:38.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.856+0000 7f7ed1d95740 1 -- 172.21.15.105:0/2230321965 wait complete. 2024-04-15T15:19:38.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.856+0000 7f7ed1d95740 1 -- start start 2024-04-15T15:19:38.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.857+0000 7f7ed1d95740 1 Processor -- start 2024-04-15T15:19:38.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.857+0000 7f7ed1d95740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55997539f960 0x5599753c6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:38.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.857+0000 7f7ed1d95740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5599753a6e10 0x5599753babd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:38.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.857+0000 7f7ed1d95740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5599753a71f0 0x5599753bb110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:38.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.857+0000 7f7ed1d95740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5599753a23a0 con 0x55997539f960 2024-04-15T15:19:38.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.857+0000 7f7ed1d95740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec80b5bd0 con 0x5599753a71f0 2024-04-15T15:19:38.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.857+0000 7f7ed1d95740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x5599753a2c40 con 0x5599753a6e10 2024-04-15T15:19:38.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.857+0000 7f7ed1d95740 10 client.0 _ll_register_callbacks cb 0x7f7ec0009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-15T15:19:38.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.857+0000 7f7eccf51640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55997539f960 0x5599753c6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:38.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.857+0000 7f7eccf51640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55997539f960 0x5599753c6f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:40110/0 (socket says 172.21.15.105:40110) 2024-04-15T15:19:38.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.857+0000 7f7eccf51640 1 -- 172.21.15.105:0/1349489011 learned_addr learned my addr 172.21.15.105:0/1349489011 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:38.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.858+0000 7f7ecd752640 1 --2- 172.21.15.105:0/1349489011 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5599753a71f0 0x5599753bb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:38.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.858+0000 7f7ec7fff640 1 --2- 172.21.15.105:0/1349489011 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5599753a6e10 0x5599753babd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:38.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.858+0000 7f7eccf51640 1 -- 172.21.15.105:0/1349489011 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5599753a6e10 msgr2=0x5599753babd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:38.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.858+0000 7f7eccf51640 1 --2- 172.21.15.105:0/1349489011 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5599753a6e10 0x5599753babd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.858+0000 7f7eccf51640 1 -- 172.21.15.105:0/1349489011 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5599753a71f0 msgr2=0x5599753bb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:38.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.858+0000 7f7eccf51640 1 --2- 172.21.15.105:0/1349489011 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5599753a71f0 0x5599753bb110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.858+0000 7f7eccf51640 1 -- 172.21.15.105:0/1349489011 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599753c3950 con 0x55997539f960 2024-04-15T15:19:38.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.858+0000 7f7ec7fff640 1 --2- 172.21.15.105:0/1349489011 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5599753a6e10 0x5599753babd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:38.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.858+0000 7f7eccf51640 1 --2- 172.21.15.105:0/1349489011 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55997539f960 0x5599753c6f20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8025040 tx=0x7f7eb8000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83b75764e4ba0333 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:38.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.858+0000 7f7ea77fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-04-15T15:19:38.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.858+0000 7f7ea77fe640 1 -- 172.21.15.105:0/1349489011 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb8004e50 con 0x55997539f960 2024-04-15T15:19:38.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.858+0000 7f7ea77fe640 1 -- 172.21.15.105:0/1349489011 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7eb8010040 con 0x55997539f960 2024-04-15T15:19:38.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.858+0000 7f7ed1d95740 1 -- 172.21.15.105:0/1349489011 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7eb8028070 con 0x55997539f960 2024-04-15T15:19:38.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.858+0000 7f7ea77fe640 1 -- 172.21.15.105:0/1349489011 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) ==== 24+0+0 (secure 0 0 0) 0x7f7eb803c050 con 0x55997539f960 2024-04-15T15:19:38.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.858+0000 7f7ed1d95740 10 client.4656 fetch_fsmap learned FSMap version 6 2024-04-15T15:19:38.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.858+0000 7f7ed1d95740 1 -- 172.21.15.105:0/1349489011 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) -- 0x5599753d4f80 con 0x55997539f960 2024-04-15T15:19:38.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.858+0000 7f7ea77fe640 1 -- 172.21.15.105:0/1349489011 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7eb8028070 con 0x55997539f960 2024-04-15T15:19:38.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.858+0000 7f7ea77fe640 1 -- 172.21.15.105:0/1349489011 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 6) ==== 3494+0+0 (secure 0 0 0) 0x7f7eb803f020 con 0x55997539f960 2024-04-15T15:19:38.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.858+0000 7f7ed1d95740 10 client.4656 fetch_fsmap finished waiting for FSMap version 6 2024-04-15T15:19:38.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.858+0000 7f7ed1d95740 10 client.4656 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-15T15:19:38.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.858+0000 7f7ed1d95740 20 client.4656 populate_metadata read hostname 'smithi105' 2024-04-15T15:19:38.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.858+0000 7f7ed1d95740 1 --2- 172.21.15.105:0/1349489011 >> [v2:172.21.15.154:6832/1529675504,v1:172.21.15.154:6834/1529675504] conn(0x5599753e7a20 0x5599753e7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:38.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.858+0000 7f7ed1d95740 4 client.4656 mds_command: new command op to 4291 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-15T15:19:38.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.858+0000 7f7ed1d95740 1 -- 172.21.15.105:0/1349489011 --> [v2:172.21.15.154:6832/1529675504,v1:172.21.15.154:6834/1529675504] -- command(tid 0: {"prefix": "status"}) -- 0x5599752ec250 con 0x5599753e7a20 2024-04-15T15:19:38.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.859+0000 7f7ec7fff640 1 --2- 172.21.15.105:0/1349489011 >> [v2:172.21.15.154:6832/1529675504,v1:172.21.15.154:6834/1529675504] conn(0x5599753e7a20 0x5599753e7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:38.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.859+0000 7f7ec7fff640 1 --2- 172.21.15.105:0/1349489011 >> [v2:172.21.15.154:6832/1529675504,v1:172.21.15.154:6834/1529675504] conn(0x5599753e7a20 0x5599753e7e00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd33dd763b547705 server_cookie=66ce97b4ef5b2b05 in_seq=0 out_seq=0 2024-04-15T15:19:38.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.859+0000 7f7ea77fe640 10 client.4656 ms_handle_connect on v2:172.21.15.154:6832/1529675504 2024-04-15T15:19:38.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.859+0000 7f7ea77fe640 1 -- 172.21.15.105:0/1349489011 <== mds.0 v2:172.21.15.154:6832/1529675504 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x5599752ec250 con 0x5599753e7a20 2024-04-15T15:19:38.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.859+0000 7f7ea77fe640 10 client.4656 handle_command_reply: tid=0 2024-04-15T15:19:38.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.859+0000 7f7ed1d95740 1 client.4656 shutdown 2024-04-15T15:19:38.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.859+0000 7f7ed1d95740 1 -- 172.21.15.105:0/1349489011 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55997539f960 msgr2=0x5599753c6f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:38.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.859+0000 7f7ed1d95740 1 --2- 172.21.15.105:0/1349489011 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55997539f960 0x5599753c6f20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8025040 tx=0x7f7eb8000ed0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.859+0000 7f7ed1d95740 1 -- 172.21.15.105:0/1349489011 shutdown_connections 2024-04-15T15:19:38.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.859+0000 7f7ed1d95740 1 --2- 172.21.15.105:0/1349489011 >> [v2:172.21.15.154:6832/1529675504,v1:172.21.15.154:6834/1529675504] conn(0x5599753e7a20 0x5599753e7e00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.859+0000 7f7ed1d95740 1 --2- 172.21.15.105:0/1349489011 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5599753a6e10 0x5599753babd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.859+0000 7f7ed1d95740 1 --2- 172.21.15.105:0/1349489011 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5599753a71f0 0x5599753bb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.859+0000 7f7ea77fe640 0 client.4656 ms_handle_reset on v2:172.21.15.154:6832/1529675504 2024-04-15T15:19:38.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.859+0000 7f7ed1d95740 1 --2- 172.21.15.105:0/1349489011 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55997539f960 0x5599753c6f20 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.859+0000 7f7ed1d95740 1 -- 172.21.15.105:0/1349489011 >> 172.21.15.105:0/1349489011 conn(0x55997537a2f0 msgr2=0x5599753753e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:38.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.859+0000 7f7ed1d95740 1 -- 172.21.15.105:0/1349489011 shutdown_connections 2024-04-15T15:19:38.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.859+0000 7f7ed1d95740 1 -- 172.21.15.105:0/1349489011 wait complete. 2024-04-15T15:19:38.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.860+0000 7f7ed1d95740 20 client.4656 trim_cache size 0 max 16384 2024-04-15T15:19:38.867 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-04-15T15:19:38.868 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "09e4d2a4-a324-4793-af77-644b25e520e1", 2024-04-15T15:19:38.868 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 0, 2024-04-15T15:19:38.868 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 4291, 2024-04-15T15:19:38.868 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2024-04-15T15:19:38.868 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2024-04-15T15:19:38.868 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2024-04-15T15:19:38.868 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 12.427538617, 2024-04-15T15:19:38.868 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 6, 2024-04-15T15:19:38.868 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 20, 2024-04-15T15:19:38.868 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 20, 2024-04-15T15:19:38.868 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 19.509216938000002 2024-04-15T15:19:38.868 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-04-15T15:19:38.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.860+0000 7f7ecf1b5640 1 -- 172.21.15.105:0/3038865120 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f7eb406c920 msgr2=0x7f7eb406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:38.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.860+0000 7f7ecf1b5640 1 --2- 172.21.15.105:0/3038865120 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f7eb406c920 0x7f7eb406ede0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8007fd0 tx=0x7f7eb8007de0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.860+0000 7f7ecf1b5640 1 -- 172.21.15.105:0/3038865120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7ec809b780 msgr2=0x7f7ec80ca250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:38.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.860+0000 7f7ecf1b5640 1 --2- 172.21.15.105:0/3038865120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7ec809b780 0x7f7ec80ca250 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc00ebe0 tx=0x7f7ebc00c520 comp rx=0 tx=0).stop 2024-04-15T15:19:38.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.860+0000 7f7ecf1b5640 1 -- 172.21.15.105:0/3038865120 shutdown_connections 2024-04-15T15:19:38.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.860+0000 7f7ecf1b5640 1 --2- 172.21.15.105:0/3038865120 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ec80d10f0 0x7f7ec80b7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.860+0000 7f7ecf1b5640 1 --2- 172.21.15.105:0/3038865120 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f7eb406c920 0x7f7eb406ede0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.860+0000 7f7ecf1b5640 1 --2- 172.21.15.105:0/3038865120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7ec809b780 0x7f7ec80ca250 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.860+0000 7f7ecf1b5640 1 --2- 172.21.15.105:0/3038865120 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ec809b150 0x7f7ec80cc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.860+0000 7f7ecf1b5640 1 -- 172.21.15.105:0/3038865120 >> 172.21.15.105:0/3038865120 conn(0x7f7ec80d2570 msgr2=0x7f7ec806d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:38.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.860+0000 7f7ecf1b5640 1 -- 172.21.15.105:0/3038865120 shutdown_connections 2024-04-15T15:19:38.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:38.860+0000 7f7ecf1b5640 1 -- 172.21.15.105:0/3038865120 wait complete. 2024-04-15T15:19:38.872 INFO:teuthology.run_tasks:Running task sequential... 2024-04-15T15:19:38.882 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-15T15:19:38.882 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-15T15:19:38.882 INFO:teuthology.task.print:Enabling mgr modules 2024-04-15T15:19:38.882 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-15T15:19:38.909 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-15T15:19:38.909 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-15T15:19:38.909 INFO:tasks.ceph_fuse:client.0 config is {} 2024-04-15T15:19:38.909 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-15T15:19:38.909 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-15T15:19:38.909 INFO:tasks.ceph_fuse:client.2 config is {} 2024-04-15T15:19:38.909 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-15T15:19:38.909 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-15T15:19:38.910 INFO:tasks.ceph_fuse:client.1 config is {} 2024-04-15T15:19:38.910 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-15T15:19:38.910 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-15T15:19:38.910 INFO:tasks.ceph_fuse:client.3 config is {} 2024-04-15T15:19:38.910 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-15T15:19:38.910 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-15T15:19:38.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:38.910 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2024-04-15T15:19:38.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:38.926 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link delete ceph-brx 2024-04-15T15:19:38.996 INFO:teuthology.orchestra.run.smithi105.stderr:Cannot find device "ceph-brx" 2024-04-15T15:19:38.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-15T15:19:38.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:38.998 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list 2024-04-15T15:19:39.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:39.017 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link delete ceph-brx 2024-04-15T15:19:39.094 INFO:teuthology.orchestra.run.smithi154.stderr:Cannot find device "ceph-brx" 2024-04-15T15:19:39.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-15T15:19:39.096 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-15T15:19:39.096 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-15T15:19:39.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.197+0000 7f43d1172640 1 Processor -- start 2024-04-15T15:19:39.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.197+0000 7f43d1172640 1 -- start start 2024-04-15T15:19:39.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.198+0000 7f43d1172640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f43cc06c0d0 0x7f43cc166570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:39.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.198+0000 7f43d1172640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f43cc0b3c80 0x7f43cc168ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:39.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.198+0000 7f43d1172640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43cc0c8270 con 0x7f43cc0d0540 2024-04-15T15:19:39.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.198+0000 7f43d1172640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f43cc0d0920 con 0x7f43cc06c0d0 2024-04-15T15:19:39.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.198+0000 7f43d1172640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f43cc0c7cb0 con 0x7f43cc0b3c80 2024-04-15T15:19:39.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.198+0000 7f43c259c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f43cc06c0d0 0x7f43cc166570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:39.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.198+0000 7f43c259c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f43cc06c0d0 0x7f43cc166570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:40120/0 (socket says 172.21.15.105:40120) 2024-04-15T15:19:39.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.198+0000 7f43c259c640 1 -- 172.21.15.105:0/87761254 learned_addr learned my addr 172.21.15.105:0/87761254 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:39.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.198+0000 7f43cb59e640 1 --2- 172.21.15.105:0/87761254 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f43cc0b3c80 0x7f43cc168ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:39.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.198+0000 7f43c259c640 1 -- 172.21.15.105:0/87761254 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f43cc0b3c80 msgr2=0x7f43cc168ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:39.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.198+0000 7f43c259c640 1 --2- 172.21.15.105:0/87761254 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f43cc0b3c80 0x7f43cc168ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:39.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.198+0000 7f43c259c640 1 -- 172.21.15.105:0/87761254 >> v1:172.21.15.154:6789/0 conn(0x7f43cc0d0540 legacy=0x7f43cc06b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:39.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.198+0000 7f43c259c640 1 -- 172.21.15.105:0/87761254 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43cc0c7f90 con 0x7f43cc06c0d0 2024-04-15T15:19:39.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.199+0000 7f43c259c640 1 --2- 172.21.15.105:0/87761254 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43cc06c0d0 0x7f43cc166570 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f43b4009870 tx=0x7f43b402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84aa89a59dbdcdf7 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:39.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.199+0000 7f43ca59c640 1 -- 172.21.15.105:0/87761254 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b4034070 con 0x7f43cc06c0d0 2024-04-15T15:19:39.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.199+0000 7f43ca59c640 1 -- 172.21.15.105:0/87761254 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43b4030840 con 0x7f43cc06c0d0 2024-04-15T15:19:39.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.199+0000 7f43ca59c640 1 -- 172.21.15.105:0/87761254 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b4030c10 con 0x7f43cc06c0d0 2024-04-15T15:19:39.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.200+0000 7f43d1172640 1 -- 172.21.15.105:0/87761254 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43cc06c0d0 msgr2=0x7f43cc166570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:39.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.200+0000 7f43d1172640 1 --2- 172.21.15.105:0/87761254 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43cc06c0d0 0x7f43cc166570 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f43b4009870 tx=0x7f43b402b980 comp rx=0 tx=0).stop 2024-04-15T15:19:39.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.200+0000 7f43d1172640 1 -- 172.21.15.105:0/87761254 shutdown_connections 2024-04-15T15:19:39.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.200+0000 7f43d1172640 1 --2- 172.21.15.105:0/87761254 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f43cc0b3c80 0x7f43cc168ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:39.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.200+0000 7f43d1172640 1 --2- 172.21.15.105:0/87761254 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43cc06c0d0 0x7f43cc166570 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:39.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.200+0000 7f43d1172640 1 -- 172.21.15.105:0/87761254 >> 172.21.15.105:0/87761254 conn(0x7f43cc09aa40 msgr2=0x7f43cc0b30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:39.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.200+0000 7f43d1172640 1 -- 172.21.15.105:0/87761254 shutdown_connections 2024-04-15T15:19:39.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.200+0000 7f43d1172640 1 -- 172.21.15.105:0/87761254 wait complete. 2024-04-15T15:19:39.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.201+0000 7f43d1172640 1 Processor -- start 2024-04-15T15:19:39.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.201+0000 7f43d1172640 1 -- start start 2024-04-15T15:19:39.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.202+0000 7f43d1172640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43cc06c0d0 0x7f43cc15af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:39.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.202+0000 7f43d1172640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43cc0b3c80 0x7f43cc15ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:39.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.202+0000 7f43d1172640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f43cc0d0540 0x7f43cc15f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:39.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.202+0000 7f43d1172640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f43cc070e40 con 0x7f43cc06c0d0 2024-04-15T15:19:39.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.202+0000 7f43d1172640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f43cc06f180 con 0x7f43cc0b3c80 2024-04-15T15:19:39.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.202+0000 7f43d1172640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f43cc070fc0 con 0x7f43cc0d0540 2024-04-15T15:19:39.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.202+0000 7f43cad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43cc06c0d0 0x7f43cc15af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:39.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.202+0000 7f43cad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43cc06c0d0 0x7f43cc15af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:40132/0 (socket says 172.21.15.105:40132) 2024-04-15T15:19:39.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.202+0000 7f43cad9d640 1 -- 172.21.15.105:0/3971351888 learned_addr learned my addr 172.21.15.105:0/3971351888 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:39.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.202+0000 7f43c259c640 1 --2- 172.21.15.105:0/3971351888 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43cc0b3c80 0x7f43cc15ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:39.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.202+0000 7f43cb59e640 1 --2- 172.21.15.105:0/3971351888 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f43cc0d0540 0x7f43cc15f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:39.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.203+0000 7f43cad9d640 1 -- 172.21.15.105:0/3971351888 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f43cc0d0540 msgr2=0x7f43cc15f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:39.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.203+0000 7f43cad9d640 1 --2- 172.21.15.105:0/3971351888 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f43cc0d0540 0x7f43cc15f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:39.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.203+0000 7f43cad9d640 1 -- 172.21.15.105:0/3971351888 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43cc0b3c80 msgr2=0x7f43cc15ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:39.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.203+0000 7f43cad9d640 1 --2- 172.21.15.105:0/3971351888 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43cc0b3c80 0x7f43cc15ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:39.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.203+0000 7f43cad9d640 1 -- 172.21.15.105:0/3971351888 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43cc15b4a0 con 0x7f43cc06c0d0 2024-04-15T15:19:39.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.203+0000 7f43cb59e640 1 --2- 172.21.15.105:0/3971351888 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f43cc0d0540 0x7f43cc15f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:19:39.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.203+0000 7f43cad9d640 1 --2- 172.21.15.105:0/3971351888 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43cc06c0d0 0x7f43cc15af60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f43bc00c400 tx=0x7f43bc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:39.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.203+0000 7f43c3fff640 1 -- 172.21.15.105:0/3971351888 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43bc0052d0 con 0x7f43cc06c0d0 2024-04-15T15:19:39.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.203+0000 7f43c3fff640 1 -- 172.21.15.105:0/3971351888 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43bc005490 con 0x7f43cc06c0d0 2024-04-15T15:19:39.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.203+0000 7f43c3fff640 1 -- 172.21.15.105:0/3971351888 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43bc007260 con 0x7f43cc06c0d0 2024-04-15T15:19:39.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.203+0000 7f43d1172640 1 -- 172.21.15.105:0/3971351888 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43cc09f270 con 0x7f43cc06c0d0 2024-04-15T15:19:39.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.203+0000 7f43d1172640 1 -- 172.21.15.105:0/3971351888 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43cc15fb00 con 0x7f43cc06c0d0 2024-04-15T15:19:39.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.204+0000 7f43d1172640 1 -- 172.21.15.105:0/3971351888 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43cc068a40 con 0x7f43cc06c0d0 2024-04-15T15:19:39.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.204+0000 7f43c3fff640 1 -- 172.21.15.105:0/3971351888 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f43bc007400 con 0x7f43cc06c0d0 2024-04-15T15:19:39.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.205+0000 7f43c3fff640 1 --2- 172.21.15.105:0/3971351888 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f43ac06c8f0 0x7f43ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:39.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.205+0000 7f43c3fff640 1 -- 172.21.15.105:0/3971351888 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f43bc08b030 con 0x7f43cc06c0d0 2024-04-15T15:19:39.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.205+0000 7f43c259c640 1 --2- 172.21.15.105:0/3971351888 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f43ac06c8f0 0x7f43ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:39.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.205+0000 7f43c259c640 1 --2- 172.21.15.105:0/3971351888 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f43ac06c8f0 0x7f43ac06edb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f43b4036000 tx=0x7f43b403b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:39.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.209+0000 7f43c3fff640 1 -- 172.21.15.105:0/3971351888 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f43bc058e10 con 0x7f43cc06c0d0 2024-04-15T15:19:39.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.422+0000 7f43d1172640 1 -- 172.21.15.105:0/3971351888 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f43cc0691f0 con 0x7f43cc06c0d0 2024-04-15T15:19:39.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.422+0000 7f43c3fff640 1 -- 172.21.15.105:0/3971351888 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) ==== 53+0+73 (secure 0 0 0) 0x7f43bc05d110 con 0x7f43cc06c0d0 2024-04-15T15:19:39.424 INFO:teuthology.orchestra.run.smithi105.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-15T15:19:39.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.424+0000 7f43d1172640 1 -- 172.21.15.105:0/3971351888 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f43ac06c8f0 msgr2=0x7f43ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:39.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.424+0000 7f43d1172640 1 --2- 172.21.15.105:0/3971351888 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f43ac06c8f0 0x7f43ac06edb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f43b4036000 tx=0x7f43b403b040 comp rx=0 tx=0).stop 2024-04-15T15:19:39.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.424+0000 7f43d1172640 1 -- 172.21.15.105:0/3971351888 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43cc06c0d0 msgr2=0x7f43cc15af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:39.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.424+0000 7f43d1172640 1 --2- 172.21.15.105:0/3971351888 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43cc06c0d0 0x7f43cc15af60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f43bc00c400 tx=0x7f43bc00c8c0 comp rx=0 tx=0).stop 2024-04-15T15:19:39.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.424+0000 7f43d1172640 1 -- 172.21.15.105:0/3971351888 shutdown_connections 2024-04-15T15:19:39.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.424+0000 7f43d1172640 1 --2- 172.21.15.105:0/3971351888 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f43cc0d0540 0x7f43cc15f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:39.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.424+0000 7f43d1172640 1 --2- 172.21.15.105:0/3971351888 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f43ac06c8f0 0x7f43ac06edb0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:39.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.424+0000 7f43d1172640 1 --2- 172.21.15.105:0/3971351888 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43cc0b3c80 0x7f43cc15ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:39.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.424+0000 7f43d1172640 1 --2- 172.21.15.105:0/3971351888 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43cc06c0d0 0x7f43cc15af60 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:39.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.424+0000 7f43d1172640 1 -- 172.21.15.105:0/3971351888 >> 172.21.15.105:0/3971351888 conn(0x7f43cc09aa40 msgr2=0x7f43cc0af980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:39.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.424+0000 7f43d1172640 1 -- 172.21.15.105:0/3971351888 shutdown_connections 2024-04-15T15:19:39.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39.424+0000 7f43d1172640 1 -- 172.21.15.105:0/3971351888 wait complete. 2024-04-15T15:19:39.437 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-15T15:19:39.437 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-15T15:19:39.437 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi105.front.sepia.ceph.com 2024-04-15T15:19:39.437 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-15T15:19:39.437 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-15T15:19:39.437 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-15T15:19:39.437 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-15T15:19:39.437 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-15T15:19:39.437 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-15T15:19:39.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:39.438 DEBUG:teuthology.orchestra.run.smithi105:> ip addr 2024-04-15T15:19:39.453 INFO:teuthology.orchestra.run.smithi105.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-15T15:19:39.453 INFO:teuthology.orchestra.run.smithi105.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-15T15:19:39.453 INFO:teuthology.orchestra.run.smithi105.stdout: inet 127.0.0.1/8 scope host lo 2024-04-15T15:19:39.453 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:39.453 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 ::1/128 scope host 2024-04-15T15:19:39.453 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:39.454 INFO:teuthology.orchestra.run.smithi105.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:19:39.454 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b8 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:19:39.454 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f0 2024-04-15T15:19:39.454 INFO:teuthology.orchestra.run.smithi105.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:19:39.454 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b9 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:19:39.454 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f1 2024-04-15T15:19:39.454 INFO:teuthology.orchestra.run.smithi105.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:19:39.454 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:28 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:19:39.454 INFO:teuthology.orchestra.run.smithi105.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-15T15:19:39.454 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:29 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:19:39.454 INFO:teuthology.orchestra.run.smithi105.stdout: inet 172.21.15.105/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-15T15:19:39.454 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft 42708sec preferred_lft 42708sec 2024-04-15T15:19:39.454 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::ec4:7aff:fe8f:d429/64 scope link 2024-04-15T15:19:39.455 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:39.455 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-15T15:19:39.455 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:19:39.455 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-04-15T15:19:39.455 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link add name ceph-brx type bridge 2024-04-15T15:19:39.455 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip addr flush dev ceph-brx 2024-04-15T15:19:39.455 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set ceph-brx up 2024-04-15T15:19:39.456 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-15T15:19:39.456 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-04-15T15:19:39.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:19:39.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:19:39.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:39.673 DEBUG:teuthology.orchestra.run.smithi105:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-15T15:19:39.702 INFO:teuthology.orchestra.run.smithi105.stdout:1 2024-04-15T15:19:39.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:39.704 DEBUG:teuthology.orchestra.run.smithi105:> route 2024-04-15T15:19:39.773 INFO:teuthology.orchestra.run.smithi105.stdout:Kernel IP routing table 2024-04-15T15:19:39.774 INFO:teuthology.orchestra.run.smithi105.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-15T15:19:39.774 INFO:teuthology.orchestra.run.smithi105.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-15T15:19:39.774 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-15T15:19:39.774 INFO:teuthology.orchestra.run.smithi105.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-15T15:19:39.774 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:19:39.774 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-04-15T15:19:39.775 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-15T15:19:39.775 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-15T15:19:39.775 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-15T15:19:39.775 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-04-15T15:19:39.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:19:40.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:19:40.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:40.073 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2024-04-15T15:19:40.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:40.089 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list-id 2024-04-15T15:19:40.144 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:19:40.145 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-04-15T15:19:40.145 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-15T15:19:40.145 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-15T15:19:40.145 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-04-15T15:19:40.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:19:40.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:19:40.269 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-15T15:19:40.269 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:19:40.269 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-04-15T15:19:40.269 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-15T15:19:40.269 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-15T15:19:40.270 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-15T15:19:40.270 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-15T15:19:40.270 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-15T15:19:40.270 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-04-15T15:19:40.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:19:40.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:19:40.406 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:19:40.407 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-04-15T15:19:40.407 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set brx.0 up 2024-04-15T15:19:40.407 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set dev brx.0 master ceph-brx 2024-04-15T15:19:40.407 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-04-15T15:19:40.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:19:40.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:19:40.491 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-15T15:19:40.491 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-15T15:19:40.491 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-15T15:19:40.519 INFO:teuthology.orchestra.run.smithi105.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-15T15:19:40.519 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-15T15:19:40.520 DEBUG:teuthology.orchestra.run.smithi105:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-15T15:19:40.575 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2024-04-15T15:19:40.644 DEBUG:teuthology.orchestra.run.smithi105:> cat /proc/self/mounts | awk '{print $2}' 2024-04-15T15:19:40.701 INFO:teuthology.orchestra.run.smithi105.stdout:/proc 2024-04-15T15:19:40.701 INFO:teuthology.orchestra.run.smithi105.stdout:/sys 2024-04-15T15:19:40.701 INFO:teuthology.orchestra.run.smithi105.stdout:/dev 2024-04-15T15:19:40.701 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/security 2024-04-15T15:19:40.701 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/shm 2024-04-15T15:19:40.701 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/pts 2024-04-15T15:19:40.701 INFO:teuthology.orchestra.run.smithi105.stdout:/run 2024-04-15T15:19:40.701 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/cgroup 2024-04-15T15:19:40.701 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/pstore 2024-04-15T15:19:40.702 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/bpf 2024-04-15T15:19:40.702 INFO:teuthology.orchestra.run.smithi105.stdout:/ 2024-04-15T15:19:40.702 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/selinux 2024-04-15T15:19:40.702 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/sys/fs/binfmt_misc 2024-04-15T15:19:40.702 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/mqueue 2024-04-15T15:19:40.702 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/hugepages 2024-04-15T15:19:40.702 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/debug 2024-04-15T15:19:40.702 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/tracing 2024-04-15T15:19:40.702 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/fuse/connections 2024-04-15T15:19:40.702 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/config 2024-04-15T15:19:40.702 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-sysctl.service 2024-04-15T15:19:40.702 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-15T15:19:40.703 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-15T15:19:40.703 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/nfs/rpc_pipefs 2024-04-15T15:19:40.703 INFO:teuthology.orchestra.run.smithi105.stdout:/run/user/1000 2024-04-15T15:19:40.703 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph 2024-04-15T15:19:40.703 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-0 2024-04-15T15:19:40.703 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-1 2024-04-15T15:19:40.703 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-2 2024-04-15T15:19:40.703 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-3 2024-04-15T15:19:40.703 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns 2024-04-15T15:19:40.703 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-15T15:19:40.703 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-15T15:19:40.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:40.704 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2024-04-15T15:19:40.757 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-15T15:19:40.757 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-15T15:19:40.799 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2024-04-15T15:19:40.826 DEBUG:teuthology.orchestra.run.smithi105:> cat /proc/self/mounts | awk '{print $2}' 2024-04-15T15:19:40.882 INFO:teuthology.orchestra.run.smithi105.stdout:/proc 2024-04-15T15:19:40.883 INFO:teuthology.orchestra.run.smithi105.stdout:/sys 2024-04-15T15:19:40.883 INFO:teuthology.orchestra.run.smithi105.stdout:/dev 2024-04-15T15:19:40.883 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/security 2024-04-15T15:19:40.883 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/shm 2024-04-15T15:19:40.883 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/pts 2024-04-15T15:19:40.883 INFO:teuthology.orchestra.run.smithi105.stdout:/run 2024-04-15T15:19:40.883 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/cgroup 2024-04-15T15:19:40.883 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/pstore 2024-04-15T15:19:40.883 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/bpf 2024-04-15T15:19:40.883 INFO:teuthology.orchestra.run.smithi105.stdout:/ 2024-04-15T15:19:40.883 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/selinux 2024-04-15T15:19:40.883 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/sys/fs/binfmt_misc 2024-04-15T15:19:40.883 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/mqueue 2024-04-15T15:19:40.883 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/hugepages 2024-04-15T15:19:40.884 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/debug 2024-04-15T15:19:40.884 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/tracing 2024-04-15T15:19:40.884 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/fuse/connections 2024-04-15T15:19:40.884 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/config 2024-04-15T15:19:40.884 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-sysctl.service 2024-04-15T15:19:40.884 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-15T15:19:40.884 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-15T15:19:40.884 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/nfs/rpc_pipefs 2024-04-15T15:19:40.884 INFO:teuthology.orchestra.run.smithi105.stdout:/run/user/1000 2024-04-15T15:19:40.884 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph 2024-04-15T15:19:40.884 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-0 2024-04-15T15:19:40.884 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-1 2024-04-15T15:19:40.884 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-2 2024-04-15T15:19:40.885 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-3 2024-04-15T15:19:40.885 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns 2024-04-15T15:19:40.885 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-15T15:19:40.885 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-15T15:19:40.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:40.885 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2024-04-15T15:19:40.887 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi105.stderr:2024-04-15T15:19:40.886+0000 7f0419a44480 -1 init, newargv = 0x55b0b230b4c0 newargc=15 2024-04-15T15:19:40.888 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi105.stderr:ceph-fuse[46367]: starting ceph client 2024-04-15T15:19:40.892 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi105.stderr:ceph-fuse[46367]: starting fuse 2024-04-15T15:19:40.902 INFO:teuthology.orchestra.run.smithi105.stdout:42 2024-04-15T15:19:40.903 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-15T15:19:40.903 DEBUG:teuthology.orchestra.run.smithi105:> sudo stdin-killer -- python3 -c ' 2024-04-15T15:19:40.903 DEBUG:teuthology.orchestra.run.smithi105:> import glob 2024-04-15T15:19:40.903 DEBUG:teuthology.orchestra.run.smithi105:> import re 2024-04-15T15:19:40.903 DEBUG:teuthology.orchestra.run.smithi105:> import os 2024-04-15T15:19:40.903 DEBUG:teuthology.orchestra.run.smithi105:> import subprocess 2024-04-15T15:19:40.903 DEBUG:teuthology.orchestra.run.smithi105:> 2024-04-15T15:19:40.903 DEBUG:teuthology.orchestra.run.smithi105:> def _find_admin_socket(client_name): 2024-04-15T15:19:40.903 DEBUG:teuthology.orchestra.run.smithi105:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-15T15:19:40.903 DEBUG:teuthology.orchestra.run.smithi105:> files = glob.glob(asok_path) 2024-04-15T15:19:40.903 DEBUG:teuthology.orchestra.run.smithi105:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-15T15:19:40.903 DEBUG:teuthology.orchestra.run.smithi105:> 2024-04-15T15:19:40.904 DEBUG:teuthology.orchestra.run.smithi105:> # Given a non-glob path, it better be there 2024-04-15T15:19:40.904 DEBUG:teuthology.orchestra.run.smithi105:> if "*" not in asok_path: 2024-04-15T15:19:40.904 DEBUG:teuthology.orchestra.run.smithi105:> assert(len(files) == 1) 2024-04-15T15:19:40.904 DEBUG:teuthology.orchestra.run.smithi105:> return files[0] 2024-04-15T15:19:40.904 DEBUG:teuthology.orchestra.run.smithi105:> 2024-04-15T15:19:40.904 DEBUG:teuthology.orchestra.run.smithi105:> for f in files: 2024-04-15T15:19:40.904 DEBUG:teuthology.orchestra.run.smithi105:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-15T15:19:40.904 DEBUG:teuthology.orchestra.run.smithi105:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-15T15:19:40.904 DEBUG:teuthology.orchestra.run.smithi105:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-15T15:19:40.904 DEBUG:teuthology.orchestra.run.smithi105:> contents = proc_f.read() 2024-04-15T15:19:40.904 DEBUG:teuthology.orchestra.run.smithi105:> if mountpoint in contents: 2024-04-15T15:19:40.904 DEBUG:teuthology.orchestra.run.smithi105:> return f 2024-04-15T15:19:40.904 DEBUG:teuthology.orchestra.run.smithi105:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-15T15:19:40.904 DEBUG:teuthology.orchestra.run.smithi105:> 2024-04-15T15:19:40.904 DEBUG:teuthology.orchestra.run.smithi105:> print(_find_admin_socket("client.0")) 2024-04-15T15:19:40.905 DEBUG:teuthology.orchestra.run.smithi105:> ' 2024-04-15T15:19:41.024 INFO:teuthology.orchestra.run.smithi105.stdout:/var/run/ceph/ceph-client.0.46367.asok 2024-04-15T15:19:41.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:19:41.034 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.46367.asok 2024-04-15T15:19:41.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:41.035 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.46367.asok status 2024-04-15T15:19:41.172 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-04-15T15:19:41.172 INFO:teuthology.orchestra.run.smithi105.stdout: "metadata": { 2024-04-15T15:19:41.172 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_sha1": "9bf1c4ba967dd24077be6e5b87edda4ca87a6e14", 2024-04-15T15:19:41.173 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_version": "ceph version 19.0.0-2861-g9bf1c4ba (9bf1c4ba967dd24077be6e5b87edda4ca87a6e14) squid (dev)", 2024-04-15T15:19:41.173 INFO:teuthology.orchestra.run.smithi105.stdout: "entity_id": "0", 2024-04-15T15:19:41.173 INFO:teuthology.orchestra.run.smithi105.stdout: "hostname": "smithi105", 2024-04-15T15:19:41.173 INFO:teuthology.orchestra.run.smithi105.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-15T15:19:41.173 INFO:teuthology.orchestra.run.smithi105.stdout: "pid": "46367", 2024-04-15T15:19:41.173 INFO:teuthology.orchestra.run.smithi105.stdout: "root": "/" 2024-04-15T15:19:41.173 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-04-15T15:19:41.173 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_count": 0, 2024-04-15T15:19:41.173 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_pinned_count": 0, 2024-04-15T15:19:41.173 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 4668, 2024-04-15T15:19:41.173 INFO:teuthology.orchestra.run.smithi105.stdout: "inst": { 2024-04-15T15:19:41.173 INFO:teuthology.orchestra.run.smithi105.stdout: "name": { 2024-04-15T15:19:41.173 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "client", 2024-04-15T15:19:41.173 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 4668 2024-04-15T15:19:41.174 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-04-15T15:19:41.174 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": { 2024-04-15T15:19:41.174 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-04-15T15:19:41.174 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "192.168.0.1:0", 2024-04-15T15:19:41.174 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 1466503458 2024-04-15T15:19:41.174 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-04-15T15:19:41.174 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-04-15T15:19:41.174 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": { 2024-04-15T15:19:41.174 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-04-15T15:19:41.174 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "192.168.0.1:0", 2024-04-15T15:19:41.175 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 1466503458 2024-04-15T15:19:41.175 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-04-15T15:19:41.175 INFO:teuthology.orchestra.run.smithi105.stdout: "inst_str": "client.4668 192.168.0.1:0/1466503458", 2024-04-15T15:19:41.175 INFO:teuthology.orchestra.run.smithi105.stdout: "addr_str": "192.168.0.1:0/1466503458", 2024-04-15T15:19:41.175 INFO:teuthology.orchestra.run.smithi105.stdout: "inode_count": 1, 2024-04-15T15:19:41.175 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_epoch": 6, 2024-04-15T15:19:41.175 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_epoch": 20, 2024-04-15T15:19:41.175 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_epoch_barrier": 0, 2024-04-15T15:19:41.176 INFO:teuthology.orchestra.run.smithi105.stdout: "blocklisted": false, 2024-04-15T15:19:41.176 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs" 2024-04-15T15:19:41.176 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-04-15T15:19:41.181 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-15T15:19:41.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.282+0000 7f61ec150640 1 Processor -- start 2024-04-15T15:19:41.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.282+0000 7f61ec150640 1 -- start start 2024-04-15T15:19:41.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.283+0000 7f61ec150640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f61e40ce7e0 0x7f61e415cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:41.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.283+0000 7f61ec150640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f61e40b3160 0x7f61e40cebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:41.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.283+0000 7f61ec150640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f61e40cd320 0x7f61e40cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:41.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.283+0000 7f61ec150640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f61e415d280 con 0x7f61e40cd320 2024-04-15T15:19:41.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.283+0000 7f61ec150640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f61e40cf100 con 0x7f61e40ce7e0 2024-04-15T15:19:41.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.283+0000 7f61ec150640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f61e415d520 con 0x7f61e40b3160 2024-04-15T15:19:41.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.283+0000 7f61ea6ed640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f61e40cd320 0x7f61e40cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:41.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.283+0000 7f61ea6ed640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f61e40cd320 0x7f61e40cd770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51536/0 (socket says 172.21.15.105:51536) 2024-04-15T15:19:41.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.283+0000 7f61ea6ed640 1 -- 172.21.15.105:0/2350035176 learned_addr learned my addr 172.21.15.105:0/2350035176 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:41.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.283+0000 7f61e9eec640 1 --2- 172.21.15.105:0/2350035176 >> v2:172.21.15.154:3300/0 conn(0x7f61e40ce7e0 0x7f61e415cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:41.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.283+0000 7f61e96eb640 1 --2- 172.21.15.105:0/2350035176 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f61e40b3160 0x7f61e40cebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:41.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.283+0000 7f61ea6ed640 1 -- 172.21.15.105:0/2350035176 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f61e40b3160 msgr2=0x7f61e40cebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:41.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.283+0000 7f61ea6ed640 1 --2- 172.21.15.105:0/2350035176 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f61e40b3160 0x7f61e40cebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:41.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.283+0000 7f61ea6ed640 1 -- 172.21.15.105:0/2350035176 >> v2:172.21.15.154:3300/0 conn(0x7f61e40ce7e0 msgr2=0x7f61e415cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:41.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.283+0000 7f61ea6ed640 1 --2- 172.21.15.105:0/2350035176 >> v2:172.21.15.154:3300/0 conn(0x7f61e40ce7e0 0x7f61e415cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:41.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.283+0000 7f61ea6ed640 1 -- 172.21.15.105:0/2350035176 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61e415d800 con 0x7f61e40cd320 2024-04-15T15:19:41.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.284+0000 7f61ea6ed640 1 --2- 172.21.15.105:0/2350035176 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61e40cd320 0x7f61e40cd770 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f61e406e000 tx=0x7f61dc02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5bc119c85639877 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:41.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.284+0000 7f61e8eea640 1 -- 172.21.15.105:0/2350035176 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc02ca70 con 0x7f61e40cd320 2024-04-15T15:19:41.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.284+0000 7f61e8eea640 1 -- 172.21.15.105:0/2350035176 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61dc02cc10 con 0x7f61e40cd320 2024-04-15T15:19:41.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.284+0000 7f61e8eea640 1 -- 172.21.15.105:0/2350035176 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc0343f0 con 0x7f61e40cd320 2024-04-15T15:19:41.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.285+0000 7f61ec150640 1 -- 172.21.15.105:0/2350035176 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61e40cd320 msgr2=0x7f61e40cd770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:41.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.285+0000 7f61ec150640 1 --2- 172.21.15.105:0/2350035176 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61e40cd320 0x7f61e40cd770 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f61e406e000 tx=0x7f61dc02bed0 comp rx=0 tx=0).stop 2024-04-15T15:19:41.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.285+0000 7f61ec150640 1 -- 172.21.15.105:0/2350035176 shutdown_connections 2024-04-15T15:19:41.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.285+0000 7f61ec150640 1 --2- 172.21.15.105:0/2350035176 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61e40cd320 0x7f61e40cd770 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:41.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.285+0000 7f61ec150640 1 --2- 172.21.15.105:0/2350035176 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f61e40b3160 0x7f61e40cebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:41.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.285+0000 7f61ec150640 1 --2- 172.21.15.105:0/2350035176 >> v2:172.21.15.154:3300/0 conn(0x7f61e40ce7e0 0x7f61e415cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:41.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.285+0000 7f61ec150640 1 -- 172.21.15.105:0/2350035176 >> 172.21.15.105:0/2350035176 conn(0x7f61e40b2640 msgr2=0x7f61e40b2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:41.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.285+0000 7f61ec150640 1 -- 172.21.15.105:0/2350035176 shutdown_connections 2024-04-15T15:19:41.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.285+0000 7f61ec150640 1 -- 172.21.15.105:0/2350035176 wait complete. 2024-04-15T15:19:41.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.286+0000 7f61ec150640 1 Processor -- start 2024-04-15T15:19:41.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.286+0000 7f61ec150640 1 -- start start 2024-04-15T15:19:41.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.287+0000 7f61ec150640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61e40b3160 0x7f61e40ad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:41.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.287+0000 7f61ec150640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61e40cd320 0x7f61e40af190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:41.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.287+0000 7f61ec150640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f61e40ce7e0 0x7f61e40b4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:41.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.287+0000 7f61ec150640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f61e415e1b0 con 0x7f61e40cd320 2024-04-15T15:19:41.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.287+0000 7f61ec150640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f61e415ddc0 con 0x7f61e40b3160 2024-04-15T15:19:41.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.287+0000 7f61ec150640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f61e4145930 con 0x7f61e40ce7e0 2024-04-15T15:19:41.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.287+0000 7f61e96eb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61e40cd320 0x7f61e40af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:41.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.287+0000 7f61e96eb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61e40cd320 0x7f61e40af190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51542/0 (socket says 172.21.15.105:51542) 2024-04-15T15:19:41.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.287+0000 7f61e96eb640 1 -- 172.21.15.105:0/2189323237 learned_addr learned my addr 172.21.15.105:0/2189323237 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:41.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.287+0000 7f61e9eec640 1 --2- 172.21.15.105:0/2189323237 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61e40b3160 0x7f61e40ad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:41.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.287+0000 7f61ea6ed640 1 --2- 172.21.15.105:0/2189323237 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f61e40ce7e0 0x7f61e40b4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:41.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.287+0000 7f61e96eb640 1 -- 172.21.15.105:0/2189323237 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f61e40ce7e0 msgr2=0x7f61e40b4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:41.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.287+0000 7f61e96eb640 1 --2- 172.21.15.105:0/2189323237 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f61e40ce7e0 0x7f61e40b4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:41.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.287+0000 7f61e96eb640 1 -- 172.21.15.105:0/2189323237 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61e40b3160 msgr2=0x7f61e40ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:41.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.287+0000 7f61e96eb640 1 --2- 172.21.15.105:0/2189323237 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61e40b3160 0x7f61e40ad8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:41.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.287+0000 7f61e96eb640 1 -- 172.21.15.105:0/2189323237 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61e40b5270 con 0x7f61e40cd320 2024-04-15T15:19:41.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.287+0000 7f61e9eec640 1 --2- 172.21.15.105:0/2189323237 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61e40b3160 0x7f61e40ad8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:19:41.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.287+0000 7f61e96eb640 1 --2- 172.21.15.105:0/2189323237 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61e40cd320 0x7f61e40af190 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f61d400e870 tx=0x7f61d400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:41.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.287+0000 7f61da7fc640 1 -- 172.21.15.105:0/2189323237 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d4017070 con 0x7f61e40cd320 2024-04-15T15:19:41.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.287+0000 7f61da7fc640 1 -- 172.21.15.105:0/2189323237 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61d400ee60 con 0x7f61e40cd320 2024-04-15T15:19:41.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.287+0000 7f61da7fc640 1 -- 172.21.15.105:0/2189323237 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d400cb80 con 0x7f61e40cd320 2024-04-15T15:19:41.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.287+0000 7f61ec150640 1 -- 172.21.15.105:0/2189323237 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61e40b5550 con 0x7f61e40cd320 2024-04-15T15:19:41.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.287+0000 7f61ec150640 1 -- 172.21.15.105:0/2189323237 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61e4141a40 con 0x7f61e40cd320 2024-04-15T15:19:41.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.288+0000 7f61ec150640 1 -- 172.21.15.105:0/2189323237 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61e4142040 con 0x7f61e40cd320 2024-04-15T15:19:41.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.289+0000 7f61da7fc640 1 -- 172.21.15.105:0/2189323237 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f61d401a070 con 0x7f61e40cd320 2024-04-15T15:19:41.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.289+0000 7f61da7fc640 1 --2- 172.21.15.105:0/2189323237 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f61c806c920 0x7f61c806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:41.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.289+0000 7f61da7fc640 1 -- 172.21.15.105:0/2189323237 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f61d408bd00 con 0x7f61e40cd320 2024-04-15T15:19:41.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.289+0000 7f61e9eec640 1 --2- 172.21.15.105:0/2189323237 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f61c806c920 0x7f61c806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:41.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.290+0000 7f61e9eec640 1 --2- 172.21.15.105:0/2189323237 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f61c806c920 0x7f61c806ede0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f61e413feb0 tx=0x7f61e0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:41.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.294+0000 7f61da7fc640 1 -- 172.21.15.105:0/2189323237 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f61e4142040 con 0x7f61e40cd320 2024-04-15T15:19:41.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.511+0000 7f61ec150640 1 -- 172.21.15.105:0/2189323237 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f61e4068a40 con 0x7f61e40cd320 2024-04-15T15:19:41.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.512+0000 7f61da7fc640 1 -- 172.21.15.105:0/2189323237 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) ==== 53+0+73 (secure 0 0 0) 0x7f61d405a130 con 0x7f61e40cd320 2024-04-15T15:19:41.513 INFO:teuthology.orchestra.run.smithi105.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-15T15:19:41.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.513+0000 7f61ec150640 1 -- 172.21.15.105:0/2189323237 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f61c806c920 msgr2=0x7f61c806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:41.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.513+0000 7f61ec150640 1 --2- 172.21.15.105:0/2189323237 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f61c806c920 0x7f61c806ede0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f61e413feb0 tx=0x7f61e0008040 comp rx=0 tx=0).stop 2024-04-15T15:19:41.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.513+0000 7f61ec150640 1 -- 172.21.15.105:0/2189323237 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61e40cd320 msgr2=0x7f61e40af190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:41.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.513+0000 7f61ec150640 1 --2- 172.21.15.105:0/2189323237 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61e40cd320 0x7f61e40af190 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f61d400e870 tx=0x7f61d400ed30 comp rx=0 tx=0).stop 2024-04-15T15:19:41.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.513+0000 7f61ec150640 1 -- 172.21.15.105:0/2189323237 shutdown_connections 2024-04-15T15:19:41.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.513+0000 7f61ec150640 1 --2- 172.21.15.105:0/2189323237 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f61e40ce7e0 0x7f61e40b4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:41.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.513+0000 7f61ec150640 1 --2- 172.21.15.105:0/2189323237 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f61c806c920 0x7f61c806ede0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:41.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.513+0000 7f61ec150640 1 --2- 172.21.15.105:0/2189323237 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61e40cd320 0x7f61e40af190 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:41.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.513+0000 7f61ec150640 1 --2- 172.21.15.105:0/2189323237 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61e40b3160 0x7f61e40ad8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:41.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.513+0000 7f61ec150640 1 -- 172.21.15.105:0/2189323237 >> 172.21.15.105:0/2189323237 conn(0x7f61e40b2640 msgr2=0x7f61e40b1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:41.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.514+0000 7f61ec150640 1 -- 172.21.15.105:0/2189323237 shutdown_connections 2024-04-15T15:19:41.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41.514+0000 7f61ec150640 1 -- 172.21.15.105:0/2189323237 wait complete. 2024-04-15T15:19:41.526 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-15T15:19:41.526 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-15T15:19:41.526 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi105.front.sepia.ceph.com 2024-04-15T15:19:41.526 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-15T15:19:41.526 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-15T15:19:41.526 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-15T15:19:41.526 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-15T15:19:41.526 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-15T15:19:41.526 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-15T15:19:41.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:41.527 DEBUG:teuthology.orchestra.run.smithi105:> ip addr 2024-04-15T15:19:41.542 INFO:teuthology.orchestra.run.smithi105.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-15T15:19:41.542 INFO:teuthology.orchestra.run.smithi105.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-15T15:19:41.543 INFO:teuthology.orchestra.run.smithi105.stdout: inet 127.0.0.1/8 scope host lo 2024-04-15T15:19:41.543 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:41.543 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 ::1/128 scope host 2024-04-15T15:19:41.543 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:41.543 INFO:teuthology.orchestra.run.smithi105.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:19:41.543 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b8 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:19:41.543 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f0 2024-04-15T15:19:41.543 INFO:teuthology.orchestra.run.smithi105.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:19:41.543 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b9 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:19:41.543 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f1 2024-04-15T15:19:41.543 INFO:teuthology.orchestra.run.smithi105.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:19:41.543 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:28 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:19:41.543 INFO:teuthology.orchestra.run.smithi105.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-15T15:19:41.543 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:29 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:19:41.543 INFO:teuthology.orchestra.run.smithi105.stdout: inet 172.21.15.105/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-15T15:19:41.543 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft 42706sec preferred_lft 42706sec 2024-04-15T15:19:41.544 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::ec4:7aff:fe8f:d429/64 scope link 2024-04-15T15:19:41.544 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:41.544 INFO:teuthology.orchestra.run.smithi105.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-15T15:19:41.544 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 9e:e8:fb:17:06:9d brd ff:ff:ff:ff:ff:ff 2024-04-15T15:19:41.544 INFO:teuthology.orchestra.run.smithi105.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-15T15:19:41.544 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:41.544 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::7c5f:a9ff:fe78:3a4f/64 scope link 2024-04-15T15:19:41.544 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:41.544 INFO:teuthology.orchestra.run.smithi105.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-15T15:19:41.544 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 9e:e8:fb:17:06:9d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-15T15:19:41.544 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::9ce8:fbff:fe17:69d/64 scope link tentative 2024-04-15T15:19:41.544 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:41.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:41.545 DEBUG:teuthology.orchestra.run.smithi105:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-15T15:19:41.609 INFO:teuthology.orchestra.run.smithi105.stdout:1 2024-04-15T15:19:41.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:41.611 DEBUG:teuthology.orchestra.run.smithi105:> route 2024-04-15T15:19:41.667 INFO:teuthology.orchestra.run.smithi105.stdout:Kernel IP routing table 2024-04-15T15:19:41.667 INFO:teuthology.orchestra.run.smithi105.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-15T15:19:41.667 INFO:teuthology.orchestra.run.smithi105.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-15T15:19:41.667 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-15T15:19:41.667 INFO:teuthology.orchestra.run.smithi105.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-15T15:19:41.668 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:19:41.668 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-04-15T15:19:41.668 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-15T15:19:41.668 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-15T15:19:41.668 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-15T15:19:41.668 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-04-15T15:19:41.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:19:41.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:19:41.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:41.806 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2024-04-15T15:19:41.821 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-15T15:19:41.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:41.822 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list-id 2024-04-15T15:19:41.877 INFO:teuthology.orchestra.run.smithi105.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-15T15:19:41.877 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:19:41.877 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-04-15T15:19:41.877 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-15T15:19:41.877 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-15T15:19:41.877 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-04-15T15:19:41.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:19:41.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:19:42.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:42.001 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-15T15:19:42.028 INFO:teuthology.orchestra.run.smithi105.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-15T15:19:42.029 INFO:teuthology.orchestra.run.smithi105.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-15T15:19:42.029 INFO:teuthology.orchestra.run.smithi105.stdout: inet 127.0.0.1/8 scope host lo 2024-04-15T15:19:42.029 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:42.029 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 ::1/128 scope host 2024-04-15T15:19:42.029 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:42.029 INFO:teuthology.orchestra.run.smithi105.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-15T15:19:42.029 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 4a:5f:69:3d:9b:17 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-15T15:19:42.029 INFO:teuthology.orchestra.run.smithi105.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-15T15:19:42.029 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:42.029 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::485f:69ff:fe3d:9b17/64 scope link tentative 2024-04-15T15:19:42.029 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:42.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:42.031 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-15T15:19:42.098 INFO:teuthology.orchestra.run.smithi105.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-15T15:19:42.099 INFO:teuthology.orchestra.run.smithi105.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-15T15:19:42.099 INFO:teuthology.orchestra.run.smithi105.stdout: inet 127.0.0.1/8 scope host lo 2024-04-15T15:19:42.099 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:42.099 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 ::1/128 scope host 2024-04-15T15:19:42.099 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:42.099 INFO:teuthology.orchestra.run.smithi105.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-15T15:19:42.099 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 4a:5f:69:3d:9b:17 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-15T15:19:42.099 INFO:teuthology.orchestra.run.smithi105.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-15T15:19:42.099 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:42.099 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::485f:69ff:fe3d:9b17/64 scope link tentative 2024-04-15T15:19:42.099 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:42.100 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-15T15:19:42.101 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:19:42.101 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-04-15T15:19:42.101 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-15T15:19:42.101 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-15T15:19:42.101 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-15T15:19:42.101 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-15T15:19:42.101 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-15T15:19:42.101 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-04-15T15:19:42.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:19:42.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:19:42.269 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:19:42.269 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-04-15T15:19:42.269 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set brx.1 up 2024-04-15T15:19:42.269 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set dev brx.1 master ceph-brx 2024-04-15T15:19:42.269 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-04-15T15:19:42.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:19:42.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:19:42.353 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-04-15T15:19:42.353 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-15T15:19:42.353 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-15T15:19:42.368 INFO:teuthology.orchestra.run.smithi105.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-15T15:19:42.368 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-15T15:19:42.369 DEBUG:teuthology.orchestra.run.smithi105:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-15T15:19:42.424 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2024-04-15T15:19:42.493 DEBUG:teuthology.orchestra.run.smithi105:> cat /proc/self/mounts | awk '{print $2}' 2024-04-15T15:19:42.549 INFO:teuthology.orchestra.run.smithi105.stdout:/proc 2024-04-15T15:19:42.549 INFO:teuthology.orchestra.run.smithi105.stdout:/sys 2024-04-15T15:19:42.549 INFO:teuthology.orchestra.run.smithi105.stdout:/dev 2024-04-15T15:19:42.549 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/security 2024-04-15T15:19:42.550 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/shm 2024-04-15T15:19:42.550 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/pts 2024-04-15T15:19:42.550 INFO:teuthology.orchestra.run.smithi105.stdout:/run 2024-04-15T15:19:42.550 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/cgroup 2024-04-15T15:19:42.550 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/pstore 2024-04-15T15:19:42.550 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/bpf 2024-04-15T15:19:42.550 INFO:teuthology.orchestra.run.smithi105.stdout:/ 2024-04-15T15:19:42.550 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/selinux 2024-04-15T15:19:42.550 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/sys/fs/binfmt_misc 2024-04-15T15:19:42.550 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/mqueue 2024-04-15T15:19:42.550 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/hugepages 2024-04-15T15:19:42.550 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/debug 2024-04-15T15:19:42.550 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/tracing 2024-04-15T15:19:42.550 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/fuse/connections 2024-04-15T15:19:42.551 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/config 2024-04-15T15:19:42.551 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-sysctl.service 2024-04-15T15:19:42.553 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-15T15:19:42.553 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-15T15:19:42.553 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/nfs/rpc_pipefs 2024-04-15T15:19:42.553 INFO:teuthology.orchestra.run.smithi105.stdout:/run/user/1000 2024-04-15T15:19:42.553 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph 2024-04-15T15:19:42.553 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-0 2024-04-15T15:19:42.553 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-1 2024-04-15T15:19:42.553 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-2 2024-04-15T15:19:42.553 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-3 2024-04-15T15:19:42.554 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns 2024-04-15T15:19:42.554 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-15T15:19:42.554 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-15T15:19:42.554 INFO:teuthology.orchestra.run.smithi105.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-15T15:19:42.554 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-15T15:19:42.554 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-15T15:19:42.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:42.555 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2024-04-15T15:19:42.611 INFO:teuthology.orchestra.run.smithi105.stdout:42 2024-04-15T15:19:42.611 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [42] 2024-04-15T15:19:42.611 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-04-15T15:19:42.654 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2024-04-15T15:19:42.681 DEBUG:teuthology.orchestra.run.smithi105:> cat /proc/self/mounts | awk '{print $2}' 2024-04-15T15:19:42.738 INFO:teuthology.orchestra.run.smithi105.stdout:/proc 2024-04-15T15:19:42.738 INFO:teuthology.orchestra.run.smithi105.stdout:/sys 2024-04-15T15:19:42.738 INFO:teuthology.orchestra.run.smithi105.stdout:/dev 2024-04-15T15:19:42.738 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/security 2024-04-15T15:19:42.738 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/shm 2024-04-15T15:19:42.738 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/pts 2024-04-15T15:19:42.738 INFO:teuthology.orchestra.run.smithi105.stdout:/run 2024-04-15T15:19:42.738 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/cgroup 2024-04-15T15:19:42.739 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/pstore 2024-04-15T15:19:42.739 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/bpf 2024-04-15T15:19:42.739 INFO:teuthology.orchestra.run.smithi105.stdout:/ 2024-04-15T15:19:42.739 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/selinux 2024-04-15T15:19:42.739 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/sys/fs/binfmt_misc 2024-04-15T15:19:42.739 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/mqueue 2024-04-15T15:19:42.739 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/hugepages 2024-04-15T15:19:42.739 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/debug 2024-04-15T15:19:42.739 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/tracing 2024-04-15T15:19:42.739 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/fuse/connections 2024-04-15T15:19:42.739 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/config 2024-04-15T15:19:42.739 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-sysctl.service 2024-04-15T15:19:42.739 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-15T15:19:42.739 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-15T15:19:42.739 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/nfs/rpc_pipefs 2024-04-15T15:19:42.740 INFO:teuthology.orchestra.run.smithi105.stdout:/run/user/1000 2024-04-15T15:19:42.740 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph 2024-04-15T15:19:42.740 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-0 2024-04-15T15:19:42.740 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-1 2024-04-15T15:19:42.740 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-2 2024-04-15T15:19:42.740 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-3 2024-04-15T15:19:42.740 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns 2024-04-15T15:19:42.740 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-15T15:19:42.741 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-15T15:19:42.741 INFO:teuthology.orchestra.run.smithi105.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-15T15:19:42.741 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-15T15:19:42.741 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-15T15:19:42.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:42.742 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2024-04-15T15:19:42.743 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi105.stderr:2024-04-15T15:19:42.740+0000 7fc04d246480 -1 init, newargv = 0x55ebb796f390 newargc=15 2024-04-15T15:19:42.743 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi105.stderr:ceph-fuse[46865]: starting ceph client 2024-04-15T15:19:42.746 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi105.stderr:ceph-fuse[46865]: starting fuse 2024-04-15T15:19:42.757 INFO:teuthology.orchestra.run.smithi105.stdout:42 2024-04-15T15:19:42.757 INFO:teuthology.orchestra.run.smithi105.stdout:44 2024-04-15T15:19:42.758 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42, 44] 2024-04-15T15:19:42.758 DEBUG:teuthology.orchestra.run.smithi105:> sudo stdin-killer -- python3 -c ' 2024-04-15T15:19:42.758 DEBUG:teuthology.orchestra.run.smithi105:> import glob 2024-04-15T15:19:42.758 DEBUG:teuthology.orchestra.run.smithi105:> import re 2024-04-15T15:19:42.758 DEBUG:teuthology.orchestra.run.smithi105:> import os 2024-04-15T15:19:42.758 DEBUG:teuthology.orchestra.run.smithi105:> import subprocess 2024-04-15T15:19:42.758 DEBUG:teuthology.orchestra.run.smithi105:> 2024-04-15T15:19:42.758 DEBUG:teuthology.orchestra.run.smithi105:> def _find_admin_socket(client_name): 2024-04-15T15:19:42.758 DEBUG:teuthology.orchestra.run.smithi105:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-04-15T15:19:42.758 DEBUG:teuthology.orchestra.run.smithi105:> files = glob.glob(asok_path) 2024-04-15T15:19:42.758 DEBUG:teuthology.orchestra.run.smithi105:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-04-15T15:19:42.759 DEBUG:teuthology.orchestra.run.smithi105:> 2024-04-15T15:19:42.759 DEBUG:teuthology.orchestra.run.smithi105:> # Given a non-glob path, it better be there 2024-04-15T15:19:42.759 DEBUG:teuthology.orchestra.run.smithi105:> if "*" not in asok_path: 2024-04-15T15:19:42.759 DEBUG:teuthology.orchestra.run.smithi105:> assert(len(files) == 1) 2024-04-15T15:19:42.759 DEBUG:teuthology.orchestra.run.smithi105:> return files[0] 2024-04-15T15:19:42.759 DEBUG:teuthology.orchestra.run.smithi105:> 2024-04-15T15:19:42.759 DEBUG:teuthology.orchestra.run.smithi105:> for f in files: 2024-04-15T15:19:42.759 DEBUG:teuthology.orchestra.run.smithi105:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-15T15:19:42.759 DEBUG:teuthology.orchestra.run.smithi105:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-15T15:19:42.759 DEBUG:teuthology.orchestra.run.smithi105:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-15T15:19:42.759 DEBUG:teuthology.orchestra.run.smithi105:> contents = proc_f.read() 2024-04-15T15:19:42.759 DEBUG:teuthology.orchestra.run.smithi105:> if mountpoint in contents: 2024-04-15T15:19:42.759 DEBUG:teuthology.orchestra.run.smithi105:> return f 2024-04-15T15:19:42.759 DEBUG:teuthology.orchestra.run.smithi105:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-15T15:19:42.759 DEBUG:teuthology.orchestra.run.smithi105:> 2024-04-15T15:19:42.760 DEBUG:teuthology.orchestra.run.smithi105:> print(_find_admin_socket("client.2")) 2024-04-15T15:19:42.760 DEBUG:teuthology.orchestra.run.smithi105:> ' 2024-04-15T15:19:42.880 INFO:teuthology.orchestra.run.smithi105.stdout:/var/run/ceph/ceph-client.2.46865.asok 2024-04-15T15:19:42.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:19:42.891 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.46865.asok 2024-04-15T15:19:42.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:42.891 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.46865.asok status 2024-04-15T15:19:43.029 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-04-15T15:19:43.029 INFO:teuthology.orchestra.run.smithi105.stdout: "metadata": { 2024-04-15T15:19:43.029 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_sha1": "9bf1c4ba967dd24077be6e5b87edda4ca87a6e14", 2024-04-15T15:19:43.029 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_version": "ceph version 19.0.0-2861-g9bf1c4ba (9bf1c4ba967dd24077be6e5b87edda4ca87a6e14) squid (dev)", 2024-04-15T15:19:43.029 INFO:teuthology.orchestra.run.smithi105.stdout: "entity_id": "2", 2024-04-15T15:19:43.029 INFO:teuthology.orchestra.run.smithi105.stdout: "hostname": "smithi105", 2024-04-15T15:19:43.029 INFO:teuthology.orchestra.run.smithi105.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-04-15T15:19:43.029 INFO:teuthology.orchestra.run.smithi105.stdout: "pid": "46865", 2024-04-15T15:19:43.029 INFO:teuthology.orchestra.run.smithi105.stdout: "root": "/" 2024-04-15T15:19:43.029 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-04-15T15:19:43.029 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_count": 0, 2024-04-15T15:19:43.030 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_pinned_count": 0, 2024-04-15T15:19:43.030 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 4683, 2024-04-15T15:19:43.030 INFO:teuthology.orchestra.run.smithi105.stdout: "inst": { 2024-04-15T15:19:43.030 INFO:teuthology.orchestra.run.smithi105.stdout: "name": { 2024-04-15T15:19:43.030 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "client", 2024-04-15T15:19:43.030 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 4683 2024-04-15T15:19:43.030 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-04-15T15:19:43.030 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": { 2024-04-15T15:19:43.030 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-04-15T15:19:43.030 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "192.168.0.2:0", 2024-04-15T15:19:43.030 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 1882522336 2024-04-15T15:19:43.030 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-04-15T15:19:43.030 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-04-15T15:19:43.030 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": { 2024-04-15T15:19:43.030 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-04-15T15:19:43.031 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "192.168.0.2:0", 2024-04-15T15:19:43.031 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 1882522336 2024-04-15T15:19:43.031 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-04-15T15:19:43.031 INFO:teuthology.orchestra.run.smithi105.stdout: "inst_str": "client.4683 192.168.0.2:0/1882522336", 2024-04-15T15:19:43.031 INFO:teuthology.orchestra.run.smithi105.stdout: "addr_str": "192.168.0.2:0/1882522336", 2024-04-15T15:19:43.031 INFO:teuthology.orchestra.run.smithi105.stdout: "inode_count": 1, 2024-04-15T15:19:43.031 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_epoch": 6, 2024-04-15T15:19:43.031 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_epoch": 20, 2024-04-15T15:19:43.031 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_epoch_barrier": 0, 2024-04-15T15:19:43.031 INFO:teuthology.orchestra.run.smithi105.stdout: "blocklisted": false, 2024-04-15T15:19:43.031 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs" 2024-04-15T15:19:43.031 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-04-15T15:19:43.037 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-15T15:19:43.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.138+0000 7eff8b2f8640 1 Processor -- start 2024-04-15T15:19:43.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.138+0000 7eff8b2f8640 1 -- start start 2024-04-15T15:19:43.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.139+0000 7eff8b2f8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7eff8413ecd0 0x7eff8406efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:43.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.139+0000 7eff8b2f8640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7eff8406f500 0x7eff8415bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:43.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.139+0000 7eff8b2f8640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff8415f280 con 0x7eff8413e6f0 2024-04-15T15:19:43.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.139+0000 7eff8b2f8640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7eff8415f560 con 0x7eff8406f500 2024-04-15T15:19:43.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.139+0000 7eff8b2f8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7eff8415f840 con 0x7eff8413ecd0 2024-04-15T15:19:43.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.139+0000 7eff89895640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7eff8406f500 0x7eff8415bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:43.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.139+0000 7eff89895640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7eff8406f500 0x7eff8415bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51554/0 (socket says 172.21.15.105:51554) 2024-04-15T15:19:43.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.139+0000 7eff89895640 1 -- 172.21.15.105:0/1695386202 learned_addr learned my addr 172.21.15.105:0/1695386202 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:43.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.139+0000 7eff88893640 1 --2- 172.21.15.105:0/1695386202 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7eff8413ecd0 0x7eff8406efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:43.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.140+0000 7eff89895640 1 -- 172.21.15.105:0/1695386202 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7eff8413ecd0 msgr2=0x7eff8406efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:43.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.140+0000 7eff89895640 1 --2- 172.21.15.105:0/1695386202 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7eff8413ecd0 0x7eff8406efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:43.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.140+0000 7eff89895640 1 -- 172.21.15.105:0/1695386202 >> v1:172.21.15.154:6789/0 conn(0x7eff8413e6f0 legacy=0x7eff840cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:43.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.140+0000 7eff89895640 1 -- 172.21.15.105:0/1695386202 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff8415fb20 con 0x7eff8406f500 2024-04-15T15:19:43.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.140+0000 7eff89895640 1 --2- 172.21.15.105:0/1695386202 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff8406f500 0x7eff8415bcd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7eff7400bb10 tx=0x7eff7402e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f9a2eb028774998 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:43.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.140+0000 7eff83fff640 1 -- 172.21.15.105:0/1695386202 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7403b070 con 0x7eff8406f500 2024-04-15T15:19:43.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.141+0000 7eff83fff640 1 -- 172.21.15.105:0/1695386202 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff74036440 con 0x7eff8406f500 2024-04-15T15:19:43.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.141+0000 7eff83fff640 1 -- 172.21.15.105:0/1695386202 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff74036810 con 0x7eff8406f500 2024-04-15T15:19:43.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.141+0000 7eff8b2f8640 1 -- 172.21.15.105:0/1695386202 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff8406f500 msgr2=0x7eff8415bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:43.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.141+0000 7eff8b2f8640 1 --2- 172.21.15.105:0/1695386202 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff8406f500 0x7eff8415bcd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7eff7400bb10 tx=0x7eff7402e0c0 comp rx=0 tx=0).stop 2024-04-15T15:19:43.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.141+0000 7eff8b2f8640 1 -- 172.21.15.105:0/1695386202 shutdown_connections 2024-04-15T15:19:43.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.141+0000 7eff8b2f8640 1 --2- 172.21.15.105:0/1695386202 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff8406f500 0x7eff8415bcd0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:43.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.141+0000 7eff8b2f8640 1 --2- 172.21.15.105:0/1695386202 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7eff8413ecd0 0x7eff8406efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:43.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.141+0000 7eff8b2f8640 1 -- 172.21.15.105:0/1695386202 >> 172.21.15.105:0/1695386202 conn(0x7eff84067e10 msgr2=0x7eff840b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:43.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.141+0000 7eff8b2f8640 1 -- 172.21.15.105:0/1695386202 shutdown_connections 2024-04-15T15:19:43.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.141+0000 7eff8b2f8640 1 -- 172.21.15.105:0/1695386202 wait complete. 2024-04-15T15:19:43.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.142+0000 7eff8b2f8640 1 Processor -- start 2024-04-15T15:19:43.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.142+0000 7eff8b2f8640 1 -- start start 2024-04-15T15:19:43.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.143+0000 7eff8b2f8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff8406f500 0x7eff841543c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:43.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.143+0000 7eff8b2f8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff8413e6f0 0x7eff84153d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:43.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.143+0000 7eff8b2f8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7eff8413ecd0 0x7eff84139a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:43.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.143+0000 7eff8b2f8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7eff840cf670 con 0x7eff8406f500 2024-04-15T15:19:43.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.143+0000 7eff8b2f8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7eff840d0230 con 0x7eff8413e6f0 2024-04-15T15:19:43.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.143+0000 7eff8b2f8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7eff840cf7f0 con 0x7eff8413ecd0 2024-04-15T15:19:43.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.143+0000 7eff89094640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff8406f500 0x7eff841543c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:43.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.143+0000 7eff89094640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff8406f500 0x7eff841543c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51556/0 (socket says 172.21.15.105:51556) 2024-04-15T15:19:43.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.143+0000 7eff89094640 1 -- 172.21.15.105:0/2449006931 learned_addr learned my addr 172.21.15.105:0/2449006931 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:43.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.143+0000 7eff89094640 1 -- 172.21.15.105:0/2449006931 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7eff8413ecd0 msgr2=0x7eff84139a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:43.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.143+0000 7eff89094640 1 --2- 172.21.15.105:0/2449006931 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7eff8413ecd0 0x7eff84139a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:43.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.143+0000 7eff89094640 1 -- 172.21.15.105:0/2449006931 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff8413e6f0 msgr2=0x7eff84153d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:43.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.143+0000 7eff89094640 1 --2- 172.21.15.105:0/2449006931 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff8413e6f0 0x7eff84153d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:43.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.143+0000 7eff89094640 1 -- 172.21.15.105:0/2449006931 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff841383b0 con 0x7eff8406f500 2024-04-15T15:19:43.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.144+0000 7eff89094640 1 --2- 172.21.15.105:0/2449006931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff8406f500 0x7eff841543c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7eff7c00c400 tx=0x7eff7c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:43.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.144+0000 7eff81ffb640 1 -- 172.21.15.105:0/2449006931 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7c0052d0 con 0x7eff8406f500 2024-04-15T15:19:43.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.144+0000 7eff81ffb640 1 -- 172.21.15.105:0/2449006931 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff7c005490 con 0x7eff8406f500 2024-04-15T15:19:43.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.144+0000 7eff8b2f8640 1 -- 172.21.15.105:0/2449006931 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff8415c8c0 con 0x7eff8406f500 2024-04-15T15:19:43.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.144+0000 7eff81ffb640 1 -- 172.21.15.105:0/2449006931 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7c007260 con 0x7eff8406f500 2024-04-15T15:19:43.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.144+0000 7eff8b2f8640 1 -- 172.21.15.105:0/2449006931 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff8415ce80 con 0x7eff8406f500 2024-04-15T15:19:43.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.145+0000 7eff8b2f8640 1 -- 172.21.15.105:0/2449006931 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff84069410 con 0x7eff8406f500 2024-04-15T15:19:43.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.145+0000 7eff81ffb640 1 -- 172.21.15.105:0/2449006931 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7eff7c007400 con 0x7eff8406f500 2024-04-15T15:19:43.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.146+0000 7eff81ffb640 1 --2- 172.21.15.105:0/2449006931 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7eff7006c8f0 0x7eff7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:43.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.146+0000 7eff81ffb640 1 -- 172.21.15.105:0/2449006931 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7eff7c08b050 con 0x7eff8406f500 2024-04-15T15:19:43.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.146+0000 7eff88893640 1 --2- 172.21.15.105:0/2449006931 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7eff7006c8f0 0x7eff7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:43.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.146+0000 7eff88893640 1 --2- 172.21.15.105:0/2449006931 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7eff7006c8f0 0x7eff7006edb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7eff780070b0 tx=0x7eff78007040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:43.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.150+0000 7eff81ffb640 1 -- 172.21.15.105:0/2449006931 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7eff84069410 con 0x7eff8406f500 2024-04-15T15:19:43.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.364+0000 7eff8b2f8640 1 -- 172.21.15.105:0/2449006931 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7eff840b3800 con 0x7eff8406f500 2024-04-15T15:19:43.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.364+0000 7eff81ffb640 1 -- 172.21.15.105:0/2449006931 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) ==== 53+0+73 (secure 0 0 0) 0x7eff7c059510 con 0x7eff8406f500 2024-04-15T15:19:43.366 INFO:teuthology.orchestra.run.smithi105.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-15T15:19:43.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.366+0000 7eff8b2f8640 1 -- 172.21.15.105:0/2449006931 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7eff7006c8f0 msgr2=0x7eff7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:43.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.366+0000 7eff8b2f8640 1 --2- 172.21.15.105:0/2449006931 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7eff7006c8f0 0x7eff7006edb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7eff780070b0 tx=0x7eff78007040 comp rx=0 tx=0).stop 2024-04-15T15:19:43.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.366+0000 7eff8b2f8640 1 -- 172.21.15.105:0/2449006931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff8406f500 msgr2=0x7eff841543c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:43.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.366+0000 7eff8b2f8640 1 --2- 172.21.15.105:0/2449006931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff8406f500 0x7eff841543c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7eff7c00c400 tx=0x7eff7c00c8c0 comp rx=0 tx=0).stop 2024-04-15T15:19:43.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.366+0000 7eff8b2f8640 1 -- 172.21.15.105:0/2449006931 shutdown_connections 2024-04-15T15:19:43.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.366+0000 7eff8b2f8640 1 --2- 172.21.15.105:0/2449006931 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7eff8413ecd0 0x7eff84139a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:43.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.366+0000 7eff8b2f8640 1 --2- 172.21.15.105:0/2449006931 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7eff7006c8f0 0x7eff7006edb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:43.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.366+0000 7eff8b2f8640 1 --2- 172.21.15.105:0/2449006931 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff8413e6f0 0x7eff84153d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:43.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.366+0000 7eff8b2f8640 1 --2- 172.21.15.105:0/2449006931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff8406f500 0x7eff841543c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:43.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.366+0000 7eff8b2f8640 1 -- 172.21.15.105:0/2449006931 >> 172.21.15.105:0/2449006931 conn(0x7eff84067e10 msgr2=0x7eff840b3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:43.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.366+0000 7eff8b2f8640 1 -- 172.21.15.105:0/2449006931 shutdown_connections 2024-04-15T15:19:43.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:43.366+0000 7eff8b2f8640 1 -- 172.21.15.105:0/2449006931 wait complete. 2024-04-15T15:19:43.379 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-15T15:19:43.379 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-15T15:19:43.379 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi154.front.sepia.ceph.com 2024-04-15T15:19:43.379 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-15T15:19:43.379 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-15T15:19:43.379 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-15T15:19:43.379 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-15T15:19:43.379 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-15T15:19:43.379 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-15T15:19:43.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:43.379 DEBUG:teuthology.orchestra.run.smithi154:> ip addr 2024-04-15T15:19:43.399 INFO:teuthology.orchestra.run.smithi154.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-15T15:19:43.399 INFO:teuthology.orchestra.run.smithi154.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-15T15:19:43.399 INFO:teuthology.orchestra.run.smithi154.stdout: inet 127.0.0.1/8 scope host lo 2024-04-15T15:19:43.399 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:43.399 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 ::1/128 scope host 2024-04-15T15:19:43.399 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:43.399 INFO:teuthology.orchestra.run.smithi154.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:19:43.399 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:54 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:19:43.399 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f0 2024-04-15T15:19:43.399 INFO:teuthology.orchestra.run.smithi154.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:19:43.399 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:55 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:19:43.400 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f1 2024-04-15T15:19:43.400 INFO:teuthology.orchestra.run.smithi154.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:19:43.400 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:60 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:19:43.400 INFO:teuthology.orchestra.run.smithi154.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-15T15:19:43.400 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:61 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:19:43.400 INFO:teuthology.orchestra.run.smithi154.stdout: inet 172.21.15.154/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-15T15:19:43.400 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft 42692sec preferred_lft 42692sec 2024-04-15T15:19:43.400 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::ec4:7aff:fe88:7761/64 scope link 2024-04-15T15:19:43.400 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:43.400 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-15T15:19:43.400 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:19:43.400 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2024-04-15T15:19:43.401 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link add name ceph-brx type bridge 2024-04-15T15:19:43.401 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip addr flush dev ceph-brx 2024-04-15T15:19:43.401 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link set ceph-brx up 2024-04-15T15:19:43.401 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-15T15:19:43.401 DEBUG:teuthology.orchestra.run.smithi154:> ') 2024-04-15T15:19:43.495 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:19:43.618 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:19:43.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:43.625 DEBUG:teuthology.orchestra.run.smithi154:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-15T15:19:43.653 INFO:teuthology.orchestra.run.smithi154.stdout:1 2024-04-15T15:19:43.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:43.655 DEBUG:teuthology.orchestra.run.smithi154:> route 2024-04-15T15:19:43.721 INFO:teuthology.orchestra.run.smithi154.stdout:Kernel IP routing table 2024-04-15T15:19:43.721 INFO:teuthology.orchestra.run.smithi154.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-15T15:19:43.721 INFO:teuthology.orchestra.run.smithi154.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-15T15:19:43.721 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-15T15:19:43.721 INFO:teuthology.orchestra.run.smithi154.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-15T15:19:43.722 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:19:43.722 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2024-04-15T15:19:43.722 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-15T15:19:43.722 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-15T15:19:43.722 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-15T15:19:43.722 DEBUG:teuthology.orchestra.run.smithi154:> ') 2024-04-15T15:19:43.810 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:19:44.051 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:19:44.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:44.058 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list 2024-04-15T15:19:44.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:44.074 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list-id 2024-04-15T15:19:44.130 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:19:44.130 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2024-04-15T15:19:44.131 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-15T15:19:44.131 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-15T15:19:44.131 DEBUG:teuthology.orchestra.run.smithi154:> ') 2024-04-15T15:19:44.222 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:19:44.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:19:44.260 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-15T15:19:44.260 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:19:44.260 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2024-04-15T15:19:44.260 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-15T15:19:44.260 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-15T15:19:44.260 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-15T15:19:44.260 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-15T15:19:44.261 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-15T15:19:44.261 DEBUG:teuthology.orchestra.run.smithi154:> ') 2024-04-15T15:19:44.310 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:19:44.398 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:19:44.402 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:19:44.402 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2024-04-15T15:19:44.402 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link set brx.0 up 2024-04-15T15:19:44.402 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link set dev brx.0 master ceph-brx 2024-04-15T15:19:44.402 DEBUG:teuthology.orchestra.run.smithi154:> ') 2024-04-15T15:19:44.451 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:19:44.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:19:44.487 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-15T15:19:44.488 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-15T15:19:44.488 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-15T15:19:44.503 INFO:teuthology.orchestra.run.smithi154.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-15T15:19:44.503 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-15T15:19:44.503 DEBUG:teuthology.orchestra.run.smithi154:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-15T15:19:44.559 DEBUG:teuthology.orchestra.run.smithi154:> sudo modprobe fuse 2024-04-15T15:19:44.628 DEBUG:teuthology.orchestra.run.smithi154:> cat /proc/self/mounts | awk '{print $2}' 2024-04-15T15:19:44.684 INFO:teuthology.orchestra.run.smithi154.stdout:/proc 2024-04-15T15:19:44.684 INFO:teuthology.orchestra.run.smithi154.stdout:/sys 2024-04-15T15:19:44.685 INFO:teuthology.orchestra.run.smithi154.stdout:/dev 2024-04-15T15:19:44.685 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/security 2024-04-15T15:19:44.685 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/shm 2024-04-15T15:19:44.685 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/pts 2024-04-15T15:19:44.685 INFO:teuthology.orchestra.run.smithi154.stdout:/run 2024-04-15T15:19:44.685 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/cgroup 2024-04-15T15:19:44.685 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/pstore 2024-04-15T15:19:44.685 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/bpf 2024-04-15T15:19:44.685 INFO:teuthology.orchestra.run.smithi154.stdout:/ 2024-04-15T15:19:44.685 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/selinux 2024-04-15T15:19:44.685 INFO:teuthology.orchestra.run.smithi154.stdout:/proc/sys/fs/binfmt_misc 2024-04-15T15:19:44.685 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/mqueue 2024-04-15T15:19:44.685 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/hugepages 2024-04-15T15:19:44.686 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/debug 2024-04-15T15:19:44.686 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/tracing 2024-04-15T15:19:44.686 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-15T15:19:44.686 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/fuse/connections 2024-04-15T15:19:44.686 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-sysctl.service 2024-04-15T15:19:44.686 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/config 2024-04-15T15:19:44.686 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-15T15:19:44.686 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/nfs/rpc_pipefs 2024-04-15T15:19:44.686 INFO:teuthology.orchestra.run.smithi154.stdout:/run/user/1000 2024-04-15T15:19:44.686 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph 2024-04-15T15:19:44.686 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-4 2024-04-15T15:19:44.686 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-5 2024-04-15T15:19:44.686 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-6 2024-04-15T15:19:44.687 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-7 2024-04-15T15:19:44.687 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns 2024-04-15T15:19:44.687 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-15T15:19:44.687 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-15T15:19:44.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:44.687 DEBUG:teuthology.orchestra.run.smithi154:> ls /sys/fs/fuse/connections 2024-04-15T15:19:44.740 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-15T15:19:44.740 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-15T15:19:44.781 DEBUG:teuthology.orchestra.run.smithi154:> sudo modprobe fuse 2024-04-15T15:19:44.809 DEBUG:teuthology.orchestra.run.smithi154:> cat /proc/self/mounts | awk '{print $2}' 2024-04-15T15:19:44.866 INFO:teuthology.orchestra.run.smithi154.stdout:/proc 2024-04-15T15:19:44.867 INFO:teuthology.orchestra.run.smithi154.stdout:/sys 2024-04-15T15:19:44.867 INFO:teuthology.orchestra.run.smithi154.stdout:/dev 2024-04-15T15:19:44.867 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/security 2024-04-15T15:19:44.867 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/shm 2024-04-15T15:19:44.867 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/pts 2024-04-15T15:19:44.867 INFO:teuthology.orchestra.run.smithi154.stdout:/run 2024-04-15T15:19:44.867 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/cgroup 2024-04-15T15:19:44.867 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/pstore 2024-04-15T15:19:44.867 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/bpf 2024-04-15T15:19:44.867 INFO:teuthology.orchestra.run.smithi154.stdout:/ 2024-04-15T15:19:44.867 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/selinux 2024-04-15T15:19:44.868 INFO:teuthology.orchestra.run.smithi154.stdout:/proc/sys/fs/binfmt_misc 2024-04-15T15:19:44.868 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/mqueue 2024-04-15T15:19:44.868 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/hugepages 2024-04-15T15:19:44.868 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/debug 2024-04-15T15:19:44.868 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/tracing 2024-04-15T15:19:44.868 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-15T15:19:44.868 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/fuse/connections 2024-04-15T15:19:44.868 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-sysctl.service 2024-04-15T15:19:44.868 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/config 2024-04-15T15:19:44.868 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-15T15:19:44.868 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/nfs/rpc_pipefs 2024-04-15T15:19:44.868 INFO:teuthology.orchestra.run.smithi154.stdout:/run/user/1000 2024-04-15T15:19:44.869 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph 2024-04-15T15:19:44.869 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-4 2024-04-15T15:19:44.869 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-5 2024-04-15T15:19:44.869 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-6 2024-04-15T15:19:44.869 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-7 2024-04-15T15:19:44.869 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns 2024-04-15T15:19:44.869 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-15T15:19:44.869 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-15T15:19:44.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:44.870 DEBUG:teuthology.orchestra.run.smithi154:> ls /sys/fs/fuse/connections 2024-04-15T15:19:44.873 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi154.stderr:2024-04-15T15:19:44.872+0000 7febff23c480 -1 init, newargv = 0x5598fbc9b6b0 newargc=15 2024-04-15T15:19:44.873 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi154.stderr:ceph-fuse[43921]: starting ceph client 2024-04-15T15:19:44.878 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi154.stderr:ceph-fuse[43921]: starting fuse 2024-04-15T15:19:44.889 INFO:teuthology.orchestra.run.smithi154.stdout:43 2024-04-15T15:19:44.890 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-04-15T15:19:44.890 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2024-04-15T15:19:44.890 DEBUG:teuthology.orchestra.run.smithi154:> import glob 2024-04-15T15:19:44.890 DEBUG:teuthology.orchestra.run.smithi154:> import re 2024-04-15T15:19:44.890 DEBUG:teuthology.orchestra.run.smithi154:> import os 2024-04-15T15:19:44.890 DEBUG:teuthology.orchestra.run.smithi154:> import subprocess 2024-04-15T15:19:44.890 DEBUG:teuthology.orchestra.run.smithi154:> 2024-04-15T15:19:44.890 DEBUG:teuthology.orchestra.run.smithi154:> def _find_admin_socket(client_name): 2024-04-15T15:19:44.890 DEBUG:teuthology.orchestra.run.smithi154:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-15T15:19:44.890 DEBUG:teuthology.orchestra.run.smithi154:> files = glob.glob(asok_path) 2024-04-15T15:19:44.891 DEBUG:teuthology.orchestra.run.smithi154:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-15T15:19:44.891 DEBUG:teuthology.orchestra.run.smithi154:> 2024-04-15T15:19:44.891 DEBUG:teuthology.orchestra.run.smithi154:> # Given a non-glob path, it better be there 2024-04-15T15:19:44.891 DEBUG:teuthology.orchestra.run.smithi154:> if "*" not in asok_path: 2024-04-15T15:19:44.891 DEBUG:teuthology.orchestra.run.smithi154:> assert(len(files) == 1) 2024-04-15T15:19:44.891 DEBUG:teuthology.orchestra.run.smithi154:> return files[0] 2024-04-15T15:19:44.891 DEBUG:teuthology.orchestra.run.smithi154:> 2024-04-15T15:19:44.891 DEBUG:teuthology.orchestra.run.smithi154:> for f in files: 2024-04-15T15:19:44.891 DEBUG:teuthology.orchestra.run.smithi154:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-15T15:19:44.891 DEBUG:teuthology.orchestra.run.smithi154:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-15T15:19:44.891 DEBUG:teuthology.orchestra.run.smithi154:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-15T15:19:44.891 DEBUG:teuthology.orchestra.run.smithi154:> contents = proc_f.read() 2024-04-15T15:19:44.891 DEBUG:teuthology.orchestra.run.smithi154:> if mountpoint in contents: 2024-04-15T15:19:44.891 DEBUG:teuthology.orchestra.run.smithi154:> return f 2024-04-15T15:19:44.891 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-15T15:19:44.891 DEBUG:teuthology.orchestra.run.smithi154:> 2024-04-15T15:19:44.892 DEBUG:teuthology.orchestra.run.smithi154:> print(_find_admin_socket("client.1")) 2024-04-15T15:19:44.892 DEBUG:teuthology.orchestra.run.smithi154:> ' 2024-04-15T15:19:45.013 INFO:teuthology.orchestra.run.smithi154.stdout:/var/run/ceph/ceph-client.1.43921.asok 2024-04-15T15:19:45.017 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:19:45.024 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.43921.asok 2024-04-15T15:19:45.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:45.025 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.43921.asok status 2024-04-15T15:19:45.165 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-04-15T15:19:45.165 INFO:teuthology.orchestra.run.smithi154.stdout: "metadata": { 2024-04-15T15:19:45.165 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_sha1": "9bf1c4ba967dd24077be6e5b87edda4ca87a6e14", 2024-04-15T15:19:45.165 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_version": "ceph version 19.0.0-2861-g9bf1c4ba (9bf1c4ba967dd24077be6e5b87edda4ca87a6e14) squid (dev)", 2024-04-15T15:19:45.165 INFO:teuthology.orchestra.run.smithi154.stdout: "entity_id": "1", 2024-04-15T15:19:45.165 INFO:teuthology.orchestra.run.smithi154.stdout: "hostname": "smithi154", 2024-04-15T15:19:45.165 INFO:teuthology.orchestra.run.smithi154.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-15T15:19:45.166 INFO:teuthology.orchestra.run.smithi154.stdout: "pid": "43921", 2024-04-15T15:19:45.166 INFO:teuthology.orchestra.run.smithi154.stdout: "root": "/" 2024-04-15T15:19:45.166 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-15T15:19:45.166 INFO:teuthology.orchestra.run.smithi154.stdout: "dentry_count": 0, 2024-04-15T15:19:45.166 INFO:teuthology.orchestra.run.smithi154.stdout: "dentry_pinned_count": 0, 2024-04-15T15:19:45.166 INFO:teuthology.orchestra.run.smithi154.stdout: "id": 4484, 2024-04-15T15:19:45.166 INFO:teuthology.orchestra.run.smithi154.stdout: "inst": { 2024-04-15T15:19:45.166 INFO:teuthology.orchestra.run.smithi154.stdout: "name": { 2024-04-15T15:19:45.166 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "client", 2024-04-15T15:19:45.166 INFO:teuthology.orchestra.run.smithi154.stdout: "num": 4484 2024-04-15T15:19:45.166 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-15T15:19:45.166 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": { 2024-04-15T15:19:45.166 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v1", 2024-04-15T15:19:45.167 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "192.168.0.1:0", 2024-04-15T15:19:45.167 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 2994305512 2024-04-15T15:19:45.167 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-15T15:19:45.167 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-15T15:19:45.167 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": { 2024-04-15T15:19:45.167 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v1", 2024-04-15T15:19:45.167 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "192.168.0.1:0", 2024-04-15T15:19:45.167 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 2994305512 2024-04-15T15:19:45.167 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-15T15:19:45.167 INFO:teuthology.orchestra.run.smithi154.stdout: "inst_str": "client.4484 192.168.0.1:0/2994305512", 2024-04-15T15:19:45.167 INFO:teuthology.orchestra.run.smithi154.stdout: "addr_str": "192.168.0.1:0/2994305512", 2024-04-15T15:19:45.167 INFO:teuthology.orchestra.run.smithi154.stdout: "inode_count": 1, 2024-04-15T15:19:45.167 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_epoch": 6, 2024-04-15T15:19:45.168 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_epoch": 20, 2024-04-15T15:19:45.168 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_epoch_barrier": 0, 2024-04-15T15:19:45.168 INFO:teuthology.orchestra.run.smithi154.stdout: "blocklisted": false, 2024-04-15T15:19:45.168 INFO:teuthology.orchestra.run.smithi154.stdout: "fs_name": "cephfs" 2024-04-15T15:19:45.168 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-04-15T15:19:45.173 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-15T15:19:45.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.286+0000 7fbac040d640 1 Processor -- start 2024-04-15T15:19:45.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.286+0000 7fbac040d640 1 -- start start 2024-04-15T15:19:45.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.287+0000 7fbac040d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbab8145390 0x7fbab8071eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:45.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.287+0000 7fbac040d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbab8145f60 0x7fbab80723f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:45.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.287+0000 7fbac040d640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbab8145770 con 0x7fbab815cbf0 2024-04-15T15:19:45.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.287+0000 7fbac040d640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fbab81605c0 con 0x7fbab8145f60 2024-04-15T15:19:45.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.287+0000 7fbac040d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fbab81608a0 con 0x7fbab8145390 2024-04-15T15:19:45.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.287+0000 7fbabd9a8640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbab8145f60 0x7fbab80723f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:45.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.287+0000 7fbabd9a8640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbab8145f60 0x7fbab80723f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51560/0 (socket says 172.21.15.105:51560) 2024-04-15T15:19:45.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.287+0000 7fbabd9a8640 1 -- 172.21.15.105:0/3143895436 learned_addr learned my addr 172.21.15.105:0/3143895436 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:45.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.288+0000 7fbabe1a9640 1 --2- 172.21.15.105:0/3143895436 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbab8145390 0x7fbab8071eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:45.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.288+0000 7fbabd9a8640 1 -- 172.21.15.105:0/3143895436 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbab8145390 msgr2=0x7fbab8071eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:45.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.288+0000 7fbabd9a8640 1 --2- 172.21.15.105:0/3143895436 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbab8145390 0x7fbab8071eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:45.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.288+0000 7fbabd9a8640 1 -- 172.21.15.105:0/3143895436 >> v1:172.21.15.154:6789/0 conn(0x7fbab815cbf0 legacy=0x7fbab80ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:45.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.288+0000 7fbabd9a8640 1 -- 172.21.15.105:0/3143895436 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbab8160b80 con 0x7fbab8145f60 2024-04-15T15:19:45.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.288+0000 7fbabe1a9640 1 --2- 172.21.15.105:0/3143895436 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbab8145390 0x7fbab8071eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:45.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.288+0000 7fbabd9a8640 1 --2- 172.21.15.105:0/3143895436 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbab8145f60 0x7fbab80723f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbaa8009870 tx=0x7fbaa802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=493ee8785d29004 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:45.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.289+0000 7fbabd1a7640 1 -- 172.21.15.105:0/3143895436 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa8035020 con 0x7fbab8145f60 2024-04-15T15:19:45.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.289+0000 7fbabd1a7640 1 -- 172.21.15.105:0/3143895436 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbaa8031640 con 0x7fbab8145f60 2024-04-15T15:19:45.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.289+0000 7fbabd1a7640 1 -- 172.21.15.105:0/3143895436 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa802d040 con 0x7fbab8145f60 2024-04-15T15:19:45.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.289+0000 7fbac040d640 1 -- 172.21.15.105:0/3143895436 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbab8145f60 msgr2=0x7fbab80723f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:45.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.289+0000 7fbac040d640 1 --2- 172.21.15.105:0/3143895436 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbab8145f60 0x7fbab80723f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbaa8009870 tx=0x7fbaa802b8f0 comp rx=0 tx=0).stop 2024-04-15T15:19:45.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.289+0000 7fbac040d640 1 -- 172.21.15.105:0/3143895436 shutdown_connections 2024-04-15T15:19:45.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.289+0000 7fbac040d640 1 --2- 172.21.15.105:0/3143895436 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbab8145f60 0x7fbab80723f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:45.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.289+0000 7fbac040d640 1 --2- 172.21.15.105:0/3143895436 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbab8145390 0x7fbab8071eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:45.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.289+0000 7fbac040d640 1 -- 172.21.15.105:0/3143895436 >> 172.21.15.105:0/3143895436 conn(0x7fbab8067e10 msgr2=0x7fbab80b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:45.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.289+0000 7fbac040d640 1 -- 172.21.15.105:0/3143895436 shutdown_connections 2024-04-15T15:19:45.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.289+0000 7fbac040d640 1 -- 172.21.15.105:0/3143895436 wait complete. 2024-04-15T15:19:45.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.290+0000 7fbac040d640 1 Processor -- start 2024-04-15T15:19:45.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.291+0000 7fbac040d640 1 -- start start 2024-04-15T15:19:45.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.291+0000 7fbac040d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbab8145390 0x7fbab81e76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:45.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.291+0000 7fbac040d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbab8145f60 0x7fbab80b4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:45.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.291+0000 7fbac040d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbab815cbf0 0x7fbab80b5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:45.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.291+0000 7fbac040d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbab8160360 con 0x7fbab815cbf0 2024-04-15T15:19:45.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.291+0000 7fbac040d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbab81601e0 con 0x7fbab8145f60 2024-04-15T15:19:45.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.291+0000 7fbac040d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fbab81459a0 con 0x7fbab8145390 2024-04-15T15:19:45.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.291+0000 7fbabe9aa640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbab815cbf0 0x7fbab80b5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:45.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.291+0000 7fbabe9aa640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbab815cbf0 0x7fbab80b5250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51572/0 (socket says 172.21.15.105:51572) 2024-04-15T15:19:45.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.291+0000 7fbabe9aa640 1 -- 172.21.15.105:0/3338512270 learned_addr learned my addr 172.21.15.105:0/3338512270 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:45.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.292+0000 7fbabd9a8640 1 --2- 172.21.15.105:0/3338512270 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbab8145f60 0x7fbab80b4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:45.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.292+0000 7fbabe1a9640 1 --2- 172.21.15.105:0/3338512270 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbab8145390 0x7fbab81e76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:45.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.292+0000 7fbabe9aa640 1 -- 172.21.15.105:0/3338512270 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbab8145390 msgr2=0x7fbab81e76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:45.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.292+0000 7fbabe9aa640 1 --2- 172.21.15.105:0/3338512270 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbab8145390 0x7fbab81e76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:45.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.292+0000 7fbabe9aa640 1 -- 172.21.15.105:0/3338512270 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbab8145f60 msgr2=0x7fbab80b4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:45.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.292+0000 7fbabe9aa640 1 --2- 172.21.15.105:0/3338512270 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbab8145f60 0x7fbab80b4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:45.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.292+0000 7fbabe9aa640 1 -- 172.21.15.105:0/3338512270 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbab80b37f0 con 0x7fbab815cbf0 2024-04-15T15:19:45.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.292+0000 7fbabe1a9640 1 --2- 172.21.15.105:0/3338512270 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbab8145390 0x7fbab81e76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:19:45.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.292+0000 7fbabe9aa640 1 --2- 172.21.15.105:0/3338512270 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbab815cbf0 0x7fbab80b5250 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fbab000b440 tx=0x7fbab000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:45.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.292+0000 7fbaaeffd640 1 -- 172.21.15.105:0/3338512270 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab0011830 con 0x7fbab815cbf0 2024-04-15T15:19:45.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.292+0000 7fbaaeffd640 1 -- 172.21.15.105:0/3338512270 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbab0016070 con 0x7fbab815cbf0 2024-04-15T15:19:45.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.292+0000 7fbaaeffd640 1 -- 172.21.15.105:0/3338512270 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab0011c10 con 0x7fbab815cbf0 2024-04-15T15:19:45.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.292+0000 7fbac040d640 1 -- 172.21.15.105:0/3338512270 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbab80b3ad0 con 0x7fbab815cbf0 2024-04-15T15:19:45.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.292+0000 7fbac040d640 1 -- 172.21.15.105:0/3338512270 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbab80b45b0 con 0x7fbab815cbf0 2024-04-15T15:19:45.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.293+0000 7fbac040d640 1 -- 172.21.15.105:0/3338512270 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbab8068a40 con 0x7fbab815cbf0 2024-04-15T15:19:45.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.294+0000 7fbaaeffd640 1 -- 172.21.15.105:0/3338512270 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbab0004020 con 0x7fbab815cbf0 2024-04-15T15:19:45.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.294+0000 7fbaaeffd640 1 --2- 172.21.15.105:0/3338512270 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fba9c06c8f0 0x7fba9c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:45.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.294+0000 7fbaaeffd640 1 -- 172.21.15.105:0/3338512270 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbab008b4f0 con 0x7fbab815cbf0 2024-04-15T15:19:45.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.294+0000 7fbabe1a9640 1 --2- 172.21.15.105:0/3338512270 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fba9c06c8f0 0x7fba9c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:45.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.295+0000 7fbabe1a9640 1 --2- 172.21.15.105:0/3338512270 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fba9c06c8f0 0x7fba9c06edb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbab400e550 tx=0x7fbab4007ab0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:45.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.299+0000 7fbaaeffd640 1 -- 172.21.15.105:0/3338512270 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbab0020020 con 0x7fbab815cbf0 2024-04-15T15:19:45.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.513+0000 7fbac040d640 1 -- 172.21.15.105:0/3338512270 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fbab815f200 con 0x7fbab815cbf0 2024-04-15T15:19:45.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.513+0000 7fbaaeffd640 1 -- 172.21.15.105:0/3338512270 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) ==== 53+0+73 (secure 0 0 0) 0x7fbab0059920 con 0x7fbab815cbf0 2024-04-15T15:19:45.515 INFO:teuthology.orchestra.run.smithi105.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-15T15:19:45.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.515+0000 7fbac040d640 1 -- 172.21.15.105:0/3338512270 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fba9c06c8f0 msgr2=0x7fba9c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:45.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.515+0000 7fbac040d640 1 --2- 172.21.15.105:0/3338512270 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fba9c06c8f0 0x7fba9c06edb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbab400e550 tx=0x7fbab4007ab0 comp rx=0 tx=0).stop 2024-04-15T15:19:45.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.515+0000 7fbac040d640 1 -- 172.21.15.105:0/3338512270 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbab815cbf0 msgr2=0x7fbab80b5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:45.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.515+0000 7fbac040d640 1 --2- 172.21.15.105:0/3338512270 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbab815cbf0 0x7fbab80b5250 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fbab000b440 tx=0x7fbab000b900 comp rx=0 tx=0).stop 2024-04-15T15:19:45.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.515+0000 7fbac040d640 1 -- 172.21.15.105:0/3338512270 shutdown_connections 2024-04-15T15:19:45.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.515+0000 7fbac040d640 1 --2- 172.21.15.105:0/3338512270 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fba9c06c8f0 0x7fba9c06edb0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:45.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.515+0000 7fbac040d640 1 --2- 172.21.15.105:0/3338512270 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbab815cbf0 0x7fbab80b5250 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:45.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.515+0000 7fbac040d640 1 --2- 172.21.15.105:0/3338512270 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbab8145f60 0x7fbab80b4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:45.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.515+0000 7fbac040d640 1 --2- 172.21.15.105:0/3338512270 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbab8145390 0x7fbab81e76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:45.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.515+0000 7fbac040d640 1 -- 172.21.15.105:0/3338512270 >> 172.21.15.105:0/3338512270 conn(0x7fbab8067e10 msgr2=0x7fbab806b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:45.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.515+0000 7fbac040d640 1 -- 172.21.15.105:0/3338512270 shutdown_connections 2024-04-15T15:19:45.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:45.515+0000 7fbac040d640 1 -- 172.21.15.105:0/3338512270 wait complete. 2024-04-15T15:19:45.528 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-15T15:19:45.528 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-15T15:19:45.528 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi154.front.sepia.ceph.com 2024-04-15T15:19:45.528 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-15T15:19:45.528 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-15T15:19:45.528 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-15T15:19:45.528 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-15T15:19:45.528 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-15T15:19:45.528 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-15T15:19:45.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:45.529 DEBUG:teuthology.orchestra.run.smithi154:> ip addr 2024-04-15T15:19:45.545 INFO:teuthology.orchestra.run.smithi154.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-15T15:19:45.545 INFO:teuthology.orchestra.run.smithi154.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-15T15:19:45.545 INFO:teuthology.orchestra.run.smithi154.stdout: inet 127.0.0.1/8 scope host lo 2024-04-15T15:19:45.545 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:45.545 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 ::1/128 scope host 2024-04-15T15:19:45.545 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:45.545 INFO:teuthology.orchestra.run.smithi154.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:19:45.545 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:54 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:19:45.545 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f0 2024-04-15T15:19:45.546 INFO:teuthology.orchestra.run.smithi154.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:19:45.546 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:55 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:19:45.546 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f1 2024-04-15T15:19:45.546 INFO:teuthology.orchestra.run.smithi154.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:19:45.546 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:60 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:19:45.546 INFO:teuthology.orchestra.run.smithi154.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-15T15:19:45.546 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:61 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:19:45.546 INFO:teuthology.orchestra.run.smithi154.stdout: inet 172.21.15.154/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-15T15:19:45.546 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft 42690sec preferred_lft 42690sec 2024-04-15T15:19:45.546 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::ec4:7aff:fe88:7761/64 scope link 2024-04-15T15:19:45.546 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:45.546 INFO:teuthology.orchestra.run.smithi154.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-15T15:19:45.546 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether ce:c3:4d:70:5f:15 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:19:45.547 INFO:teuthology.orchestra.run.smithi154.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-15T15:19:45.547 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:45.547 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::1cc6:e4ff:fe35:1850/64 scope link tentative 2024-04-15T15:19:45.547 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:45.547 INFO:teuthology.orchestra.run.smithi154.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-15T15:19:45.547 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether ce:c3:4d:70:5f:15 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-15T15:19:45.547 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::ccc3:4dff:fe70:5f15/64 scope link tentative 2024-04-15T15:19:45.547 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:45.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:45.547 DEBUG:teuthology.orchestra.run.smithi154:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-15T15:19:45.613 INFO:teuthology.orchestra.run.smithi154.stdout:1 2024-04-15T15:19:45.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:45.615 DEBUG:teuthology.orchestra.run.smithi154:> route 2024-04-15T15:19:45.672 INFO:teuthology.orchestra.run.smithi154.stdout:Kernel IP routing table 2024-04-15T15:19:45.672 INFO:teuthology.orchestra.run.smithi154.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-15T15:19:45.672 INFO:teuthology.orchestra.run.smithi154.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-15T15:19:45.672 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-15T15:19:45.672 INFO:teuthology.orchestra.run.smithi154.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-15T15:19:45.673 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:19:45.673 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2024-04-15T15:19:45.673 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-15T15:19:45.673 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-15T15:19:45.673 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-15T15:19:45.673 DEBUG:teuthology.orchestra.run.smithi154:> ') 2024-04-15T15:19:45.762 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:19:45.806 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:19:45.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:45.812 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list 2024-04-15T15:19:45.828 INFO:teuthology.orchestra.run.smithi154.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-15T15:19:45.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:45.829 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list-id 2024-04-15T15:19:45.884 INFO:teuthology.orchestra.run.smithi154.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-15T15:19:45.884 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:19:45.884 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2024-04-15T15:19:45.885 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-15T15:19:45.885 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-15T15:19:45.885 DEBUG:teuthology.orchestra.run.smithi154:> ') 2024-04-15T15:19:45.974 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:19:46.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:19:46.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:46.010 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-15T15:19:46.039 INFO:teuthology.orchestra.run.smithi154.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-15T15:19:46.039 INFO:teuthology.orchestra.run.smithi154.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-15T15:19:46.039 INFO:teuthology.orchestra.run.smithi154.stdout: inet 127.0.0.1/8 scope host lo 2024-04-15T15:19:46.039 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:46.039 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 ::1/128 scope host 2024-04-15T15:19:46.039 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:46.039 INFO:teuthology.orchestra.run.smithi154.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-15T15:19:46.040 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether de:ef:01:d2:e9:eb brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-15T15:19:46.040 INFO:teuthology.orchestra.run.smithi154.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-15T15:19:46.040 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:46.040 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::dcef:1ff:fed2:e9eb/64 scope link 2024-04-15T15:19:46.040 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:46.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:46.040 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-15T15:19:46.108 INFO:teuthology.orchestra.run.smithi154.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-15T15:19:46.108 INFO:teuthology.orchestra.run.smithi154.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-15T15:19:46.108 INFO:teuthology.orchestra.run.smithi154.stdout: inet 127.0.0.1/8 scope host lo 2024-04-15T15:19:46.108 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:46.108 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 ::1/128 scope host 2024-04-15T15:19:46.108 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:46.108 INFO:teuthology.orchestra.run.smithi154.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-15T15:19:46.108 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether de:ef:01:d2:e9:eb brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-15T15:19:46.108 INFO:teuthology.orchestra.run.smithi154.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-15T15:19:46.108 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:46.108 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::dcef:1ff:fed2:e9eb/64 scope link 2024-04-15T15:19:46.108 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:46.109 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-15T15:19:46.110 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:19:46.110 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2024-04-15T15:19:46.110 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-15T15:19:46.110 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-15T15:19:46.110 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-15T15:19:46.110 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-15T15:19:46.110 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-15T15:19:46.110 DEBUG:teuthology.orchestra.run.smithi154:> ') 2024-04-15T15:19:46.198 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:19:46.274 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:19:46.280 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:19:46.280 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2024-04-15T15:19:46.280 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link set brx.1 up 2024-04-15T15:19:46.280 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link set dev brx.1 master ceph-brx 2024-04-15T15:19:46.280 DEBUG:teuthology.orchestra.run.smithi154:> ') 2024-04-15T15:19:46.329 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:19:46.358 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:19:46.364 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-04-15T15:19:46.364 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-15T15:19:46.364 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-15T15:19:46.379 INFO:teuthology.orchestra.run.smithi154.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-15T15:19:46.379 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-15T15:19:46.380 DEBUG:teuthology.orchestra.run.smithi154:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-15T15:19:46.438 DEBUG:teuthology.orchestra.run.smithi154:> sudo modprobe fuse 2024-04-15T15:19:46.508 DEBUG:teuthology.orchestra.run.smithi154:> cat /proc/self/mounts | awk '{print $2}' 2024-04-15T15:19:46.565 INFO:teuthology.orchestra.run.smithi154.stdout:/proc 2024-04-15T15:19:46.565 INFO:teuthology.orchestra.run.smithi154.stdout:/sys 2024-04-15T15:19:46.565 INFO:teuthology.orchestra.run.smithi154.stdout:/dev 2024-04-15T15:19:46.565 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/security 2024-04-15T15:19:46.565 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/shm 2024-04-15T15:19:46.565 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/pts 2024-04-15T15:19:46.565 INFO:teuthology.orchestra.run.smithi154.stdout:/run 2024-04-15T15:19:46.565 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/cgroup 2024-04-15T15:19:46.566 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/pstore 2024-04-15T15:19:46.566 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/bpf 2024-04-15T15:19:46.566 INFO:teuthology.orchestra.run.smithi154.stdout:/ 2024-04-15T15:19:46.566 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/selinux 2024-04-15T15:19:46.566 INFO:teuthology.orchestra.run.smithi154.stdout:/proc/sys/fs/binfmt_misc 2024-04-15T15:19:46.566 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/mqueue 2024-04-15T15:19:46.566 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/hugepages 2024-04-15T15:19:46.566 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/debug 2024-04-15T15:19:46.566 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/tracing 2024-04-15T15:19:46.566 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-15T15:19:46.566 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/fuse/connections 2024-04-15T15:19:46.566 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-sysctl.service 2024-04-15T15:19:46.566 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/config 2024-04-15T15:19:46.567 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-15T15:19:46.567 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/nfs/rpc_pipefs 2024-04-15T15:19:46.567 INFO:teuthology.orchestra.run.smithi154.stdout:/run/user/1000 2024-04-15T15:19:46.567 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph 2024-04-15T15:19:46.567 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-4 2024-04-15T15:19:46.567 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-5 2024-04-15T15:19:46.567 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-6 2024-04-15T15:19:46.567 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-7 2024-04-15T15:19:46.567 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns 2024-04-15T15:19:46.567 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-15T15:19:46.567 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-15T15:19:46.567 INFO:teuthology.orchestra.run.smithi154.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-15T15:19:46.567 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-15T15:19:46.567 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-15T15:19:46.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:46.568 DEBUG:teuthology.orchestra.run.smithi154:> ls /sys/fs/fuse/connections 2024-04-15T15:19:46.621 INFO:teuthology.orchestra.run.smithi154.stdout:43 2024-04-15T15:19:46.621 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [43] 2024-04-15T15:19:46.621 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-04-15T15:19:46.664 DEBUG:teuthology.orchestra.run.smithi154:> sudo modprobe fuse 2024-04-15T15:19:46.692 DEBUG:teuthology.orchestra.run.smithi154:> cat /proc/self/mounts | awk '{print $2}' 2024-04-15T15:19:46.749 INFO:teuthology.orchestra.run.smithi154.stdout:/proc 2024-04-15T15:19:46.749 INFO:teuthology.orchestra.run.smithi154.stdout:/sys 2024-04-15T15:19:46.749 INFO:teuthology.orchestra.run.smithi154.stdout:/dev 2024-04-15T15:19:46.749 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/security 2024-04-15T15:19:46.749 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/shm 2024-04-15T15:19:46.749 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/pts 2024-04-15T15:19:46.749 INFO:teuthology.orchestra.run.smithi154.stdout:/run 2024-04-15T15:19:46.749 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/cgroup 2024-04-15T15:19:46.750 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/pstore 2024-04-15T15:19:46.750 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/bpf 2024-04-15T15:19:46.750 INFO:teuthology.orchestra.run.smithi154.stdout:/ 2024-04-15T15:19:46.750 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/selinux 2024-04-15T15:19:46.750 INFO:teuthology.orchestra.run.smithi154.stdout:/proc/sys/fs/binfmt_misc 2024-04-15T15:19:46.750 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/mqueue 2024-04-15T15:19:46.750 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/hugepages 2024-04-15T15:19:46.750 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/debug 2024-04-15T15:19:46.750 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/tracing 2024-04-15T15:19:46.750 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-15T15:19:46.750 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/fuse/connections 2024-04-15T15:19:46.750 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-sysctl.service 2024-04-15T15:19:46.751 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/config 2024-04-15T15:19:46.751 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-15T15:19:46.751 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/nfs/rpc_pipefs 2024-04-15T15:19:46.751 INFO:teuthology.orchestra.run.smithi154.stdout:/run/user/1000 2024-04-15T15:19:46.751 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph 2024-04-15T15:19:46.751 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-4 2024-04-15T15:19:46.751 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-5 2024-04-15T15:19:46.751 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-6 2024-04-15T15:19:46.751 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-7 2024-04-15T15:19:46.751 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns 2024-04-15T15:19:46.751 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-15T15:19:46.751 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-15T15:19:46.751 INFO:teuthology.orchestra.run.smithi154.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-15T15:19:46.752 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-15T15:19:46.752 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-15T15:19:46.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:46.752 DEBUG:teuthology.orchestra.run.smithi154:> ls /sys/fs/fuse/connections 2024-04-15T15:19:46.753 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi154.stderr:2024-04-15T15:19:46.751+0000 7f949f24b480 -1 init, newargv = 0x565512351280 newargc=15 2024-04-15T15:19:46.753 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi154.stderr:ceph-fuse[44378]: starting ceph client 2024-04-15T15:19:46.757 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi154.stderr:ceph-fuse[44378]: starting fuse 2024-04-15T15:19:46.769 INFO:teuthology.orchestra.run.smithi154.stdout:43 2024-04-15T15:19:46.769 INFO:teuthology.orchestra.run.smithi154.stdout:44 2024-04-15T15:19:46.769 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43, 44] 2024-04-15T15:19:46.769 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2024-04-15T15:19:46.770 DEBUG:teuthology.orchestra.run.smithi154:> import glob 2024-04-15T15:19:46.770 DEBUG:teuthology.orchestra.run.smithi154:> import re 2024-04-15T15:19:46.770 DEBUG:teuthology.orchestra.run.smithi154:> import os 2024-04-15T15:19:46.770 DEBUG:teuthology.orchestra.run.smithi154:> import subprocess 2024-04-15T15:19:46.770 DEBUG:teuthology.orchestra.run.smithi154:> 2024-04-15T15:19:46.770 DEBUG:teuthology.orchestra.run.smithi154:> def _find_admin_socket(client_name): 2024-04-15T15:19:46.770 DEBUG:teuthology.orchestra.run.smithi154:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-04-15T15:19:46.770 DEBUG:teuthology.orchestra.run.smithi154:> files = glob.glob(asok_path) 2024-04-15T15:19:46.770 DEBUG:teuthology.orchestra.run.smithi154:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-04-15T15:19:46.770 DEBUG:teuthology.orchestra.run.smithi154:> 2024-04-15T15:19:46.770 DEBUG:teuthology.orchestra.run.smithi154:> # Given a non-glob path, it better be there 2024-04-15T15:19:46.770 DEBUG:teuthology.orchestra.run.smithi154:> if "*" not in asok_path: 2024-04-15T15:19:46.770 DEBUG:teuthology.orchestra.run.smithi154:> assert(len(files) == 1) 2024-04-15T15:19:46.771 DEBUG:teuthology.orchestra.run.smithi154:> return files[0] 2024-04-15T15:19:46.771 DEBUG:teuthology.orchestra.run.smithi154:> 2024-04-15T15:19:46.771 DEBUG:teuthology.orchestra.run.smithi154:> for f in files: 2024-04-15T15:19:46.771 DEBUG:teuthology.orchestra.run.smithi154:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-15T15:19:46.771 DEBUG:teuthology.orchestra.run.smithi154:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-15T15:19:46.771 DEBUG:teuthology.orchestra.run.smithi154:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-15T15:19:46.771 DEBUG:teuthology.orchestra.run.smithi154:> contents = proc_f.read() 2024-04-15T15:19:46.771 DEBUG:teuthology.orchestra.run.smithi154:> if mountpoint in contents: 2024-04-15T15:19:46.771 DEBUG:teuthology.orchestra.run.smithi154:> return f 2024-04-15T15:19:46.771 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-15T15:19:46.771 DEBUG:teuthology.orchestra.run.smithi154:> 2024-04-15T15:19:46.772 DEBUG:teuthology.orchestra.run.smithi154:> print(_find_admin_socket("client.3")) 2024-04-15T15:19:46.772 DEBUG:teuthology.orchestra.run.smithi154:> ' 2024-04-15T15:19:46.891 INFO:teuthology.orchestra.run.smithi154.stdout:/var/run/ceph/ceph-client.3.44378.asok 2024-04-15T15:19:46.894 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:19:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:19:46.901 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.44378.asok 2024-04-15T15:19:46.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:46.902 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.44378.asok status 2024-04-15T15:19:47.039 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-04-15T15:19:47.039 INFO:teuthology.orchestra.run.smithi154.stdout: "metadata": { 2024-04-15T15:19:47.039 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_sha1": "9bf1c4ba967dd24077be6e5b87edda4ca87a6e14", 2024-04-15T15:19:47.040 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_version": "ceph version 19.0.0-2861-g9bf1c4ba (9bf1c4ba967dd24077be6e5b87edda4ca87a6e14) squid (dev)", 2024-04-15T15:19:47.040 INFO:teuthology.orchestra.run.smithi154.stdout: "entity_id": "3", 2024-04-15T15:19:47.040 INFO:teuthology.orchestra.run.smithi154.stdout: "hostname": "smithi154", 2024-04-15T15:19:47.040 INFO:teuthology.orchestra.run.smithi154.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-04-15T15:19:47.040 INFO:teuthology.orchestra.run.smithi154.stdout: "pid": "44378", 2024-04-15T15:19:47.040 INFO:teuthology.orchestra.run.smithi154.stdout: "root": "/" 2024-04-15T15:19:47.040 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-15T15:19:47.040 INFO:teuthology.orchestra.run.smithi154.stdout: "dentry_count": 0, 2024-04-15T15:19:47.040 INFO:teuthology.orchestra.run.smithi154.stdout: "dentry_pinned_count": 0, 2024-04-15T15:19:47.040 INFO:teuthology.orchestra.run.smithi154.stdout: "id": 4493, 2024-04-15T15:19:47.040 INFO:teuthology.orchestra.run.smithi154.stdout: "inst": { 2024-04-15T15:19:47.040 INFO:teuthology.orchestra.run.smithi154.stdout: "name": { 2024-04-15T15:19:47.040 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "client", 2024-04-15T15:19:47.040 INFO:teuthology.orchestra.run.smithi154.stdout: "num": 4493 2024-04-15T15:19:47.041 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-15T15:19:47.041 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": { 2024-04-15T15:19:47.041 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v1", 2024-04-15T15:19:47.041 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "192.168.0.2:0", 2024-04-15T15:19:47.041 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 423297047 2024-04-15T15:19:47.041 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-15T15:19:47.041 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-15T15:19:47.041 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": { 2024-04-15T15:19:47.041 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v1", 2024-04-15T15:19:47.041 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "192.168.0.2:0", 2024-04-15T15:19:47.041 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 423297047 2024-04-15T15:19:47.041 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-15T15:19:47.041 INFO:teuthology.orchestra.run.smithi154.stdout: "inst_str": "client.4493 192.168.0.2:0/423297047", 2024-04-15T15:19:47.041 INFO:teuthology.orchestra.run.smithi154.stdout: "addr_str": "192.168.0.2:0/423297047", 2024-04-15T15:19:47.041 INFO:teuthology.orchestra.run.smithi154.stdout: "inode_count": 1, 2024-04-15T15:19:47.042 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_epoch": 6, 2024-04-15T15:19:47.042 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_epoch": 20, 2024-04-15T15:19:47.042 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_epoch_barrier": 0, 2024-04-15T15:19:47.042 INFO:teuthology.orchestra.run.smithi154.stdout: "blocklisted": false, 2024-04-15T15:19:47.042 INFO:teuthology.orchestra.run.smithi154.stdout: "fs_name": "cephfs" 2024-04-15T15:19:47.042 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-04-15T15:19:47.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:47.050 DEBUG:teuthology.orchestra.run.smithi105:> stat --file-system '--printf=%T 2024-04-15T15:19:47.051 DEBUG:teuthology.orchestra.run.smithi105:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-15T15:19:47.070 INFO:teuthology.orchestra.run.smithi105.stdout:fuseblk 2024-04-15T15:19:47.071 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-15T15:19:47.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:47.071 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-15T15:19:47.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:47.146 DEBUG:teuthology.orchestra.run.smithi105:> stat --file-system '--printf=%T 2024-04-15T15:19:47.146 DEBUG:teuthology.orchestra.run.smithi105:> ' -- /home/ubuntu/cephtest/mnt.2 2024-04-15T15:19:47.205 INFO:teuthology.orchestra.run.smithi105.stdout:fuseblk 2024-04-15T15:19:47.205 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-04-15T15:19:47.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:47.205 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-15T15:19:47.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:47.280 DEBUG:teuthology.orchestra.run.smithi154:> stat --file-system '--printf=%T 2024-04-15T15:19:47.280 DEBUG:teuthology.orchestra.run.smithi154:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-15T15:19:47.296 INFO:teuthology.orchestra.run.smithi154.stdout:fuseblk 2024-04-15T15:19:47.296 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-15T15:19:47.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:47.297 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-15T15:19:47.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:47.367 DEBUG:teuthology.orchestra.run.smithi154:> stat --file-system '--printf=%T 2024-04-15T15:19:47.367 DEBUG:teuthology.orchestra.run.smithi154:> ' -- /home/ubuntu/cephtest/mnt.3 2024-04-15T15:19:47.423 INFO:teuthology.orchestra.run.smithi154.stdout:fuseblk 2024-04-15T15:19:47.424 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-04-15T15:19:47.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:47.424 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-15T15:19:47.492 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-15T15:19:47.508 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:19:47.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.617+0000 7f53dc631640 1 Processor -- start 2024-04-15T15:19:47.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.617+0000 7f53dc631640 1 -- start start 2024-04-15T15:19:47.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.618+0000 7f53dc631640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f53d413ec60 0x7f53d406ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:47.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.618+0000 7f53dc631640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f53d406f4d0 0x7f53d415f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:47.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.618+0000 7f53dc631640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53d415bc90 con 0x7f53d413e680 2024-04-15T15:19:47.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.618+0000 7f53dc631640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f53d40d0280 con 0x7f53d413ec60 2024-04-15T15:19:47.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.618+0000 7f53dc631640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f53d406f900 con 0x7f53d406f4d0 2024-04-15T15:19:47.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.618+0000 7f53d9bcc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f53d413ec60 0x7f53d406ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:47.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.618+0000 7f53d9bcc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f53d413ec60 0x7f53d406ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51588/0 (socket says 172.21.15.105:51588) 2024-04-15T15:19:47.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.618+0000 7f53d9bcc640 1 -- 172.21.15.105:0/1787615961 learned_addr learned my addr 172.21.15.105:0/1787615961 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:47.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.619+0000 7f53dabce640 1 --2- 172.21.15.105:0/1787615961 >> v2:172.21.15.154:3300/0 conn(0x7f53d406f4d0 0x7f53d415f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:47.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.619+0000 7f53d9bcc640 1 -- 172.21.15.105:0/1787615961 >> v2:172.21.15.154:3300/0 conn(0x7f53d406f4d0 msgr2=0x7f53d415f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:47.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.619+0000 7f53d9bcc640 1 --2- 172.21.15.105:0/1787615961 >> v2:172.21.15.154:3300/0 conn(0x7f53d406f4d0 0x7f53d415f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:47.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.619+0000 7f53d9bcc640 1 -- 172.21.15.105:0/1787615961 >> v1:172.21.15.154:6789/0 conn(0x7f53d413e680 legacy=0x7f53d40cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:47.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.619+0000 7f53d9bcc640 1 -- 172.21.15.105:0/1787615961 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53d415bf70 con 0x7f53d413ec60 2024-04-15T15:19:47.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.619+0000 7f53dabce640 1 --2- 172.21.15.105:0/1787615961 >> v2:172.21.15.154:3300/0 conn(0x7f53d406f4d0 0x7f53d415f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:47.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.619+0000 7f53d9bcc640 1 --2- 172.21.15.105:0/1787615961 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53d413ec60 0x7f53d406ef90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f53d41451d0 tx=0x7f53c402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17b31110fb7efbe0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:47.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.620+0000 7f53d93cb640 1 -- 172.21.15.105:0/1787615961 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c4035020 con 0x7f53d413ec60 2024-04-15T15:19:47.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.620+0000 7f53d93cb640 1 -- 172.21.15.105:0/1787615961 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53c4031870 con 0x7f53d413ec60 2024-04-15T15:19:47.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.620+0000 7f53d93cb640 1 -- 172.21.15.105:0/1787615961 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c4031c40 con 0x7f53d413ec60 2024-04-15T15:19:47.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.620+0000 7f53dc631640 1 -- 172.21.15.105:0/1787615961 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53d413ec60 msgr2=0x7f53d406ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:47.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.620+0000 7f53dc631640 1 --2- 172.21.15.105:0/1787615961 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53d413ec60 0x7f53d406ef90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f53d41451d0 tx=0x7f53c402b910 comp rx=0 tx=0).stop 2024-04-15T15:19:47.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.620+0000 7f53dc631640 1 -- 172.21.15.105:0/1787615961 shutdown_connections 2024-04-15T15:19:47.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.620+0000 7f53dc631640 1 --2- 172.21.15.105:0/1787615961 >> v2:172.21.15.154:3300/0 conn(0x7f53d406f4d0 0x7f53d415f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:47.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.620+0000 7f53dc631640 1 --2- 172.21.15.105:0/1787615961 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53d413ec60 0x7f53d406ef90 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:47.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.620+0000 7f53dc631640 1 -- 172.21.15.105:0/1787615961 >> 172.21.15.105:0/1787615961 conn(0x7f53d4067d50 msgr2=0x7f53d406b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:47.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.620+0000 7f53dc631640 1 -- 172.21.15.105:0/1787615961 shutdown_connections 2024-04-15T15:19:47.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.620+0000 7f53dc631640 1 -- 172.21.15.105:0/1787615961 wait complete. 2024-04-15T15:19:47.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.621+0000 7f53dc631640 1 Processor -- start 2024-04-15T15:19:47.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.621+0000 7f53dc631640 1 -- start start 2024-04-15T15:19:47.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.622+0000 7f53dc631640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53d406f4d0 0x7f53d4154c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:47.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.622+0000 7f53dc631640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f53d413e680 0x7f53d4143da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:47.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.622+0000 7f53dc631640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53d413ec60 0x7f53d41442e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:47.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.622+0000 7f53dc631640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f53d415c810 con 0x7f53d406f4d0 2024-04-15T15:19:47.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.622+0000 7f53dc631640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f53d415c530 con 0x7f53d413ec60 2024-04-15T15:19:47.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.622+0000 7f53dc631640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f53d40cf590 con 0x7f53d413e680 2024-04-15T15:19:47.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.622+0000 7f53da3cd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53d406f4d0 0x7f53d4154c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:47.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.622+0000 7f53da3cd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53d406f4d0 0x7f53d4154c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51600/0 (socket says 172.21.15.105:51600) 2024-04-15T15:19:47.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.622+0000 7f53da3cd640 1 -- 172.21.15.105:0/2994370332 learned_addr learned my addr 172.21.15.105:0/2994370332 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:47.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.622+0000 7f53d9bcc640 1 --2- 172.21.15.105:0/2994370332 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f53d413e680 0x7f53d4143da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:47.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.622+0000 7f53dabce640 1 --2- 172.21.15.105:0/2994370332 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53d413ec60 0x7f53d41442e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:47.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.622+0000 7f53da3cd640 1 -- 172.21.15.105:0/2994370332 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f53d413e680 msgr2=0x7f53d4143da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:47.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.623+0000 7f53da3cd640 1 --2- 172.21.15.105:0/2994370332 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f53d413e680 0x7f53d4143da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:47.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.623+0000 7f53da3cd640 1 -- 172.21.15.105:0/2994370332 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53d413ec60 msgr2=0x7f53d41442e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:47.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.623+0000 7f53da3cd640 1 --2- 172.21.15.105:0/2994370332 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53d413ec60 0x7f53d41442e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:47.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.623+0000 7f53da3cd640 1 -- 172.21.15.105:0/2994370332 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53d4141090 con 0x7f53d406f4d0 2024-04-15T15:19:47.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.623+0000 7f53d9bcc640 1 --2- 172.21.15.105:0/2994370332 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f53d413e680 0x7f53d4143da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:47.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.623+0000 7f53dabce640 1 --2- 172.21.15.105:0/2994370332 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53d413ec60 0x7f53d41442e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:47.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.623+0000 7f53da3cd640 1 --2- 172.21.15.105:0/2994370332 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53d406f4d0 0x7f53d4154c70 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f53d000c430 tx=0x7f53d000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:47.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.623+0000 7f53caffd640 1 -- 172.21.15.105:0/2994370332 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d0011790 con 0x7f53d406f4d0 2024-04-15T15:19:47.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.623+0000 7f53caffd640 1 -- 172.21.15.105:0/2994370332 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53d0017070 con 0x7f53d406f4d0 2024-04-15T15:19:47.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.623+0000 7f53dc631640 1 -- 172.21.15.105:0/2994370332 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53d4140310 con 0x7f53d406f4d0 2024-04-15T15:19:47.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.623+0000 7f53caffd640 1 -- 172.21.15.105:0/2994370332 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d0011bb0 con 0x7f53d406f4d0 2024-04-15T15:19:47.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.623+0000 7f53dc631640 1 -- 172.21.15.105:0/2994370332 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53d41405b0 con 0x7f53d406f4d0 2024-04-15T15:19:47.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.624+0000 7f53dc631640 1 -- 172.21.15.105:0/2994370332 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53d4068a40 con 0x7f53d406f4d0 2024-04-15T15:19:47.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.624+0000 7f53caffd640 1 -- 172.21.15.105:0/2994370332 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f53d001c460 con 0x7f53d406f4d0 2024-04-15T15:19:47.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.625+0000 7f53caffd640 1 --2- 172.21.15.105:0/2994370332 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f53b806c8f0 0x7f53b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:47.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.625+0000 7f53caffd640 1 -- 172.21.15.105:0/2994370332 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f53d008bb00 con 0x7f53d406f4d0 2024-04-15T15:19:47.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.625+0000 7f53d9bcc640 1 --2- 172.21.15.105:0/2994370332 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f53b806c8f0 0x7f53b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:47.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.625+0000 7f53d9bcc640 1 --2- 172.21.15.105:0/2994370332 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f53b806c8f0 0x7f53b806edb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f53c4036000 tx=0x7f53c4008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:47.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.629+0000 7f53caffd640 1 -- 172.21.15.105:0/2994370332 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f53d0059ea0 con 0x7f53d406f4d0 2024-04-15T15:19:47.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.806+0000 7f53dc631640 1 -- 172.21.15.105:0/2994370332 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f53d40cfc30 con 0x7f53d406f4d0 2024-04-15T15:19:47.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.807+0000 7f53caffd640 1 -- 172.21.15.105:0/2994370332 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16653 (secure 0 0 0) 0x7f53d00597a0 con 0x7f53d406f4d0 2024-04-15T15:19:47.809 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:19:47.809 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":20,"fsid":"09e4d2a4-a324-4793-af77-644b25e520e1","created":"2024-04-15T15:19:01.524543+0000","modified":"2024-04-15T15:19:27.414890+0000","last_up_change":"2024-04-15T15:19:12.825815+0000","last_in_change":"2024-04-15T15:19:03.788126+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:19:13.748585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:19:23.789287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:19:24.722250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6f6292b-2d7e-4aa0-9d1c-6ddd0f97287c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6801","nonce":2873040054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6803","nonce":2873040054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6807","nonce":2873040054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6805","nonce":2873040054}]},"public_addr":"172.21.15.105:6801/2873040054","cluster_addr":"172.21.15.105:6803/2873040054","heartbeat_back_addr":"172.21.15.105:6807/2873040054","heartbeat_front_addr":"172.21.15.105:6805/2873040054","state":["exists","up"]},{"osd":1,"uuid":"8900966a-b393-41e3-8f93-e5a43bd23622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6818","nonce":2494994479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6821","nonce":2494994479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6828","nonce":2494994479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6823","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6825","nonce":2494994479}]},"public_addr":"172.21.15.105:6818/2494994479","cluster_addr":"172.21.15.105:6821/2494994479","heartbeat_back_addr":"172.21.15.105:6828/2494994479","heartbeat_front_addr":"172.21.15.105:6825/2494994479","state":["exists","up"]},{"osd":2,"uuid":"81a5dc7a-9325-4ff5-941d-a2d5fba21f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6809","nonce":3798408699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6811","nonce":3798408699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6815","nonce":3798408699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6813","nonce":3798408699}]},"public_addr":"172.21.15.105:6809/3798408699","cluster_addr":"172.21.15.105:6811/3798408699","heartbeat_back_addr":"172.21.15.105:6815/3798408699","heartbeat_front_addr":"172.21.15.105:6813/3798408699","state":["exists","up"]},{"osd":3,"uuid":"df766ed2-5c71-4805-96f4-3d4970edc960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6819","nonce":1920720186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6824","nonce":1920720186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6831","nonce":1920720186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6827","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6829","nonce":1920720186}]},"public_addr":"172.21.15.105:6819/1920720186","cluster_addr":"172.21.15.105:6824/1920720186","heartbeat_back_addr":"172.21.15.105:6831/1920720186","heartbeat_front_addr":"172.21.15.105:6829/1920720186","state":["exists","up"]},{"osd":4,"uuid":"e1039e03-28b8-49df-b951-c91930b93c9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6810","nonce":3516494456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6813","nonce":3516494456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6820","nonce":3516494456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6817","nonce":3516494456}]},"public_addr":"172.21.15.154:6810/3516494456","cluster_addr":"172.21.15.154:6813/3516494456","heartbeat_back_addr":"172.21.15.154:6820/3516494456","heartbeat_front_addr":"172.21.15.154:6817/3516494456","state":["exists","up"]},{"osd":5,"uuid":"a9f5eaf1-66ff-405a-a9e4-ec3340812a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6801","nonce":540367751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6803","nonce":540367751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6807","nonce":540367751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6805","nonce":540367751}]},"public_addr":"172.21.15.154:6801/540367751","cluster_addr":"172.21.15.154:6803/540367751","heartbeat_back_addr":"172.21.15.154:6807/540367751","heartbeat_front_addr":"172.21.15.154:6805/540367751","state":["exists","up"]},{"osd":6,"uuid":"267d02ea-675f-428f-aae8-e0375e4d4feb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6825","nonce":1550829005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6827","nonce":1550829005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6831","nonce":1550829005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6829","nonce":1550829005}]},"public_addr":"172.21.15.154:6825/1550829005","cluster_addr":"172.21.15.154:6827/1550829005","heartbeat_back_addr":"172.21.15.154:6831/1550829005","heartbeat_front_addr":"172.21.15.154:6829/1550829005","state":["exists","up"]},{"osd":7,"uuid":"759ac6cc-896a-423c-885f-9fe90d8dcb14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6809","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6811","nonce":3920089903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6816","nonce":3920089903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6823","nonce":3920089903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6819","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6821","nonce":3920089903}]},"public_addr":"172.21.15.154:6811/3920089903","cluster_addr":"172.21.15.154:6816/3920089903","heartbeat_back_addr":"172.21.15.154:6823/3920089903","heartbeat_front_addr":"172.21.15.154:6821/3920089903","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.949509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.984072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.907278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.947391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.975447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.998896+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:09.028739+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.894119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:19:47.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.809+0000 7f53dc631640 1 -- 172.21.15.105:0/2994370332 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f53b806c8f0 msgr2=0x7f53b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:47.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.809+0000 7f53dc631640 1 --2- 172.21.15.105:0/2994370332 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f53b806c8f0 0x7f53b806edb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f53c4036000 tx=0x7f53c4008030 comp rx=0 tx=0).stop 2024-04-15T15:19:47.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.809+0000 7f53dc631640 1 -- 172.21.15.105:0/2994370332 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53d406f4d0 msgr2=0x7f53d4154c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:47.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.809+0000 7f53dc631640 1 --2- 172.21.15.105:0/2994370332 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53d406f4d0 0x7f53d4154c70 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f53d000c430 tx=0x7f53d000c8f0 comp rx=0 tx=0).stop 2024-04-15T15:19:47.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.809+0000 7f53dc631640 1 -- 172.21.15.105:0/2994370332 shutdown_connections 2024-04-15T15:19:47.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.809+0000 7f53dc631640 1 --2- 172.21.15.105:0/2994370332 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f53d413e680 0x7f53d4143da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:47.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.809+0000 7f53dc631640 1 --2- 172.21.15.105:0/2994370332 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f53b806c8f0 0x7f53b806edb0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:47.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.809+0000 7f53dc631640 1 --2- 172.21.15.105:0/2994370332 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53d413ec60 0x7f53d41442e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:47.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.809+0000 7f53dc631640 1 --2- 172.21.15.105:0/2994370332 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53d406f4d0 0x7f53d4154c70 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:47.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.809+0000 7f53dc631640 1 -- 172.21.15.105:0/2994370332 >> 172.21.15.105:0/2994370332 conn(0x7f53d4067d50 msgr2=0x7f53d406b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:47.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.809+0000 7f53dc631640 1 -- 172.21.15.105:0/2994370332 shutdown_connections 2024-04-15T15:19:47.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.809+0000 7f53dc631640 1 -- 172.21.15.105:0/2994370332 wait complete. 2024-04-15T15:19:47.822 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:19:13.748585+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-15T15:19:23.789287+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-15T15:19:24.722250+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:19:47.823 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:19:47.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.927+0000 7fead759d640 1 Processor -- start 2024-04-15T15:19:47.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.927+0000 7fead759d640 1 -- start start 2024-04-15T15:19:47.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.928+0000 7fead759d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fead0144c90 0x7fead00b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:47.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.928+0000 7fead759d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fead006ef90 0x7fead006f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:47.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.928+0000 7fead759d640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fead00b40a0 con 0x7fead00cd1d0 2024-04-15T15:19:47.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.928+0000 7fead759d640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fead006f940 con 0x7fead0144c90 2024-04-15T15:19:47.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.928+0000 7fead759d640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fead00cdcf0 con 0x7fead006ef90 2024-04-15T15:19:47.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.928+0000 7fead5339640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fead0144c90 0x7fead00b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:47.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.928+0000 7fead5b3a640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fead00cd1d0 0x7fead00cd5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:46378/0 (socket says 172.21.15.105:46378) 2024-04-15T15:19:47.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.928+0000 7fead5b3a640 1 -- 172.21.15.105:0/738216594 learned_addr learned my addr 172.21.15.105:0/738216594 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:19:47.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.928+0000 7fead4b38640 1 --2- 172.21.15.105:0/738216594 >> v2:172.21.15.154:3300/0 conn(0x7fead006ef90 0x7fead006f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:47.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.928+0000 7fead5339640 1 -- 172.21.15.105:0/738216594 >> v2:172.21.15.154:3300/0 conn(0x7fead006ef90 msgr2=0x7fead006f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:47.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.928+0000 7fead5339640 1 --2- 172.21.15.105:0/738216594 >> v2:172.21.15.154:3300/0 conn(0x7fead006ef90 0x7fead006f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:47.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.928+0000 7fead5339640 1 -- 172.21.15.105:0/738216594 >> v1:172.21.15.105:6789/0 conn(0x7fead00cd1d0 legacy=0x7fead00cd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:47.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.928+0000 7fead5339640 1 -- 172.21.15.105:0/738216594 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fead00b4380 con 0x7fead0144c90 2024-04-15T15:19:47.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.928+0000 7feac7fff640 1 -- 172.21.15.105:0/738216594 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4010767687 0 0) 0x7fead00b40a0 con 0x7fead00cd1d0 2024-04-15T15:19:47.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.928+0000 7fead4b38640 1 --2- 172.21.15.105:0/738216594 >> v2:172.21.15.154:3300/0 conn(0x7fead006ef90 0x7fead006f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:47.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.929+0000 7fead5339640 1 --2- 172.21.15.105:0/738216594 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fead0144c90 0x7fead00b4e60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fead00b0100 tx=0x7feacc02e010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81ee02acda2d5c2e server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:47.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.929+0000 7feac7fff640 1 -- 172.21.15.105:0/738216594 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feacc03b070 con 0x7fead0144c90 2024-04-15T15:19:47.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.929+0000 7feac7fff640 1 -- 172.21.15.105:0/738216594 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feacc02edc0 con 0x7fead0144c90 2024-04-15T15:19:47.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.929+0000 7feac7fff640 1 -- 172.21.15.105:0/738216594 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feacc036660 con 0x7fead0144c90 2024-04-15T15:19:47.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.930+0000 7fead759d640 1 -- 172.21.15.105:0/738216594 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fead0144c90 msgr2=0x7fead00b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:47.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.930+0000 7fead759d640 1 --2- 172.21.15.105:0/738216594 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fead0144c90 0x7fead00b4e60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fead00b0100 tx=0x7feacc02e010 comp rx=0 tx=0).stop 2024-04-15T15:19:47.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.930+0000 7fead759d640 1 -- 172.21.15.105:0/738216594 shutdown_connections 2024-04-15T15:19:47.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.930+0000 7fead759d640 1 --2- 172.21.15.105:0/738216594 >> v2:172.21.15.154:3300/0 conn(0x7fead006ef90 0x7fead006f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:47.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.930+0000 7fead759d640 1 --2- 172.21.15.105:0/738216594 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fead0144c90 0x7fead00b4e60 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:47.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.930+0000 7fead759d640 1 -- 172.21.15.105:0/738216594 >> 172.21.15.105:0/738216594 conn(0x7fead0067d50 msgr2=0x7fead006b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:47.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.930+0000 7fead759d640 1 -- 172.21.15.105:0/738216594 shutdown_connections 2024-04-15T15:19:47.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.930+0000 7fead759d640 1 -- 172.21.15.105:0/738216594 wait complete. 2024-04-15T15:19:47.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.931+0000 7fead759d640 1 Processor -- start 2024-04-15T15:19:47.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.931+0000 7fead759d640 1 -- start start 2024-04-15T15:19:47.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.932+0000 7fead759d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fead006ef90 0x7fead00cc730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:47.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.932+0000 7fead759d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fead00cd1d0 0x7fead0150a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:47.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.932+0000 7fead759d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fead0144c90 0x7fead015cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:47.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.932+0000 7fead759d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fead00b4940 con 0x7fead006ef90 2024-04-15T15:19:47.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.932+0000 7fead759d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fead00b3e50 con 0x7fead00cd1d0 2024-04-15T15:19:47.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.932+0000 7fead759d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fead0145070 con 0x7fead0144c90 2024-04-15T15:19:47.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.932+0000 7fead5339640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fead006ef90 0x7fead00cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:47.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.932+0000 7fead5339640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fead006ef90 0x7fead00cc730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51612/0 (socket says 172.21.15.105:51612) 2024-04-15T15:19:47.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.932+0000 7fead5339640 1 -- 172.21.15.105:0/1676359070 learned_addr learned my addr 172.21.15.105:0/1676359070 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:47.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.932+0000 7fead5b3a640 1 --2- 172.21.15.105:0/1676359070 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fead0144c90 0x7fead015cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:47.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.932+0000 7fead4b38640 1 --2- 172.21.15.105:0/1676359070 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fead00cd1d0 0x7fead0150a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:47.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.932+0000 7fead5339640 1 -- 172.21.15.105:0/1676359070 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fead0144c90 msgr2=0x7fead015cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:47.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.932+0000 7fead5339640 1 --2- 172.21.15.105:0/1676359070 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fead0144c90 0x7fead015cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:47.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.932+0000 7fead5339640 1 -- 172.21.15.105:0/1676359070 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fead00cd1d0 msgr2=0x7fead0150a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:47.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.932+0000 7fead5339640 1 --2- 172.21.15.105:0/1676359070 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fead00cd1d0 0x7fead0150a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:47.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.932+0000 7fead5339640 1 -- 172.21.15.105:0/1676359070 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fead013db60 con 0x7fead006ef90 2024-04-15T15:19:47.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.932+0000 7fead5339640 1 --2- 172.21.15.105:0/1676359070 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fead006ef90 0x7fead00cc730 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7feacc02e620 tx=0x7feacc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:47.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.932+0000 7feac5ffb640 1 -- 172.21.15.105:0/1676359070 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feacc047070 con 0x7fead006ef90 2024-04-15T15:19:47.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.932+0000 7feac5ffb640 1 -- 172.21.15.105:0/1676359070 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feacc03c040 con 0x7fead006ef90 2024-04-15T15:19:47.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.932+0000 7fead759d640 1 -- 172.21.15.105:0/1676359070 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fead013d880 con 0x7fead006ef90 2024-04-15T15:19:47.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.932+0000 7feac5ffb640 1 -- 172.21.15.105:0/1676359070 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feacc03b050 con 0x7fead006ef90 2024-04-15T15:19:47.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.933+0000 7fead759d640 1 -- 172.21.15.105:0/1676359070 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fead00c57a0 con 0x7fead006ef90 2024-04-15T15:19:47.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.934+0000 7fead759d640 1 -- 172.21.15.105:0/1676359070 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fead0068ab0 con 0x7fead006ef90 2024-04-15T15:19:47.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.934+0000 7feac5ffb640 1 -- 172.21.15.105:0/1676359070 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feacc04a070 con 0x7fead006ef90 2024-04-15T15:19:47.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.934+0000 7feac5ffb640 1 --2- 172.21.15.105:0/1676359070 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7feab406c8f0 0x7feab406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:47.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.935+0000 7feac5ffb640 1 -- 172.21.15.105:0/1676359070 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7feacc0afba0 con 0x7fead006ef90 2024-04-15T15:19:47.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.935+0000 7fead4b38640 1 --2- 172.21.15.105:0/1676359070 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7feab406c8f0 0x7feab406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:47.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.935+0000 7fead4b38640 1 --2- 172.21.15.105:0/1676359070 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7feab406c8f0 0x7feab406edb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fead0143930 tx=0x7feac0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:47.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:47.939+0000 7feac5ffb640 1 -- 172.21.15.105:0/1676359070 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7feacc07e060 con 0x7fead006ef90 2024-04-15T15:19:48.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.116+0000 7fead759d640 1 -- 172.21.15.105:0/1676359070 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fead00cffb0 con 0x7fead006ef90 2024-04-15T15:19:48.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.116+0000 7feac5ffb640 1 -- 172.21.15.105:0/1676359070 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7feacc07d960 con 0x7fead006ef90 2024-04-15T15:19:48.118 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-04-15T15:19:48.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.118+0000 7fead759d640 1 -- 172.21.15.105:0/1676359070 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7feab406c8f0 msgr2=0x7feab406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:48.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.118+0000 7fead759d640 1 --2- 172.21.15.105:0/1676359070 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7feab406c8f0 0x7feab406edb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fead0143930 tx=0x7feac0008040 comp rx=0 tx=0).stop 2024-04-15T15:19:48.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.118+0000 7fead759d640 1 -- 172.21.15.105:0/1676359070 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fead006ef90 msgr2=0x7fead00cc730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:48.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.118+0000 7fead759d640 1 --2- 172.21.15.105:0/1676359070 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fead006ef90 0x7fead00cc730 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7feacc02e620 tx=0x7feacc0046c0 comp rx=0 tx=0).stop 2024-04-15T15:19:48.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.118+0000 7fead759d640 1 -- 172.21.15.105:0/1676359070 shutdown_connections 2024-04-15T15:19:48.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.118+0000 7fead759d640 1 --2- 172.21.15.105:0/1676359070 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fead0144c90 0x7fead015cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:48.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.118+0000 7fead759d640 1 --2- 172.21.15.105:0/1676359070 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7feab406c8f0 0x7feab406edb0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:48.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.118+0000 7fead759d640 1 --2- 172.21.15.105:0/1676359070 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fead00cd1d0 0x7fead0150a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:48.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.118+0000 7fead759d640 1 --2- 172.21.15.105:0/1676359070 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fead006ef90 0x7fead00cc730 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:48.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.118+0000 7fead759d640 1 -- 172.21.15.105:0/1676359070 >> 172.21.15.105:0/1676359070 conn(0x7fead0067d50 msgr2=0x7fead0143e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:48.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.118+0000 7fead759d640 1 -- 172.21.15.105:0/1676359070 shutdown_connections 2024-04-15T15:19:48.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.118+0000 7fead759d640 1 -- 172.21.15.105:0/1676359070 wait complete. 2024-04-15T15:19:48.131 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-15T15:19:48.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.235+0000 7fc3bfec5640 1 Processor -- start 2024-04-15T15:19:48.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.235+0000 7fc3bfec5640 1 -- start start 2024-04-15T15:19:48.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.236+0000 7fc3bfec5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc3b80b9e80 0x7fc3b80cfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:48.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.236+0000 7fc3bfec5640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc3b80d1910 0x7fc3b8168690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:48.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.236+0000 7fc3bfec5640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3b80b7dd0 con 0x7fc3b8161df0 2024-04-15T15:19:48.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.236+0000 7fc3bfec5640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fc3b80b80b0 con 0x7fc3b80d1910 2024-04-15T15:19:48.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.236+0000 7fc3bfec5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fc3b80b8390 con 0x7fc3b80b9e80 2024-04-15T15:19:48.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.236+0000 7fc3be462640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc3b80d1910 0x7fc3b8168690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:48.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.236+0000 7fc3bd460640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc3b80b9e80 0x7fc3b80cfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:48.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.236+0000 7fc3be462640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc3b80d1910 0x7fc3b8168690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.105:33542/0 (socket says 172.21.15.105:33542) 2024-04-15T15:19:48.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.236+0000 7fc3bd460640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc3b80b9e80 0x7fc3b80cfaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.105:47950/0 (socket says 172.21.15.105:47950) 2024-04-15T15:19:48.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.236+0000 7fc3be462640 1 -- 172.21.15.105:0/2244341834 learned_addr learned my addr 172.21.15.105:0/2244341834 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:48.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.237+0000 7fc3be462640 1 -- 172.21.15.105:0/2244341834 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc3b80b9e80 msgr2=0x7fc3b80cfaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:48.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.237+0000 7fc3be462640 1 --2- 172.21.15.105:0/2244341834 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc3b80b9e80 0x7fc3b80cfaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:48.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.237+0000 7fc3be462640 1 -- 172.21.15.105:0/2244341834 >> v1:172.21.15.154:6789/0 conn(0x7fc3b8161df0 legacy=0x7fc3b80d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:48.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.237+0000 7fc3be462640 1 -- 172.21.15.105:0/2244341834 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3b80b8670 con 0x7fc3b80d1910 2024-04-15T15:19:48.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.237+0000 7fc3bcc5f640 1 -- 172.21.15.105:0/2244341834 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 553903105 0 0) 0x7fc3b80b7dd0 con 0x7fc3b8161df0 2024-04-15T15:19:48.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.237+0000 7fc3bd460640 1 --2- 172.21.15.105:0/2244341834 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc3b80b9e80 0x7fc3b80cfaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:19:48.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.237+0000 7fc3be462640 1 --2- 172.21.15.105:0/2244341834 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3b80d1910 0x7fc3b8168690 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc3b0009e50 tx=0x7fc3b002bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74a0b046f0a90966 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:48.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.237+0000 7fc3bcc5f640 1 -- 172.21.15.105:0/2244341834 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b002cb00 con 0x7fc3b80d1910 2024-04-15T15:19:48.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.238+0000 7fc3bcc5f640 1 -- 172.21.15.105:0/2244341834 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3b002cca0 con 0x7fc3b80d1910 2024-04-15T15:19:48.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.238+0000 7fc3bcc5f640 1 -- 172.21.15.105:0/2244341834 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b0034610 con 0x7fc3b80d1910 2024-04-15T15:19:48.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.238+0000 7fc3bfec5640 1 -- 172.21.15.105:0/2244341834 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3b80d1910 msgr2=0x7fc3b8168690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:48.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.238+0000 7fc3bfec5640 1 --2- 172.21.15.105:0/2244341834 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3b80d1910 0x7fc3b8168690 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc3b0009e50 tx=0x7fc3b002bf60 comp rx=0 tx=0).stop 2024-04-15T15:19:48.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.238+0000 7fc3bfec5640 1 -- 172.21.15.105:0/2244341834 shutdown_connections 2024-04-15T15:19:48.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.238+0000 7fc3bfec5640 1 --2- 172.21.15.105:0/2244341834 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3b80d1910 0x7fc3b8168690 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:48.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.238+0000 7fc3bfec5640 1 --2- 172.21.15.105:0/2244341834 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc3b80b9e80 0x7fc3b80cfaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:48.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.238+0000 7fc3bfec5640 1 -- 172.21.15.105:0/2244341834 >> 172.21.15.105:0/2244341834 conn(0x7fc3b8067720 msgr2=0x7fc3b8067b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:48.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.238+0000 7fc3bfec5640 1 -- 172.21.15.105:0/2244341834 shutdown_connections 2024-04-15T15:19:48.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.238+0000 7fc3bfec5640 1 -- 172.21.15.105:0/2244341834 wait complete. 2024-04-15T15:19:48.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.239+0000 7fc3bfec5640 1 Processor -- start 2024-04-15T15:19:48.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.239+0000 7fc3bfec5640 1 -- start start 2024-04-15T15:19:48.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.240+0000 7fc3bfec5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3b80b9e80 0x7fc3b81f85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:48.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.240+0000 7fc3bfec5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3b80d1910 0x7fc3b81f48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:48.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.240+0000 7fc3bfec5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc3b8161df0 0x7fc3b814a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:48.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.240+0000 7fc3bfec5640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b81621d0 con 0x7fc3b80d1910 2024-04-15T15:19:48.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.240+0000 7fc3bfec5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b80b7a80 con 0x7fc3b80b9e80 2024-04-15T15:19:48.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.240+0000 7fc3bfec5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fc3b8162350 con 0x7fc3b8161df0 2024-04-15T15:19:48.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.240+0000 7fc3bd460640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3b80d1910 0x7fc3b81f48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:48.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.240+0000 7fc3bd460640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3b80d1910 0x7fc3b81f48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51616/0 (socket says 172.21.15.105:51616) 2024-04-15T15:19:48.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.240+0000 7fc3bd460640 1 -- 172.21.15.105:0/2231898137 learned_addr learned my addr 172.21.15.105:0/2231898137 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:48.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.240+0000 7fc3be462640 1 --2- 172.21.15.105:0/2231898137 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc3b8161df0 0x7fc3b814a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:48.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.240+0000 7fc3bdc61640 1 --2- 172.21.15.105:0/2231898137 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3b80b9e80 0x7fc3b81f85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:48.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.241+0000 7fc3bd460640 1 -- 172.21.15.105:0/2231898137 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc3b8161df0 msgr2=0x7fc3b814a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:48.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.241+0000 7fc3bd460640 1 --2- 172.21.15.105:0/2231898137 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc3b8161df0 0x7fc3b814a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:48.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.241+0000 7fc3bd460640 1 -- 172.21.15.105:0/2231898137 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3b80b9e80 msgr2=0x7fc3b81f85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:48.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.241+0000 7fc3bd460640 1 --2- 172.21.15.105:0/2231898137 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3b80b9e80 0x7fc3b81f85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:48.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.241+0000 7fc3bd460640 1 -- 172.21.15.105:0/2231898137 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3b81f8af0 con 0x7fc3b80d1910 2024-04-15T15:19:48.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.241+0000 7fc3bd460640 1 --2- 172.21.15.105:0/2231898137 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3b80d1910 0x7fc3b81f48b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc3a800e990 tx=0x7fc3a800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:48.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.241+0000 7fc3ae7fc640 1 -- 172.21.15.105:0/2231898137 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a800c9e0 con 0x7fc3b80d1910 2024-04-15T15:19:48.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.241+0000 7fc3bfec5640 1 -- 172.21.15.105:0/2231898137 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3b80c98d0 con 0x7fc3b80d1910 2024-04-15T15:19:48.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.241+0000 7fc3ae7fc640 1 -- 172.21.15.105:0/2231898137 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3a800cb80 con 0x7fc3b80d1910 2024-04-15T15:19:48.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.241+0000 7fc3ae7fc640 1 -- 172.21.15.105:0/2231898137 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a8005990 con 0x7fc3b80d1910 2024-04-15T15:19:48.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.241+0000 7fc3bfec5640 1 -- 172.21.15.105:0/2231898137 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3b80c9e90 con 0x7fc3b80d1910 2024-04-15T15:19:48.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.242+0000 7fc3bfec5640 1 -- 172.21.15.105:0/2231898137 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3b8068a40 con 0x7fc3b80d1910 2024-04-15T15:19:48.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.242+0000 7fc3ae7fc640 1 -- 172.21.15.105:0/2231898137 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc3a8005b30 con 0x7fc3b80d1910 2024-04-15T15:19:48.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.243+0000 7fc3ae7fc640 1 --2- 172.21.15.105:0/2231898137 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fc39c06c920 0x7fc39c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:48.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.243+0000 7fc3ae7fc640 1 -- 172.21.15.105:0/2231898137 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc3a808be20 con 0x7fc3b80d1910 2024-04-15T15:19:48.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.243+0000 7fc3bdc61640 1 --2- 172.21.15.105:0/2231898137 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fc39c06c920 0x7fc39c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:48.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.243+0000 7fc3bdc61640 1 --2- 172.21.15.105:0/2231898137 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fc39c06c920 0x7fc39c06ede0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4000f60 tx=0x7fc3b4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:48.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.247+0000 7fc3ae7fc640 1 -- 172.21.15.105:0/2231898137 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc3b81f8af0 con 0x7fc3b80d1910 2024-04-15T15:19:48.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.420+0000 7fc3bfec5640 1 -- 172.21.15.105:0/2231898137 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc3b80c95f0 con 0x7fc3b80d1910 2024-04-15T15:19:48.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.421+0000 7fc3ae7fc640 1 -- 172.21.15.105:0/2231898137 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fc3a8017020 con 0x7fc3b80d1910 2024-04-15T15:19:48.423 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-04-15T15:19:48.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.423+0000 7fc3bfec5640 1 -- 172.21.15.105:0/2231898137 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fc39c06c920 msgr2=0x7fc39c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:48.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.423+0000 7fc3bfec5640 1 --2- 172.21.15.105:0/2231898137 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fc39c06c920 0x7fc39c06ede0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4000f60 tx=0x7fc3b4009040 comp rx=0 tx=0).stop 2024-04-15T15:19:48.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.423+0000 7fc3bfec5640 1 -- 172.21.15.105:0/2231898137 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3b80d1910 msgr2=0x7fc3b81f48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:48.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.423+0000 7fc3bfec5640 1 --2- 172.21.15.105:0/2231898137 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3b80d1910 0x7fc3b81f48b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc3a800e990 tx=0x7fc3a800ee50 comp rx=0 tx=0).stop 2024-04-15T15:19:48.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.423+0000 7fc3bfec5640 1 -- 172.21.15.105:0/2231898137 shutdown_connections 2024-04-15T15:19:48.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.423+0000 7fc3bfec5640 1 --2- 172.21.15.105:0/2231898137 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc3b8161df0 0x7fc3b814a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:48.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.423+0000 7fc3bfec5640 1 --2- 172.21.15.105:0/2231898137 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fc39c06c920 0x7fc39c06ede0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:48.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.423+0000 7fc3bfec5640 1 --2- 172.21.15.105:0/2231898137 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3b80d1910 0x7fc3b81f48b0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:48.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.423+0000 7fc3bfec5640 1 --2- 172.21.15.105:0/2231898137 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3b80b9e80 0x7fc3b81f85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:48.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.423+0000 7fc3bfec5640 1 -- 172.21.15.105:0/2231898137 >> 172.21.15.105:0/2231898137 conn(0x7fc3b8067720 msgr2=0x7fc3b80b3ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:48.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.423+0000 7fc3bfec5640 1 -- 172.21.15.105:0/2231898137 shutdown_connections 2024-04-15T15:19:48.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.423+0000 7fc3bfec5640 1 -- 172.21.15.105:0/2231898137 wait complete. 2024-04-15T15:19:48.436 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-15T15:19:48.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.542+0000 7ff1c5715640 1 Processor -- start 2024-04-15T15:19:48.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.542+0000 7ff1c5715640 1 -- start start 2024-04-15T15:19:48.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.543+0000 7ff1c5715640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff1c015e8d0 0x7ff1c00d2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:48.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.543+0000 7ff1c5715640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff1c00ba480 0x7ff1c00d1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:48.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.543+0000 7ff1c5715640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1c00d3400 con 0x7ff1c00d1650 2024-04-15T15:19:48.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.543+0000 7ff1c5715640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7ff1c00d1ac0 con 0x7ff1c015e8d0 2024-04-15T15:19:48.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.543+0000 7ff1c5715640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7ff1c00d36e0 con 0x7ff1c00ba480 2024-04-15T15:19:48.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.543+0000 7ff1beffd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff1c015e8d0 0x7ff1c00d2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:48.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.543+0000 7ff1bf7fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7ff1c00d1650 0x7ff1c00b61d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:46380/0 (socket says 172.21.15.105:46380) 2024-04-15T15:19:48.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.543+0000 7ff1bf7fe640 1 -- 172.21.15.105:0/1943921718 learned_addr learned my addr 172.21.15.105:0/1943921718 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:19:48.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.543+0000 7ff1b67fc640 1 --2- 172.21.15.105:0/1943921718 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff1c00ba480 0x7ff1c00d1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:48.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.543+0000 7ff1beffd640 1 -- 172.21.15.105:0/1943921718 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff1c00ba480 msgr2=0x7ff1c00d1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:48.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.543+0000 7ff1beffd640 1 --2- 172.21.15.105:0/1943921718 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff1c00ba480 0x7ff1c00d1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:48.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.543+0000 7ff1beffd640 1 -- 172.21.15.105:0/1943921718 >> v1:172.21.15.105:6789/0 conn(0x7ff1c00d1650 legacy=0x7ff1c00b61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:48.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.543+0000 7ff1beffd640 1 -- 172.21.15.105:0/1943921718 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1c00d39c0 con 0x7ff1c015e8d0 2024-04-15T15:19:48.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.544+0000 7ff1beffd640 1 --2- 172.21.15.105:0/1943921718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1c015e8d0 0x7ff1c00d2ab0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff1b00097a0 tx=0x7ff1b002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6abfafde7269678f server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:48.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.544+0000 7ff1be7fc640 1 -- 172.21.15.105:0/1943921718 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b0034070 con 0x7ff1c015e8d0 2024-04-15T15:19:48.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.544+0000 7ff1be7fc640 1 -- 172.21.15.105:0/1943921718 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1b0030610 con 0x7ff1c015e8d0 2024-04-15T15:19:48.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.544+0000 7ff1be7fc640 1 -- 172.21.15.105:0/1943921718 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b00309e0 con 0x7ff1c015e8d0 2024-04-15T15:19:48.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.544+0000 7ff1c5715640 1 -- 172.21.15.105:0/1943921718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1c015e8d0 msgr2=0x7ff1c00d2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:48.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.544+0000 7ff1c5715640 1 --2- 172.21.15.105:0/1943921718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1c015e8d0 0x7ff1c00d2ab0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff1b00097a0 tx=0x7ff1b002b960 comp rx=0 tx=0).stop 2024-04-15T15:19:48.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.544+0000 7ff1c5715640 1 -- 172.21.15.105:0/1943921718 shutdown_connections 2024-04-15T15:19:48.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.544+0000 7ff1c5715640 1 --2- 172.21.15.105:0/1943921718 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff1c00ba480 0x7ff1c00d1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:48.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.544+0000 7ff1c5715640 1 --2- 172.21.15.105:0/1943921718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1c015e8d0 0x7ff1c00d2ab0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:48.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.544+0000 7ff1c5715640 1 -- 172.21.15.105:0/1943921718 >> 172.21.15.105:0/1943921718 conn(0x7ff1c006d460 msgr2=0x7ff1c0067720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:48.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.545+0000 7ff1c5715640 1 -- 172.21.15.105:0/1943921718 shutdown_connections 2024-04-15T15:19:48.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.545+0000 7ff1c5715640 1 -- 172.21.15.105:0/1943921718 wait complete. 2024-04-15T15:19:48.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.546+0000 7ff1c5715640 1 Processor -- start 2024-04-15T15:19:48.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.546+0000 7ff1c5715640 1 -- start start 2024-04-15T15:19:48.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.546+0000 7ff1c5715640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1c00ba480 0x7ff1c0156150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:48.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.546+0000 7ff1c5715640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1c00d1650 0x7ff1c00c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:48.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.547+0000 7ff1c5715640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff1c015e8d0 0x7ff1c00c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:48.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.547+0000 7ff1c5715640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c00d44a0 con 0x7ff1c00ba480 2024-04-15T15:19:48.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.547+0000 7ff1c5715640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c00d3fc0 con 0x7ff1c00d1650 2024-04-15T15:19:48.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.547+0000 7ff1c5715640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7ff1c00b8890 con 0x7ff1c015e8d0 2024-04-15T15:19:48.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.547+0000 7ff1beffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1c00ba480 0x7ff1c0156150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:48.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.547+0000 7ff1beffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1c00ba480 0x7ff1c0156150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51636/0 (socket says 172.21.15.105:51636) 2024-04-15T15:19:48.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.547+0000 7ff1beffd640 1 -- 172.21.15.105:0/3162895554 learned_addr learned my addr 172.21.15.105:0/3162895554 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:48.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.547+0000 7ff1b67fc640 1 --2- 172.21.15.105:0/3162895554 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1c00d1650 0x7ff1c00c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:48.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.547+0000 7ff1bf7fe640 1 --2- 172.21.15.105:0/3162895554 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff1c015e8d0 0x7ff1c00c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:48.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.547+0000 7ff1beffd640 1 -- 172.21.15.105:0/3162895554 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff1c015e8d0 msgr2=0x7ff1c00c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:48.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.547+0000 7ff1beffd640 1 --2- 172.21.15.105:0/3162895554 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff1c015e8d0 0x7ff1c00c92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:48.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.547+0000 7ff1beffd640 1 -- 172.21.15.105:0/3162895554 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1c00d1650 msgr2=0x7ff1c00c8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:48.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.547+0000 7ff1beffd640 1 --2- 172.21.15.105:0/3162895554 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1c00d1650 0x7ff1c00c8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:48.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.547+0000 7ff1beffd640 1 -- 172.21.15.105:0/3162895554 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1c00ce270 con 0x7ff1c00ba480 2024-04-15T15:19:48.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.547+0000 7ff1beffd640 1 --2- 172.21.15.105:0/3162895554 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1c00ba480 0x7ff1c0156150 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff1b002bf70 tx=0x7ff1b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:48.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.547+0000 7ff1b7fff640 1 -- 172.21.15.105:0/3162895554 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b0034040 con 0x7ff1c00ba480 2024-04-15T15:19:48.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.547+0000 7ff1b7fff640 1 -- 172.21.15.105:0/3162895554 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1b0039a10 con 0x7ff1c00ba480 2024-04-15T15:19:48.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.547+0000 7ff1c5715640 1 -- 172.21.15.105:0/3162895554 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1c00ce510 con 0x7ff1c00ba480 2024-04-15T15:19:48.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.547+0000 7ff1b7fff640 1 -- 172.21.15.105:0/3162895554 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b0039e20 con 0x7ff1c00ba480 2024-04-15T15:19:48.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.547+0000 7ff1c5715640 1 -- 172.21.15.105:0/3162895554 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1c00ccb50 con 0x7ff1c00ba480 2024-04-15T15:19:48.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.548+0000 7ff1c5715640 1 -- 172.21.15.105:0/3162895554 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1c0068ab0 con 0x7ff1c00ba480 2024-04-15T15:19:48.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.549+0000 7ff1b7fff640 1 -- 172.21.15.105:0/3162895554 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff1b0037070 con 0x7ff1c00ba480 2024-04-15T15:19:48.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.549+0000 7ff1b7fff640 1 --2- 172.21.15.105:0/3162895554 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff1a006c8f0 0x7ff1a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:48.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.549+0000 7ff1b7fff640 1 -- 172.21.15.105:0/3162895554 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff1b00adbc0 con 0x7ff1c00ba480 2024-04-15T15:19:48.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.549+0000 7ff1b67fc640 1 --2- 172.21.15.105:0/3162895554 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff1a006c8f0 0x7ff1a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:48.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.550+0000 7ff1b67fc640 1 --2- 172.21.15.105:0/3162895554 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff1a006c8f0 0x7ff1a006edb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8006fd0 tx=0x7ff1a8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:48.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.553+0000 7ff1b7fff640 1 -- 172.21.15.105:0/3162895554 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff1b007bff0 con 0x7ff1c00ba480 2024-04-15T15:19:48.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.729+0000 7ff1c5715640 1 -- 172.21.15.105:0/3162895554 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff1c0069150 con 0x7ff1c00ba480 2024-04-15T15:19:48.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.729+0000 7ff1b7fff640 1 -- 172.21.15.105:0/3162895554 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7ff1b007b8f0 con 0x7ff1c00ba480 2024-04-15T15:19:48.731 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-04-15T15:19:48.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.731+0000 7ff1c5715640 1 -- 172.21.15.105:0/3162895554 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff1a006c8f0 msgr2=0x7ff1a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:48.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.731+0000 7ff1c5715640 1 --2- 172.21.15.105:0/3162895554 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff1a006c8f0 0x7ff1a006edb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8006fd0 tx=0x7ff1a8008040 comp rx=0 tx=0).stop 2024-04-15T15:19:48.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.731+0000 7ff1c5715640 1 -- 172.21.15.105:0/3162895554 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1c00ba480 msgr2=0x7ff1c0156150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:48.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.731+0000 7ff1c5715640 1 --2- 172.21.15.105:0/3162895554 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1c00ba480 0x7ff1c0156150 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff1b002bf70 tx=0x7ff1b00046c0 comp rx=0 tx=0).stop 2024-04-15T15:19:48.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.731+0000 7ff1c5715640 1 -- 172.21.15.105:0/3162895554 shutdown_connections 2024-04-15T15:19:48.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.731+0000 7ff1c5715640 1 --2- 172.21.15.105:0/3162895554 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff1c015e8d0 0x7ff1c00c92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:48.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.731+0000 7ff1c5715640 1 --2- 172.21.15.105:0/3162895554 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff1a006c8f0 0x7ff1a006edb0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:48.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.731+0000 7ff1c5715640 1 --2- 172.21.15.105:0/3162895554 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1c00d1650 0x7ff1c00c8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:48.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.731+0000 7ff1c5715640 1 --2- 172.21.15.105:0/3162895554 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1c00ba480 0x7ff1c0156150 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:48.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.731+0000 7ff1c5715640 1 -- 172.21.15.105:0/3162895554 >> 172.21.15.105:0/3162895554 conn(0x7ff1c006d460 msgr2=0x7ff1c00b3250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:48.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.731+0000 7ff1c5715640 1 -- 172.21.15.105:0/3162895554 shutdown_connections 2024-04-15T15:19:48.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.731+0000 7ff1c5715640 1 -- 172.21.15.105:0/3162895554 wait complete. 2024-04-15T15:19:48.744 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:19:48.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.846+0000 7f8fb8a55640 1 Processor -- start 2024-04-15T15:19:48.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.846+0000 7f8fb8a55640 1 -- start start 2024-04-15T15:19:48.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.847+0000 7f8fb8a55640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8fb415ecf0 0x7f8fb406e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:48.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.847+0000 7f8fb8a55640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8fb40d0f10 0x7f8fb40ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:48.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.847+0000 7f8fb8a55640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fb406d2a0 con 0x7f8fb40cd190 2024-04-15T15:19:48.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.847+0000 7f8fb8a55640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f8fb406d580 con 0x7f8fb415ecf0 2024-04-15T15:19:48.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.847+0000 7f8fb8a55640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f8fb406d860 con 0x7f8fb40d0f10 2024-04-15T15:19:48.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.847+0000 7f8fb2d9d640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f8fb40cd190 0x7f8fb409aa40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:46392/0 (socket says 172.21.15.105:46392) 2024-04-15T15:19:48.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.847+0000 7f8fb2d9d640 1 -- 172.21.15.105:0/4261863975 learned_addr learned my addr 172.21.15.105:0/4261863975 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:19:48.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.847+0000 7f8fb259c640 1 --2- 172.21.15.105:0/4261863975 >> v2:172.21.15.154:3300/0 conn(0x7f8fb415ecf0 0x7f8fb406e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:48.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.847+0000 7f8fa9d9b640 1 --2- 172.21.15.105:0/4261863975 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8fb40d0f10 0x7f8fb40ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:48.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.847+0000 7f8fb1d9b640 1 -- 172.21.15.105:0/4261863975 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3466170204 0 0) 0x7f8fb406d2a0 con 0x7f8fb40cd190 2024-04-15T15:19:48.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.847+0000 7f8fb1d9b640 1 -- 172.21.15.105:0/4261863975 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8fa0003520 con 0x7f8fb40cd190 2024-04-15T15:19:48.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.847+0000 7f8fb1d9b640 1 -- 172.21.15.105:0/4261863975 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2375151344 0 0) 0x7f8fa0003520 con 0x7f8fb40cd190 2024-04-15T15:19:48.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.847+0000 7f8fb1d9b640 1 -- 172.21.15.105:0/4261863975 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8fb40d0f10 msgr2=0x7f8fb40ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:48.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.847+0000 7f8fb1d9b640 1 --2- 172.21.15.105:0/4261863975 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8fb40d0f10 0x7f8fb40ccc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:48.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.847+0000 7f8fb1d9b640 1 -- 172.21.15.105:0/4261863975 >> v2:172.21.15.154:3300/0 conn(0x7f8fb415ecf0 msgr2=0x7f8fb406e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:48.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.847+0000 7f8fb1d9b640 1 --2- 172.21.15.105:0/4261863975 >> v2:172.21.15.154:3300/0 conn(0x7f8fb415ecf0 0x7f8fb406e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:48.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.847+0000 7f8fb1d9b640 1 -- 172.21.15.105:0/4261863975 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fb406db40 con 0x7f8fb40cd190 2024-04-15T15:19:48.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.847+0000 7f8fb1d9b640 1 -- 172.21.15.105:0/4261863975 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7f8fac0019f0 con 0x7f8fb40cd190 2024-04-15T15:19:48.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.847+0000 7f8fb259c640 1 --2- 172.21.15.105:0/4261863975 >> v2:172.21.15.154:3300/0 conn(0x7f8fb415ecf0 0x7f8fb406e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:19:48.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.847+0000 7f8fb1d9b640 1 -- 172.21.15.105:0/4261863975 >> v1:172.21.15.105:6789/0 conn(0x7f8fb40cd190 legacy=0x7f8fb409aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:48.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.847+0000 7f8fb1d9b640 1 --2- 172.21.15.105:0/4261863975 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8fa00039b0 0x7f8fa0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:48.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.847+0000 7f8fb1d9b640 1 --2- 172.21.15.105:0/4261863975 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8fb40d0f10 0x7f8fa0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:48.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.847+0000 7f8fb1d9b640 1 --2- 172.21.15.105:0/4261863975 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fa0004d40 0x7f8fa00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:48.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.847+0000 7f8fb1d9b640 1 -- 172.21.15.105:0/4261863975 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb406d580 con 0x7f8fa00039b0 2024-04-15T15:19:48.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.847+0000 7f8fb1d9b640 1 -- 172.21.15.105:0/4261863975 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb406d860 con 0x7f8fa0004d40 2024-04-15T15:19:48.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.847+0000 7f8fb1d9b640 1 -- 172.21.15.105:0/4261863975 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f8fa000b9c0 con 0x7f8fb40d0f10 2024-04-15T15:19:48.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.848+0000 7f8fb259c640 1 --2- 172.21.15.105:0/4261863975 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8fa00039b0 0x7f8fa0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:48.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.848+0000 7f8fa9d9b640 1 --2- 172.21.15.105:0/4261863975 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8fb40d0f10 0x7f8fa0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:48.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.848+0000 7f8fb2d9d640 1 --2- 172.21.15.105:0/4261863975 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fa0004d40 0x7f8fa00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:48.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.848+0000 7f8fb259c640 1 -- 172.21.15.105:0/4261863975 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8fb40d0f10 msgr2=0x7f8fa0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:48.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.848+0000 7f8fb259c640 1 --2- 172.21.15.105:0/4261863975 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8fb40d0f10 0x7f8fa0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:48.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.848+0000 7f8fb259c640 1 -- 172.21.15.105:0/4261863975 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fa0004d40 msgr2=0x7f8fa00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:48.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.848+0000 7f8fb259c640 1 --2- 172.21.15.105:0/4261863975 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fa0004d40 0x7f8fa00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:48.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.848+0000 7f8fb259c640 1 -- 172.21.15.105:0/4261863975 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fa0003520 con 0x7f8fa00039b0 2024-04-15T15:19:48.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.849+0000 7f8fb259c640 1 --2- 172.21.15.105:0/4261863975 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8fa00039b0 0x7f8fa0003dd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8fa4009c90 tx=0x7f8fa0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9097f4538df5ba7f server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:48.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.849+0000 7f8fb1d9b640 1 -- 172.21.15.105:0/4261863975 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa4039070 con 0x7f8fa00039b0 2024-04-15T15:19:48.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.849+0000 7f8fb1d9b640 1 -- 172.21.15.105:0/4261863975 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fa402c980 con 0x7f8fa00039b0 2024-04-15T15:19:48.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.849+0000 7f8fb1d9b640 1 -- 172.21.15.105:0/4261863975 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa4009860 con 0x7f8fa00039b0 2024-04-15T15:19:48.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.849+0000 7f8fb8a55640 1 -- 172.21.15.105:0/4261863975 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8fa00039b0 msgr2=0x7f8fa0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:48.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.849+0000 7f8fb8a55640 1 --2- 172.21.15.105:0/4261863975 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8fa00039b0 0x7f8fa0003dd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8fa4009c90 tx=0x7f8fa0005830 comp rx=0 tx=0).stop 2024-04-15T15:19:48.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.849+0000 7f8fb2d9d640 1 -- 172.21.15.105:0/4261863975 reap_dead start 2024-04-15T15:19:48.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.849+0000 7f8fb8a55640 1 -- 172.21.15.105:0/4261863975 shutdown_connections 2024-04-15T15:19:48.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.849+0000 7f8fb8a55640 1 -- 172.21.15.105:0/4261863975 >> 172.21.15.105:0/4261863975 conn(0x7f8fb40d2600 msgr2=0x7f8fb40d2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:48.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.849+0000 7f8fb8a55640 1 -- 172.21.15.105:0/4261863975 shutdown_connections 2024-04-15T15:19:48.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.849+0000 7f8fb8a55640 1 -- 172.21.15.105:0/4261863975 wait complete. 2024-04-15T15:19:48.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.851+0000 7f8fb8a55640 1 Processor -- start 2024-04-15T15:19:48.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.851+0000 7f8fb8a55640 1 -- start start 2024-04-15T15:19:48.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.852+0000 7f8fb8a55640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8fa00039b0 0x7f8fb41f5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:48.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.852+0000 7f8fb8a55640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8fb41f7160 0x7f8fb409cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:48.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.852+0000 7f8fb8a55640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fb409d470 0x7f8fb409e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:48.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.852+0000 7f8fb8a55640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb415f0d0 con 0x7f8fa00039b0 2024-04-15T15:19:48.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.852+0000 7f8fb8a55640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb40d1370 con 0x7f8fb409d470 2024-04-15T15:19:48.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.852+0000 7f8fb8a55640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f8fb415f250 con 0x7f8fb41f7160 2024-04-15T15:19:48.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.852+0000 7f8fb259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8fa00039b0 0x7f8fb41f5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:48.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.852+0000 7f8fb259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8fa00039b0 0x7f8fb41f5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51660/0 (socket says 172.21.15.105:51660) 2024-04-15T15:19:48.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.852+0000 7f8fb259c640 1 -- 172.21.15.105:0/3537335508 learned_addr learned my addr 172.21.15.105:0/3537335508 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:48.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.852+0000 7f8fa9d9b640 1 --2- 172.21.15.105:0/3537335508 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8fb41f7160 0x7f8fb409cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:48.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.852+0000 7f8fb2d9d640 1 --2- 172.21.15.105:0/3537335508 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fb409d470 0x7f8fb409e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:48.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.852+0000 7f8fb259c640 1 -- 172.21.15.105:0/3537335508 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8fb41f7160 msgr2=0x7f8fb409cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:48.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.852+0000 7f8fb259c640 1 --2- 172.21.15.105:0/3537335508 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8fb41f7160 0x7f8fb409cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:48.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.852+0000 7f8fb259c640 1 -- 172.21.15.105:0/3537335508 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fb409d470 msgr2=0x7f8fb409e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:48.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.852+0000 7f8fb259c640 1 --2- 172.21.15.105:0/3537335508 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fb409d470 0x7f8fb409e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:48.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.852+0000 7f8fb259c640 1 -- 172.21.15.105:0/3537335508 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fb41f5ac0 con 0x7f8fa00039b0 2024-04-15T15:19:48.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.852+0000 7f8fb259c640 1 --2- 172.21.15.105:0/3537335508 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8fa00039b0 0x7f8fb41f5580 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8fa4009db0 tx=0x7f8fa4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:48.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.852+0000 7f8fab7fe640 1 -- 172.21.15.105:0/3537335508 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa4039050 con 0x7f8fa00039b0 2024-04-15T15:19:48.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.852+0000 7f8fab7fe640 1 -- 172.21.15.105:0/3537335508 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fa4045070 con 0x7f8fa00039b0 2024-04-15T15:19:48.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.852+0000 7f8fb8a55640 1 -- 172.21.15.105:0/3537335508 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fb409d910 con 0x7f8fa00039b0 2024-04-15T15:19:48.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.852+0000 7f8fab7fe640 1 -- 172.21.15.105:0/3537335508 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa4009860 con 0x7f8fa00039b0 2024-04-15T15:19:48.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.852+0000 7f8fb8a55640 1 -- 172.21.15.105:0/3537335508 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fb40c6660 con 0x7f8fa00039b0 2024-04-15T15:19:48.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.853+0000 7f8fb8a55640 1 -- 172.21.15.105:0/3537335508 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fb4068a40 con 0x7f8fa00039b0 2024-04-15T15:19:48.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.854+0000 7f8fab7fe640 1 -- 172.21.15.105:0/3537335508 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8fa4040450 con 0x7f8fa00039b0 2024-04-15T15:19:48.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.854+0000 7f8fab7fe640 1 --2- 172.21.15.105:0/3537335508 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f8f9406c8f0 0x7f8f9406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:48.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.854+0000 7f8fa9d9b640 1 --2- 172.21.15.105:0/3537335508 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f8f9406c8f0 0x7f8f9406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:48.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.854+0000 7f8fab7fe640 1 -- 172.21.15.105:0/3537335508 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8fa40ad5b0 con 0x7f8fa00039b0 2024-04-15T15:19:48.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.855+0000 7f8fa9d9b640 1 --2- 172.21.15.105:0/3537335508 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f8f9406c8f0 0x7f8f9406edb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8fb40719e0 tx=0x7f8f9c009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:48.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:48.859+0000 7f8fab7fe640 1 -- 172.21.15.105:0/3537335508 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8fa404e020 con 0x7f8fa00039b0 2024-04-15T15:19:49.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.034+0000 7f8fb8a55640 1 -- 172.21.15.105:0/3537335508 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8fb40ce930 con 0x7f8fa00039b0 2024-04-15T15:19:49.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.035+0000 7f8fab7fe640 1 -- 172.21.15.105:0/3537335508 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16653 (secure 0 0 0) 0x7f8fa407bb30 con 0x7f8fa00039b0 2024-04-15T15:19:49.037 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:19:49.037 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":20,"fsid":"09e4d2a4-a324-4793-af77-644b25e520e1","created":"2024-04-15T15:19:01.524543+0000","modified":"2024-04-15T15:19:27.414890+0000","last_up_change":"2024-04-15T15:19:12.825815+0000","last_in_change":"2024-04-15T15:19:03.788126+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:19:13.748585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:19:23.789287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:19:24.722250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6f6292b-2d7e-4aa0-9d1c-6ddd0f97287c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6801","nonce":2873040054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6803","nonce":2873040054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6807","nonce":2873040054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6805","nonce":2873040054}]},"public_addr":"172.21.15.105:6801/2873040054","cluster_addr":"172.21.15.105:6803/2873040054","heartbeat_back_addr":"172.21.15.105:6807/2873040054","heartbeat_front_addr":"172.21.15.105:6805/2873040054","state":["exists","up"]},{"osd":1,"uuid":"8900966a-b393-41e3-8f93-e5a43bd23622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6818","nonce":2494994479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6821","nonce":2494994479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6828","nonce":2494994479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6823","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6825","nonce":2494994479}]},"public_addr":"172.21.15.105:6818/2494994479","cluster_addr":"172.21.15.105:6821/2494994479","heartbeat_back_addr":"172.21.15.105:6828/2494994479","heartbeat_front_addr":"172.21.15.105:6825/2494994479","state":["exists","up"]},{"osd":2,"uuid":"81a5dc7a-9325-4ff5-941d-a2d5fba21f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6809","nonce":3798408699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6811","nonce":3798408699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6815","nonce":3798408699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6813","nonce":3798408699}]},"public_addr":"172.21.15.105:6809/3798408699","cluster_addr":"172.21.15.105:6811/3798408699","heartbeat_back_addr":"172.21.15.105:6815/3798408699","heartbeat_front_addr":"172.21.15.105:6813/3798408699","state":["exists","up"]},{"osd":3,"uuid":"df766ed2-5c71-4805-96f4-3d4970edc960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6819","nonce":1920720186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6824","nonce":1920720186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6831","nonce":1920720186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6827","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6829","nonce":1920720186}]},"public_addr":"172.21.15.105:6819/1920720186","cluster_addr":"172.21.15.105:6824/1920720186","heartbeat_back_addr":"172.21.15.105:6831/1920720186","heartbeat_front_addr":"172.21.15.105:6829/1920720186","state":["exists","up"]},{"osd":4,"uuid":"e1039e03-28b8-49df-b951-c91930b93c9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6810","nonce":3516494456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6813","nonce":3516494456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6820","nonce":3516494456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6817","nonce":3516494456}]},"public_addr":"172.21.15.154:6810/3516494456","cluster_addr":"172.21.15.154:6813/3516494456","heartbeat_back_addr":"172.21.15.154:6820/3516494456","heartbeat_front_addr":"172.21.15.154:6817/3516494456","state":["exists","up"]},{"osd":5,"uuid":"a9f5eaf1-66ff-405a-a9e4-ec3340812a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6801","nonce":540367751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6803","nonce":540367751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6807","nonce":540367751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6805","nonce":540367751}]},"public_addr":"172.21.15.154:6801/540367751","cluster_addr":"172.21.15.154:6803/540367751","heartbeat_back_addr":"172.21.15.154:6807/540367751","heartbeat_front_addr":"172.21.15.154:6805/540367751","state":["exists","up"]},{"osd":6,"uuid":"267d02ea-675f-428f-aae8-e0375e4d4feb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6825","nonce":1550829005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6827","nonce":1550829005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6831","nonce":1550829005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6829","nonce":1550829005}]},"public_addr":"172.21.15.154:6825/1550829005","cluster_addr":"172.21.15.154:6827/1550829005","heartbeat_back_addr":"172.21.15.154:6831/1550829005","heartbeat_front_addr":"172.21.15.154:6829/1550829005","state":["exists","up"]},{"osd":7,"uuid":"759ac6cc-896a-423c-885f-9fe90d8dcb14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6809","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6811","nonce":3920089903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6816","nonce":3920089903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6823","nonce":3920089903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6819","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6821","nonce":3920089903}]},"public_addr":"172.21.15.154:6811/3920089903","cluster_addr":"172.21.15.154:6816/3920089903","heartbeat_back_addr":"172.21.15.154:6823/3920089903","heartbeat_front_addr":"172.21.15.154:6821/3920089903","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.949509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.984072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.907278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.947391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.975447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.998896+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:09.028739+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.894119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:19:49.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.037+0000 7f8fb8a55640 1 -- 172.21.15.105:0/3537335508 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f8f9406c8f0 msgr2=0x7f8f9406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:49.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.037+0000 7f8fb8a55640 1 --2- 172.21.15.105:0/3537335508 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f8f9406c8f0 0x7f8f9406edb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8fb40719e0 tx=0x7f8f9c009190 comp rx=0 tx=0).stop 2024-04-15T15:19:49.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.037+0000 7f8fb8a55640 1 -- 172.21.15.105:0/3537335508 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8fa00039b0 msgr2=0x7f8fb41f5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:49.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.037+0000 7f8fb8a55640 1 --2- 172.21.15.105:0/3537335508 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8fa00039b0 0x7f8fb41f5580 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8fa4009db0 tx=0x7f8fa4004730 comp rx=0 tx=0).stop 2024-04-15T15:19:49.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.037+0000 7f8fb8a55640 1 -- 172.21.15.105:0/3537335508 shutdown_connections 2024-04-15T15:19:49.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.037+0000 7f8fb8a55640 1 --2- 172.21.15.105:0/3537335508 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8fb41f7160 0x7f8fb409cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:49.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.037+0000 7f8fb8a55640 1 --2- 172.21.15.105:0/3537335508 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f8f9406c8f0 0x7f8f9406edb0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:49.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.037+0000 7f8fb8a55640 1 --2- 172.21.15.105:0/3537335508 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fb409d470 0x7f8fb409e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:49.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.037+0000 7f8fb8a55640 1 --2- 172.21.15.105:0/3537335508 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8fa00039b0 0x7f8fb41f5580 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:49.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.037+0000 7f8fb8a55640 1 -- 172.21.15.105:0/3537335508 >> 172.21.15.105:0/3537335508 conn(0x7f8fb40d2600 msgr2=0x7f8fb409aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:49.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.037+0000 7f8fb8a55640 1 -- 172.21.15.105:0/3537335508 shutdown_connections 2024-04-15T15:19:49.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.037+0000 7f8fb8a55640 1 -- 172.21.15.105:0/3537335508 wait complete. 2024-04-15T15:19:49.050 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:19:13.748585+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-15T15:19:23.789287+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-15T15:19:24.722250+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:19:49.051 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:19:49.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.155+0000 7f7b9fdbd640 1 Processor -- start 2024-04-15T15:19:49.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.156+0000 7f7b9fdbd640 1 -- start start 2024-04-15T15:19:49.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.156+0000 7f7b9fdbd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7b9806f6a0 0x7f7b980b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:49.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.156+0000 7f7b9fdbd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7b980cd1d0 0x7f7b980cd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:49.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.156+0000 7f7b9fdbd640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b980b40a0 con 0x7f7b98144c90 2024-04-15T15:19:49.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.156+0000 7f7b9fdbd640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f7b980b3e50 con 0x7f7b980cd1d0 2024-04-15T15:19:49.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.156+0000 7f7b9fdbd640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f7b980b4380 con 0x7f7b9806f6a0 2024-04-15T15:19:49.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.157+0000 7f7b9d358640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7b9806f6a0 0x7f7b980b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:49.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.157+0000 7f7b9e35a640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7b980cd1d0 0x7f7b980cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:49.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.157+0000 7f7b9db59640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f7b98144c90 0x7f7b9806ef90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.105:34122/0 (socket says 172.21.15.105:34122) 2024-04-15T15:19:49.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.157+0000 7f7b9d358640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7b9806f6a0 0x7f7b980b4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.105:48010/0 (socket says 172.21.15.105:48010) 2024-04-15T15:19:49.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.157+0000 7f7b9db59640 1 -- 172.21.15.105:0/2181194199 learned_addr learned my addr 172.21.15.105:0/2181194199 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:19:49.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.157+0000 7f7b9d358640 1 -- 172.21.15.105:0/2181194199 >> v2:172.21.15.154:3300/0 conn(0x7f7b980cd1d0 msgr2=0x7f7b980cd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:49.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.157+0000 7f7b9d358640 1 --2- 172.21.15.105:0/2181194199 >> v2:172.21.15.154:3300/0 conn(0x7f7b980cd1d0 0x7f7b980cd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:49.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.157+0000 7f7b9d358640 1 -- 172.21.15.105:0/2181194199 >> v1:172.21.15.154:6789/0 conn(0x7f7b98144c90 legacy=0x7f7b9806ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:49.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.157+0000 7f7b9d358640 1 -- 172.21.15.105:0/2181194199 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b980b4660 con 0x7f7b9806f6a0 2024-04-15T15:19:49.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.157+0000 7f7b9e35a640 1 --2- 172.21.15.105:0/2181194199 >> v2:172.21.15.154:3300/0 conn(0x7f7b980cd1d0 0x7f7b980cd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:19:49.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.158+0000 7f7b9d358640 1 --2- 172.21.15.105:0/2181194199 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7b9806f6a0 0x7f7b980b4e60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7b880097a0 tx=0x7f7b8802b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9cbe792ac744754 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:49.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.158+0000 7f7b9cb57640 1 -- 172.21.15.105:0/2181194199 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b880315a0 con 0x7f7b9806f6a0 2024-04-15T15:19:49.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.158+0000 7f7b9cb57640 1 -- 172.21.15.105:0/2181194199 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b88037070 con 0x7f7b9806f6a0 2024-04-15T15:19:49.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.158+0000 7f7b9cb57640 1 -- 172.21.15.105:0/2181194199 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b88031960 con 0x7f7b9806f6a0 2024-04-15T15:19:49.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.159+0000 7f7b9fdbd640 1 -- 172.21.15.105:0/2181194199 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7b9806f6a0 msgr2=0x7f7b980b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:49.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.159+0000 7f7b9fdbd640 1 --2- 172.21.15.105:0/2181194199 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7b9806f6a0 0x7f7b980b4e60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7b880097a0 tx=0x7f7b8802b840 comp rx=0 tx=0).stop 2024-04-15T15:19:49.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.159+0000 7f7b9fdbd640 1 -- 172.21.15.105:0/2181194199 shutdown_connections 2024-04-15T15:19:49.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.159+0000 7f7b9fdbd640 1 --2- 172.21.15.105:0/2181194199 >> v2:172.21.15.154:3300/0 conn(0x7f7b980cd1d0 0x7f7b980cd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:49.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.159+0000 7f7b9fdbd640 1 --2- 172.21.15.105:0/2181194199 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7b9806f6a0 0x7f7b980b4e60 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:49.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.159+0000 7f7b9fdbd640 1 -- 172.21.15.105:0/2181194199 >> 172.21.15.105:0/2181194199 conn(0x7f7b98067d50 msgr2=0x7f7b9806b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:49.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.159+0000 7f7b9fdbd640 1 -- 172.21.15.105:0/2181194199 shutdown_connections 2024-04-15T15:19:49.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.159+0000 7f7b9fdbd640 1 -- 172.21.15.105:0/2181194199 wait complete. 2024-04-15T15:19:49.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.160+0000 7f7b9fdbd640 1 Processor -- start 2024-04-15T15:19:49.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.160+0000 7f7b9fdbd640 1 -- start start 2024-04-15T15:19:49.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.161+0000 7f7b9fdbd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b9806f6a0 0x7f7b98154f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:49.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.161+0000 7f7b9fdbd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7b980cd1d0 0x7f7b9815bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:49.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.161+0000 7f7b9fdbd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b98144c90 0x7f7b9815c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:49.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.161+0000 7f7b9fdbd640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7b980cdce0 con 0x7f7b98144c90 2024-04-15T15:19:49.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.161+0000 7f7b9fdbd640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7b980cdb60 con 0x7f7b9806f6a0 2024-04-15T15:19:49.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.161+0000 7f7b9fdbd640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f7b98145070 con 0x7f7b980cd1d0 2024-04-15T15:19:49.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.161+0000 7f7b9e35a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b98144c90 0x7f7b9815c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:49.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.161+0000 7f7b9e35a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b98144c90 0x7f7b9815c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51674/0 (socket says 172.21.15.105:51674) 2024-04-15T15:19:49.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.161+0000 7f7b9e35a640 1 -- 172.21.15.105:0/2563395139 learned_addr learned my addr 172.21.15.105:0/2563395139 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:49.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.161+0000 7f7b9db59640 1 --2- 172.21.15.105:0/2563395139 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b9806f6a0 0x7f7b98154f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:49.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.161+0000 7f7b9d358640 1 --2- 172.21.15.105:0/2563395139 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7b980cd1d0 0x7f7b9815bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:49.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.161+0000 7f7b9e35a640 1 -- 172.21.15.105:0/2563395139 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7b980cd1d0 msgr2=0x7f7b9815bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:49.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.161+0000 7f7b9e35a640 1 --2- 172.21.15.105:0/2563395139 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7b980cd1d0 0x7f7b9815bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:49.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.161+0000 7f7b9e35a640 1 -- 172.21.15.105:0/2563395139 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b9806f6a0 msgr2=0x7f7b98154f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:49.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.161+0000 7f7b9e35a640 1 --2- 172.21.15.105:0/2563395139 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b9806f6a0 0x7f7b98154f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:49.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.161+0000 7f7b9e35a640 1 -- 172.21.15.105:0/2563395139 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b9812c380 con 0x7f7b98144c90 2024-04-15T15:19:49.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.161+0000 7f7b9db59640 1 --2- 172.21.15.105:0/2563395139 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b9806f6a0 0x7f7b98154f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:49.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.161+0000 7f7b9e35a640 1 --2- 172.21.15.105:0/2563395139 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b98144c90 0x7f7b9815c120 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7b90009f00 tx=0x7f7b9000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:49.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.161+0000 7f7b8e7fc640 1 -- 172.21.15.105:0/2563395139 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b90018030 con 0x7f7b98144c90 2024-04-15T15:19:49.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.161+0000 7f7b8e7fc640 1 -- 172.21.15.105:0/2563395139 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b9000ce50 con 0x7f7b98144c90 2024-04-15T15:19:49.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.161+0000 7f7b9fdbd640 1 -- 172.21.15.105:0/2563395139 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b9813ac80 con 0x7f7b98144c90 2024-04-15T15:19:49.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.162+0000 7f7b8e7fc640 1 -- 172.21.15.105:0/2563395139 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b90003bb0 con 0x7f7b98144c90 2024-04-15T15:19:49.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.162+0000 7f7b9fdbd640 1 -- 172.21.15.105:0/2563395139 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b9813b240 con 0x7f7b98144c90 2024-04-15T15:19:49.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.162+0000 7f7b9fdbd640 1 -- 172.21.15.105:0/2563395139 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b98068ab0 con 0x7f7b98144c90 2024-04-15T15:19:49.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.163+0000 7f7b8e7fc640 1 -- 172.21.15.105:0/2563395139 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7b90005d10 con 0x7f7b98144c90 2024-04-15T15:19:49.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.163+0000 7f7b8e7fc640 1 --2- 172.21.15.105:0/2563395139 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f7b7c06c8f0 0x7f7b7c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:49.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.163+0000 7f7b8e7fc640 1 -- 172.21.15.105:0/2563395139 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7b9008c5a0 con 0x7f7b98144c90 2024-04-15T15:19:49.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.164+0000 7f7b9db59640 1 --2- 172.21.15.105:0/2563395139 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f7b7c06c8f0 0x7f7b7c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:49.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.164+0000 7f7b9db59640 1 --2- 172.21.15.105:0/2563395139 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f7b7c06c8f0 0x7f7b7c06edb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7b9815c7c0 tx=0x7f7b94009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:49.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.168+0000 7f7b8e7fc640 1 -- 172.21.15.105:0/2563395139 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7b90022070 con 0x7f7b98144c90 2024-04-15T15:19:49.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.343+0000 7f7b9fdbd640 1 -- 172.21.15.105:0/2563395139 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7b980cd770 con 0x7f7b98144c90 2024-04-15T15:19:49.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.343+0000 7f7b8e7fc640 1 -- 172.21.15.105:0/2563395139 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f7b90010040 con 0x7f7b98144c90 2024-04-15T15:19:49.345 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-04-15T15:19:49.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.345+0000 7f7b9fdbd640 1 -- 172.21.15.105:0/2563395139 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f7b7c06c8f0 msgr2=0x7f7b7c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:49.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.345+0000 7f7b9fdbd640 1 --2- 172.21.15.105:0/2563395139 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f7b7c06c8f0 0x7f7b7c06edb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7b9815c7c0 tx=0x7f7b94009040 comp rx=0 tx=0).stop 2024-04-15T15:19:49.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.345+0000 7f7b9fdbd640 1 -- 172.21.15.105:0/2563395139 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b98144c90 msgr2=0x7f7b9815c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:49.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.345+0000 7f7b9fdbd640 1 --2- 172.21.15.105:0/2563395139 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b98144c90 0x7f7b9815c120 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7b90009f00 tx=0x7f7b9000c490 comp rx=0 tx=0).stop 2024-04-15T15:19:49.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.345+0000 7f7b9fdbd640 1 -- 172.21.15.105:0/2563395139 shutdown_connections 2024-04-15T15:19:49.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.345+0000 7f7b9fdbd640 1 --2- 172.21.15.105:0/2563395139 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7b980cd1d0 0x7f7b9815bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:49.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.345+0000 7f7b9fdbd640 1 --2- 172.21.15.105:0/2563395139 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f7b7c06c8f0 0x7f7b7c06edb0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:49.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.345+0000 7f7b9fdbd640 1 --2- 172.21.15.105:0/2563395139 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b98144c90 0x7f7b9815c120 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:49.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.345+0000 7f7b9fdbd640 1 --2- 172.21.15.105:0/2563395139 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b9806f6a0 0x7f7b98154f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:49.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.345+0000 7f7b9fdbd640 1 -- 172.21.15.105:0/2563395139 >> 172.21.15.105:0/2563395139 conn(0x7f7b98067d50 msgr2=0x7f7b980b2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:49.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.345+0000 7f7b9fdbd640 1 -- 172.21.15.105:0/2563395139 shutdown_connections 2024-04-15T15:19:49.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.345+0000 7f7b9fdbd640 1 -- 172.21.15.105:0/2563395139 wait complete. 2024-04-15T15:19:49.358 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-15T15:19:49.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.463+0000 7fb1d4a4a640 1 Processor -- start 2024-04-15T15:19:49.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.463+0000 7fb1d4a4a640 1 -- start start 2024-04-15T15:19:49.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.464+0000 7fb1d4a4a640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb1d01620c0 0x7fb1d00b7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:49.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.464+0000 7fb1d4a4a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1d00d2b40 0x7fb1d00ba190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:49.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.464+0000 7fb1d4a4a640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1d00d2fc0 con 0x7fb1d01625c0 2024-04-15T15:19:49.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.464+0000 7fb1d4a4a640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fb1d00b8c00 con 0x7fb1d01620c0 2024-04-15T15:19:49.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.464+0000 7fb1d4a4a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fb1d00b7ff0 con 0x7fb1d00d2b40 2024-04-15T15:19:49.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.465+0000 7fb1cdd9b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1d00d2b40 0x7fb1d00ba190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:49.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.465+0000 7fb1ce59c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb1d01620c0 0x7fb1d00b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:49.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.465+0000 7fb1cdd9b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1d00d2b40 0x7fb1d00ba190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.105:48038/0 (socket says 172.21.15.105:48038) 2024-04-15T15:19:49.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.465+0000 7fb1cdd9b640 1 -- 172.21.15.105:0/1830727290 learned_addr learned my addr 172.21.15.105:0/1830727290 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:49.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.465+0000 7fb1cdd9b640 1 -- 172.21.15.105:0/1830727290 >> v2:172.21.15.154:3300/0 conn(0x7fb1d01620c0 msgr2=0x7fb1d00b7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:49.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.465+0000 7fb1cdd9b640 1 --2- 172.21.15.105:0/1830727290 >> v2:172.21.15.154:3300/0 conn(0x7fb1d01620c0 0x7fb1d00b7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:49.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.465+0000 7fb1cdd9b640 1 -- 172.21.15.105:0/1830727290 >> v1:172.21.15.154:6789/0 conn(0x7fb1d01625c0 legacy=0x7fb1d00b9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:49.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.465+0000 7fb1cdd9b640 1 -- 172.21.15.105:0/1830727290 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1d00b82d0 con 0x7fb1d00d2b40 2024-04-15T15:19:49.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.466+0000 7fb1cdd9b640 1 --2- 172.21.15.105:0/1830727290 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1d00d2b40 0x7fb1d00ba190 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8009870 tx=0x7fb1b802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6256485693fa70c server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:49.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.466+0000 7fb1cd59a640 1 -- 172.21.15.105:0/1830727290 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b8034070 con 0x7fb1d00d2b40 2024-04-15T15:19:49.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.466+0000 7fb1cd59a640 1 -- 172.21.15.105:0/1830727290 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1b8030840 con 0x7fb1d00d2b40 2024-04-15T15:19:49.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.466+0000 7fb1cd59a640 1 -- 172.21.15.105:0/1830727290 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b8030c10 con 0x7fb1d00d2b40 2024-04-15T15:19:49.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.466+0000 7fb1d4a4a640 1 -- 172.21.15.105:0/1830727290 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1d00d2b40 msgr2=0x7fb1d00ba190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:49.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.466+0000 7fb1d4a4a640 1 --2- 172.21.15.105:0/1830727290 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1d00d2b40 0x7fb1d00ba190 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8009870 tx=0x7fb1b802b980 comp rx=0 tx=0).stop 2024-04-15T15:19:49.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.467+0000 7fb1d4a4a640 1 -- 172.21.15.105:0/1830727290 shutdown_connections 2024-04-15T15:19:49.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.467+0000 7fb1d4a4a640 1 --2- 172.21.15.105:0/1830727290 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1d00d2b40 0x7fb1d00ba190 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:49.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.467+0000 7fb1d4a4a640 1 --2- 172.21.15.105:0/1830727290 >> v2:172.21.15.154:3300/0 conn(0x7fb1d01620c0 0x7fb1d00b7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:49.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.467+0000 7fb1d4a4a640 1 -- 172.21.15.105:0/1830727290 >> 172.21.15.105:0/1830727290 conn(0x7fb1d006ccb0 msgr2=0x7fb1d006d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:49.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.467+0000 7fb1d4a4a640 1 -- 172.21.15.105:0/1830727290 shutdown_connections 2024-04-15T15:19:49.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.467+0000 7fb1d4a4a640 1 -- 172.21.15.105:0/1830727290 wait complete. 2024-04-15T15:19:49.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.468+0000 7fb1d4a4a640 1 Processor -- start 2024-04-15T15:19:49.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.468+0000 7fb1d4a4a640 1 -- start start 2024-04-15T15:19:49.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.469+0000 7fb1d4a4a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1d00d2b40 0x7fb1d0156610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:49.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.469+0000 7fb1d4a4a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1d01620c0 0x7fb1d0155220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:49.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.469+0000 7fb1d4a4a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1d01625c0 0x7fb1d0152810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:49.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.469+0000 7fb1d4a4a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d00ba6d0 con 0x7fb1d00d2b40 2024-04-15T15:19:49.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.469+0000 7fb1d4a4a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d00b8890 con 0x7fb1d01625c0 2024-04-15T15:19:49.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.469+0000 7fb1d4a4a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fb1d0162a50 con 0x7fb1d01620c0 2024-04-15T15:19:49.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.469+0000 7fb1ce59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1d00d2b40 0x7fb1d0156610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:49.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.469+0000 7fb1ce59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1d00d2b40 0x7fb1d0156610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51678/0 (socket says 172.21.15.105:51678) 2024-04-15T15:19:49.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.469+0000 7fb1ce59c640 1 -- 172.21.15.105:0/645274579 learned_addr learned my addr 172.21.15.105:0/645274579 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:49.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.469+0000 7fb1ced9d640 1 --2- 172.21.15.105:0/645274579 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1d01625c0 0x7fb1d0152810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:49.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.469+0000 7fb1cdd9b640 1 --2- 172.21.15.105:0/645274579 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1d01620c0 0x7fb1d0155220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:49.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.469+0000 7fb1ce59c640 1 -- 172.21.15.105:0/645274579 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1d01620c0 msgr2=0x7fb1d0155220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:49.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.469+0000 7fb1ce59c640 1 --2- 172.21.15.105:0/645274579 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1d01620c0 0x7fb1d0155220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:49.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.469+0000 7fb1ce59c640 1 -- 172.21.15.105:0/645274579 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1d01625c0 msgr2=0x7fb1d0152810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:49.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.469+0000 7fb1ce59c640 1 --2- 172.21.15.105:0/645274579 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1d01625c0 0x7fb1d0152810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:49.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.469+0000 7fb1ce59c640 1 -- 172.21.15.105:0/645274579 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1d01570d0 con 0x7fb1d00d2b40 2024-04-15T15:19:49.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.469+0000 7fb1ce59c640 1 --2- 172.21.15.105:0/645274579 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1d00d2b40 0x7fb1d0156610 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb1c000efc0 tx=0x7fb1c000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:49.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.469+0000 7fb1c6ffd640 1 -- 172.21.15.105:0/645274579 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c0018020 con 0x7fb1d00d2b40 2024-04-15T15:19:49.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.469+0000 7fb1c6ffd640 1 -- 172.21.15.105:0/645274579 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1c0014480 con 0x7fb1d00d2b40 2024-04-15T15:19:49.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.469+0000 7fb1d4a4a640 1 -- 172.21.15.105:0/645274579 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1d01573b0 con 0x7fb1d00d2b40 2024-04-15T15:19:49.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.469+0000 7fb1c6ffd640 1 -- 172.21.15.105:0/645274579 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c0014850 con 0x7fb1d00d2b40 2024-04-15T15:19:49.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.469+0000 7fb1d4a4a640 1 -- 172.21.15.105:0/645274579 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1d0157970 con 0x7fb1d00d2b40 2024-04-15T15:19:49.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.470+0000 7fb1d4a4a640 1 -- 172.21.15.105:0/645274579 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1d0162e10 con 0x7fb1d00d2b40 2024-04-15T15:19:49.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.471+0000 7fb1c6ffd640 1 -- 172.21.15.105:0/645274579 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb1c0005d10 con 0x7fb1d00d2b40 2024-04-15T15:19:49.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.471+0000 7fb1c6ffd640 1 --2- 172.21.15.105:0/645274579 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fb1b006c8f0 0x7fb1b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:49.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.471+0000 7fb1c6ffd640 1 -- 172.21.15.105:0/645274579 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb1c008c710 con 0x7fb1d00d2b40 2024-04-15T15:19:49.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.471+0000 7fb1cdd9b640 1 --2- 172.21.15.105:0/645274579 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fb1b006c8f0 0x7fb1b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:49.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.472+0000 7fb1cdd9b640 1 --2- 172.21.15.105:0/645274579 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fb1b006c8f0 0x7fb1b006edb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8036000 tx=0x7fb1b803b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:49.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.476+0000 7fb1c6ffd640 1 -- 172.21.15.105:0/645274579 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb1d0162e10 con 0x7fb1d00d2b40 2024-04-15T15:19:49.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.652+0000 7fb1d4a4a640 1 -- 172.21.15.105:0/645274579 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb1d0068a40 con 0x7fb1d00d2b40 2024-04-15T15:19:49.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.652+0000 7fb1c6ffd640 1 -- 172.21.15.105:0/645274579 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fb1c0022070 con 0x7fb1d00d2b40 2024-04-15T15:19:49.655 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-04-15T15:19:49.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.654+0000 7fb1d4a4a640 1 -- 172.21.15.105:0/645274579 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fb1b006c8f0 msgr2=0x7fb1b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:49.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.654+0000 7fb1d4a4a640 1 --2- 172.21.15.105:0/645274579 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fb1b006c8f0 0x7fb1b006edb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8036000 tx=0x7fb1b803b040 comp rx=0 tx=0).stop 2024-04-15T15:19:49.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.654+0000 7fb1d4a4a640 1 -- 172.21.15.105:0/645274579 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1d00d2b40 msgr2=0x7fb1d0156610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:49.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.654+0000 7fb1d4a4a640 1 --2- 172.21.15.105:0/645274579 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1d00d2b40 0x7fb1d0156610 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb1c000efc0 tx=0x7fb1c000c490 comp rx=0 tx=0).stop 2024-04-15T15:19:49.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.654+0000 7fb1d4a4a640 1 -- 172.21.15.105:0/645274579 shutdown_connections 2024-04-15T15:19:49.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.654+0000 7fb1d4a4a640 1 --2- 172.21.15.105:0/645274579 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1d01620c0 0x7fb1d0155220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:49.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.654+0000 7fb1d4a4a640 1 --2- 172.21.15.105:0/645274579 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fb1b006c8f0 0x7fb1b006edb0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:49.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.654+0000 7fb1d4a4a640 1 --2- 172.21.15.105:0/645274579 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1d01625c0 0x7fb1d0152810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:49.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.654+0000 7fb1d4a4a640 1 --2- 172.21.15.105:0/645274579 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1d00d2b40 0x7fb1d0156610 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:49.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.654+0000 7fb1d4a4a640 1 -- 172.21.15.105:0/645274579 >> 172.21.15.105:0/645274579 conn(0x7fb1d006ccb0 msgr2=0x7fb1d00d4df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:49.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.654+0000 7fb1d4a4a640 1 -- 172.21.15.105:0/645274579 shutdown_connections 2024-04-15T15:19:49.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.654+0000 7fb1d4a4a640 1 -- 172.21.15.105:0/645274579 wait complete. 2024-04-15T15:19:49.668 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-15T15:19:49.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.772+0000 7f4dd359e640 1 Processor -- start 2024-04-15T15:19:49.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.772+0000 7f4dd359e640 1 -- start start 2024-04-15T15:19:49.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.773+0000 7f4dd359e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4dd41602b0 0x7f4dd4071010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:49.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.773+0000 7f4dd359e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4dd40ce950 0x7f4dd4160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:49.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.773+0000 7f4dd359e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4dd40d0260 con 0x7f4dd4160bd0 2024-04-15T15:19:49.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.773+0000 7f4dd359e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f4dd40d0010 con 0x7f4dd41602b0 2024-04-15T15:19:49.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.773+0000 7f4dd359e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f4dd4161080 con 0x7f4dd40ce950 2024-04-15T15:19:49.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.774+0000 7f4dd2d9d640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f4dd4160bd0 0x7f4dd40cd450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.105:34150/0 (socket says 172.21.15.105:34150) 2024-04-15T15:19:49.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.774+0000 7f4dd1d9b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4dd40ce950 0x7f4dd4160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:49.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.774+0000 7f4dd259c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4dd41602b0 0x7f4dd4071010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:49.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.774+0000 7f4dd2d9d640 1 -- 172.21.15.105:0/4159473884 learned_addr learned my addr 172.21.15.105:0/4159473884 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:19:49.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.774+0000 7f4dd159a640 1 -- 172.21.15.105:0/4159473884 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 658470949 0 0) 0x7f4dd40d0260 con 0x7f4dd4160bd0 2024-04-15T15:19:49.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.774+0000 7f4dd159a640 1 -- 172.21.15.105:0/4159473884 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4dc0003520 con 0x7f4dd4160bd0 2024-04-15T15:19:49.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.774+0000 7f4dd1d9b640 1 -- 172.21.15.105:0/4159473884 >> v2:172.21.15.154:3300/0 conn(0x7f4dd41602b0 msgr2=0x7f4dd4071010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:49.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.774+0000 7f4dd1d9b640 1 --2- 172.21.15.105:0/4159473884 >> v2:172.21.15.154:3300/0 conn(0x7f4dd41602b0 0x7f4dd4071010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:49.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.774+0000 7f4dd1d9b640 1 -- 172.21.15.105:0/4159473884 >> v1:172.21.15.154:6789/0 conn(0x7f4dd4160bd0 legacy=0x7f4dd40cd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:49.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.774+0000 7f4dd1d9b640 1 -- 172.21.15.105:0/4159473884 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dd415ff90 con 0x7f4dd40ce950 2024-04-15T15:19:49.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.775+0000 7f4dd1d9b640 1 --2- 172.21.15.105:0/4159473884 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4dd40ce950 0x7f4dd4160690 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc0097a0 tx=0x7f4dbc02b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95c540803b08cbd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:49.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.775+0000 7f4dd159a640 1 -- 172.21.15.105:0/4159473884 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc035020 con 0x7f4dd40ce950 2024-04-15T15:19:49.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.775+0000 7f4dd159a640 1 -- 172.21.15.105:0/4159473884 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dbc031830 con 0x7f4dd40ce950 2024-04-15T15:19:49.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.775+0000 7f4dd159a640 1 -- 172.21.15.105:0/4159473884 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc031c00 con 0x7f4dd40ce950 2024-04-15T15:19:49.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.776+0000 7f4dd359e640 1 -- 172.21.15.105:0/4159473884 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4dd40ce950 msgr2=0x7f4dd4160690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:49.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.776+0000 7f4dd359e640 1 --2- 172.21.15.105:0/4159473884 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4dd40ce950 0x7f4dd4160690 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc0097a0 tx=0x7f4dbc02b8d0 comp rx=0 tx=0).stop 2024-04-15T15:19:49.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.776+0000 7f4dd359e640 1 -- 172.21.15.105:0/4159473884 shutdown_connections 2024-04-15T15:19:49.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.776+0000 7f4dd359e640 1 --2- 172.21.15.105:0/4159473884 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4dd40ce950 0x7f4dd4160690 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:49.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.776+0000 7f4dd359e640 1 --2- 172.21.15.105:0/4159473884 >> v2:172.21.15.154:3300/0 conn(0x7f4dd41602b0 0x7f4dd4071010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:49.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.776+0000 7f4dd359e640 1 -- 172.21.15.105:0/4159473884 >> 172.21.15.105:0/4159473884 conn(0x7f4dd40d2440 msgr2=0x7f4dd40d2850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:49.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.776+0000 7f4dd359e640 1 -- 172.21.15.105:0/4159473884 shutdown_connections 2024-04-15T15:19:49.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.776+0000 7f4dd359e640 1 -- 172.21.15.105:0/4159473884 wait complete. 2024-04-15T15:19:49.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.777+0000 7f4dd359e640 1 Processor -- start 2024-04-15T15:19:49.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.777+0000 7f4dd359e640 1 -- start start 2024-04-15T15:19:49.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.778+0000 7f4dd359e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4dd40ce950 0x7f4dd4154c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:49.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.778+0000 7f4dd359e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4dd41602b0 0x7f4dd409f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:49.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.778+0000 7f4dd359e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4dd4160bd0 0x7f4dd409f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:49.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.778+0000 7f4dd359e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd409aa40 con 0x7f4dd40ce950 2024-04-15T15:19:49.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.778+0000 7f4dd359e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd4072020 con 0x7f4dd4160bd0 2024-04-15T15:19:49.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.778+0000 7f4dd359e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f4dd409abc0 con 0x7f4dd41602b0 2024-04-15T15:19:49.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.778+0000 7f4dd259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4dd40ce950 0x7f4dd4154c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:49.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.778+0000 7f4dd259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4dd40ce950 0x7f4dd4154c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51680/0 (socket says 172.21.15.105:51680) 2024-04-15T15:19:49.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.778+0000 7f4dd259c640 1 -- 172.21.15.105:0/2179331473 learned_addr learned my addr 172.21.15.105:0/2179331473 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:49.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.778+0000 7f4dd1d9b640 1 --2- 172.21.15.105:0/2179331473 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4dd41602b0 0x7f4dd409f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:49.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.778+0000 7f4dd2d9d640 1 --2- 172.21.15.105:0/2179331473 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4dd4160bd0 0x7f4dd409f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:49.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.778+0000 7f4dd259c640 1 -- 172.21.15.105:0/2179331473 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4dd41602b0 msgr2=0x7f4dd409f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:49.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.778+0000 7f4dd259c640 1 --2- 172.21.15.105:0/2179331473 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4dd41602b0 0x7f4dd409f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:49.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.778+0000 7f4dd259c640 1 -- 172.21.15.105:0/2179331473 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4dd4160bd0 msgr2=0x7f4dd409f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:49.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.778+0000 7f4dd259c640 1 --2- 172.21.15.105:0/2179331473 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4dd4160bd0 0x7f4dd409f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:49.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.778+0000 7f4dd259c640 1 -- 172.21.15.105:0/2179331473 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dd409d560 con 0x7f4dd40ce950 2024-04-15T15:19:49.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.778+0000 7f4dd2d9d640 1 --2- 172.21.15.105:0/2179331473 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4dd4160bd0 0x7f4dd409f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:19:49.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.779+0000 7f4dd259c640 1 --2- 172.21.15.105:0/2179331473 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4dd40ce950 0x7f4dd4154c20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4dc4009db0 tx=0x7f4dc400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:49.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.779+0000 7f4dcaffd640 1 -- 172.21.15.105:0/2179331473 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc4018020 con 0x7f4dd40ce950 2024-04-15T15:19:49.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.779+0000 7f4dcaffd640 1 -- 172.21.15.105:0/2179331473 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dc4014480 con 0x7f4dd40ce950 2024-04-15T15:19:49.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.779+0000 7f4dd359e640 1 -- 172.21.15.105:0/2179331473 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dd409d840 con 0x7f4dd40ce950 2024-04-15T15:19:49.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.779+0000 7f4dcaffd640 1 -- 172.21.15.105:0/2179331473 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc4014850 con 0x7f4dd40ce950 2024-04-15T15:19:49.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.779+0000 7f4dd359e640 1 -- 172.21.15.105:0/2179331473 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dd409de00 con 0x7f4dd40ce950 2024-04-15T15:19:49.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.780+0000 7f4dd359e640 1 -- 172.21.15.105:0/2179331473 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dd404fdc0 con 0x7f4dd40ce950 2024-04-15T15:19:49.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.780+0000 7f4dcaffd640 1 -- 172.21.15.105:0/2179331473 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4dc4005d10 con 0x7f4dd40ce950 2024-04-15T15:19:49.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.780+0000 7f4dcaffd640 1 --2- 172.21.15.105:0/2179331473 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f4db406c8f0 0x7f4db406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:49.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.781+0000 7f4dcaffd640 1 -- 172.21.15.105:0/2179331473 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4dc408c650 con 0x7f4dd40ce950 2024-04-15T15:19:49.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.781+0000 7f4dd1d9b640 1 --2- 172.21.15.105:0/2179331473 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f4db406c8f0 0x7f4db406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:49.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.781+0000 7f4dd1d9b640 1 --2- 172.21.15.105:0/2179331473 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f4db406c8f0 0x7f4db406edb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc036000 tx=0x7f4dbc03d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:49.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.785+0000 7f4dcaffd640 1 -- 172.21.15.105:0/2179331473 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4dc4022070 con 0x7f4dd40ce950 2024-04-15T15:19:49.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.959+0000 7f4dd359e640 1 -- 172.21.15.105:0/2179331473 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4dd40c6010 con 0x7f4dd40ce950 2024-04-15T15:19:49.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.959+0000 7f4dcaffd640 1 -- 172.21.15.105:0/2179331473 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f4dc4010040 con 0x7f4dd40ce950 2024-04-15T15:19:49.962 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-04-15T15:19:49.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.961+0000 7f4dd359e640 1 -- 172.21.15.105:0/2179331473 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f4db406c8f0 msgr2=0x7f4db406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:49.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.961+0000 7f4dd359e640 1 --2- 172.21.15.105:0/2179331473 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f4db406c8f0 0x7f4db406edb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc036000 tx=0x7f4dbc03d040 comp rx=0 tx=0).stop 2024-04-15T15:19:49.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.961+0000 7f4dd359e640 1 -- 172.21.15.105:0/2179331473 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4dd40ce950 msgr2=0x7f4dd4154c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:49.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.961+0000 7f4dd359e640 1 --2- 172.21.15.105:0/2179331473 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4dd40ce950 0x7f4dd4154c20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4dc4009db0 tx=0x7f4dc400c490 comp rx=0 tx=0).stop 2024-04-15T15:19:49.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.961+0000 7f4dd359e640 1 -- 172.21.15.105:0/2179331473 shutdown_connections 2024-04-15T15:19:49.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.961+0000 7f4dd359e640 1 --2- 172.21.15.105:0/2179331473 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4dd41602b0 0x7f4dd409f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:49.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.961+0000 7f4dd359e640 1 --2- 172.21.15.105:0/2179331473 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f4db406c8f0 0x7f4db406edb0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:49.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.961+0000 7f4dd359e640 1 --2- 172.21.15.105:0/2179331473 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4dd4160bd0 0x7f4dd409f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:49.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.961+0000 7f4dd359e640 1 --2- 172.21.15.105:0/2179331473 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4dd40ce950 0x7f4dd4154c20 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:49.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.961+0000 7f4dd359e640 1 -- 172.21.15.105:0/2179331473 >> 172.21.15.105:0/2179331473 conn(0x7f4dd40d2440 msgr2=0x7f4dd40cd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:49.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.961+0000 7f4dd359e640 1 -- 172.21.15.105:0/2179331473 shutdown_connections 2024-04-15T15:19:49.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:49.961+0000 7f4dd359e640 1 -- 172.21.15.105:0/2179331473 wait complete. 2024-04-15T15:19:49.976 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:19:50.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.076+0000 7f04514c5640 1 Processor -- start 2024-04-15T15:19:50.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.076+0000 7f04514c5640 1 -- start start 2024-04-15T15:19:50.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.077+0000 7f04514c5640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f044c06c0d0 0x7f044c0b17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:50.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.077+0000 7f04514c5640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f044c0b2ca0 con 0x7f044c15e9e0 2024-04-15T15:19:50.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.077+0000 7f04514c5640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f044c0b6130 con 0x7f044c0ae410 2024-04-15T15:19:50.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.077+0000 7f04514c5640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f044c0b5f00 con 0x7f044c06c0d0 2024-04-15T15:19:50.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.077+0000 7f044affd640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f044c15e9e0 0x7f044c0b4430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:46400/0 (socket says 172.21.15.105:46400) 2024-04-15T15:19:50.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.077+0000 7f044affd640 1 -- 172.21.15.105:0/833807212 learned_addr learned my addr 172.21.15.105:0/833807212 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:19:50.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.078+0000 7f044b7fe640 1 --2- 172.21.15.105:0/833807212 >> v2:172.21.15.154:3300/0 conn(0x7f044c06c0d0 0x7f044c0b17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:50.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.078+0000 7f0449ffb640 1 -- 172.21.15.105:0/833807212 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2935509209 0 0) 0x7f044c0b2ca0 con 0x7f044c15e9e0 2024-04-15T15:19:50.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.078+0000 7f0449ffb640 1 -- 172.21.15.105:0/833807212 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0438003520 con 0x7f044c15e9e0 2024-04-15T15:19:50.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.078+0000 7f0449ffb640 1 -- 172.21.15.105:0/833807212 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2341734762 0 0) 0x7f0438003520 con 0x7f044c15e9e0 2024-04-15T15:19:50.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.078+0000 7f0449ffb640 1 -- 172.21.15.105:0/833807212 >> v2:172.21.15.154:3300/0 conn(0x7f044c06c0d0 msgr2=0x7f044c0b17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:50.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.078+0000 7f0449ffb640 1 --2- 172.21.15.105:0/833807212 >> v2:172.21.15.154:3300/0 conn(0x7f044c06c0d0 0x7f044c0b17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:50.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.078+0000 7f0449ffb640 1 -- 172.21.15.105:0/833807212 >> v1:172.21.15.154:6789/0 conn(0x7f044c0ae410 legacy=0x7f044c06b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:50.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.078+0000 7f0449ffb640 1 -- 172.21.15.105:0/833807212 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f044c0b6410 con 0x7f044c15e9e0 2024-04-15T15:19:50.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.078+0000 7f0449ffb640 1 -- 172.21.15.105:0/833807212 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7f0444001a10 con 0x7f044c15e9e0 2024-04-15T15:19:50.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.078+0000 7f0449ffb640 1 -- 172.21.15.105:0/833807212 >> v1:172.21.15.105:6789/0 conn(0x7f044c15e9e0 legacy=0x7f044c0b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:50.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.078+0000 7f0449ffb640 1 --2- 172.21.15.105:0/833807212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04380039b0 0x7f0438003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:50.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.078+0000 7f0449ffb640 1 --2- 172.21.15.105:0/833807212 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f04380043b0 0x7f0438004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:50.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.078+0000 7f0449ffb640 1 --2- 172.21.15.105:0/833807212 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0438005080 0x7f043800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:50.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.078+0000 7f0449ffb640 1 -- 172.21.15.105:0/833807212 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f043800bfb0 con 0x7f04380039b0 2024-04-15T15:19:50.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.078+0000 7f0449ffb640 1 -- 172.21.15.105:0/833807212 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f043800c160 con 0x7f0438005080 2024-04-15T15:19:50.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.078+0000 7f0449ffb640 1 -- 172.21.15.105:0/833807212 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f043800c360 con 0x7f04380043b0 2024-04-15T15:19:50.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.078+0000 7f044affd640 1 --2- 172.21.15.105:0/833807212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04380039b0 0x7f0438003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:50.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.078+0000 7f044b7fe640 1 --2- 172.21.15.105:0/833807212 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0438005080 0x7f043800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:50.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.078+0000 7f044a7fc640 1 --2- 172.21.15.105:0/833807212 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f04380043b0 0x7f0438004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:50.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.078+0000 7f044affd640 1 -- 172.21.15.105:0/833807212 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f04380043b0 msgr2=0x7f0438004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:50.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.078+0000 7f044affd640 1 --2- 172.21.15.105:0/833807212 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f04380043b0 0x7f0438004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:50.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.078+0000 7f044affd640 1 -- 172.21.15.105:0/833807212 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0438005080 msgr2=0x7f043800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:50.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.078+0000 7f044affd640 1 --2- 172.21.15.105:0/833807212 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0438005080 0x7f043800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:50.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.078+0000 7f044affd640 1 -- 172.21.15.105:0/833807212 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0438003520 con 0x7f04380039b0 2024-04-15T15:19:50.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.078+0000 7f044b7fe640 1 --2- 172.21.15.105:0/833807212 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0438005080 0x7f043800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:19:50.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.078+0000 7f044b7fe640 1 -- 172.21.15.105:0/833807212 reap_dead start 2024-04-15T15:19:50.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.079+0000 7f044affd640 1 --2- 172.21.15.105:0/833807212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04380039b0 0x7f0438003d90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f044400a880 tx=0x7f0438005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c03891bb82f0685c server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:50.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.079+0000 7f0449ffb640 1 -- 172.21.15.105:0/833807212 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044400a4d0 con 0x7f04380039b0 2024-04-15T15:19:50.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.079+0000 7f0449ffb640 1 -- 172.21.15.105:0/833807212 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04440072d0 con 0x7f04380039b0 2024-04-15T15:19:50.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.079+0000 7f0449ffb640 1 -- 172.21.15.105:0/833807212 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04440076c0 con 0x7f04380039b0 2024-04-15T15:19:50.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.080+0000 7f04514c5640 1 -- 172.21.15.105:0/833807212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04380039b0 msgr2=0x7f0438003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:50.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.080+0000 7f04514c5640 1 --2- 172.21.15.105:0/833807212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04380039b0 0x7f0438003d90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f044400a880 tx=0x7f0438005830 comp rx=0 tx=0).stop 2024-04-15T15:19:50.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.080+0000 7f04514c5640 1 -- 172.21.15.105:0/833807212 shutdown_connections 2024-04-15T15:19:50.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.080+0000 7f04514c5640 1 --2- 172.21.15.105:0/833807212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04380039b0 0x7f0438003d90 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:50.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.080+0000 7f04514c5640 1 -- 172.21.15.105:0/833807212 >> 172.21.15.105:0/833807212 conn(0x7f044c09aa40 msgr2=0x7f044c0b6950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:50.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.080+0000 7f04514c5640 1 -- 172.21.15.105:0/833807212 shutdown_connections 2024-04-15T15:19:50.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.080+0000 7f04514c5640 1 -- 172.21.15.105:0/833807212 wait complete. 2024-04-15T15:19:50.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.081+0000 7f04514c5640 1 Processor -- start 2024-04-15T15:19:50.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.081+0000 7f04514c5640 1 -- start start 2024-04-15T15:19:50.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.082+0000 7f04514c5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f04380039b0 0x7f044c1618e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:50.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.082+0000 7f04514c5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f044c0b1660 0x7f044c0b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:50.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.082+0000 7f04514c5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f044c147c80 0x7f044c148100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:50.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.082+0000 7f04514c5640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f044c15edc0 con 0x7f044c147c80 2024-04-15T15:19:50.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.082+0000 7f04514c5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f044c0b2f80 con 0x7f044c0b1660 2024-04-15T15:19:50.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.082+0000 7f04514c5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f044c0b72a0 con 0x7f04380039b0 2024-04-15T15:19:50.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.082+0000 7f044b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f044c147c80 0x7f044c148100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:50.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.082+0000 7f044b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f044c147c80 0x7f044c148100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51700/0 (socket says 172.21.15.105:51700) 2024-04-15T15:19:50.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.082+0000 7f044b7fe640 1 -- 172.21.15.105:0/1680545803 learned_addr learned my addr 172.21.15.105:0/1680545803 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:50.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.082+0000 7f044a7fc640 1 --2- 172.21.15.105:0/1680545803 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f044c0b1660 0x7f044c0b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:50.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.082+0000 7f044affd640 1 --2- 172.21.15.105:0/1680545803 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f04380039b0 0x7f044c1618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:50.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.082+0000 7f044b7fe640 1 -- 172.21.15.105:0/1680545803 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f04380039b0 msgr2=0x7f044c1618e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:50.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.082+0000 7f044b7fe640 1 --2- 172.21.15.105:0/1680545803 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f04380039b0 0x7f044c1618e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:50.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.082+0000 7f044b7fe640 1 -- 172.21.15.105:0/1680545803 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f044c0b1660 msgr2=0x7f044c0b1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:50.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.082+0000 7f044b7fe640 1 --2- 172.21.15.105:0/1680545803 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f044c0b1660 0x7f044c0b1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:50.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.082+0000 7f044b7fe640 1 -- 172.21.15.105:0/1680545803 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f044c161e20 con 0x7f044c147c80 2024-04-15T15:19:50.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.082+0000 7f044affd640 1 --2- 172.21.15.105:0/1680545803 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f04380039b0 0x7f044c1618e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:19:50.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.082+0000 7f044a7fc640 1 --2- 172.21.15.105:0/1680545803 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f044c0b1660 0x7f044c0b1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:19:50.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.082+0000 7f044b7fe640 1 --2- 172.21.15.105:0/1680545803 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f044c147c80 0x7f044c148100 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f044c072eb0 tx=0x7f043c011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:50.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.083+0000 7f04437fe640 1 -- 172.21.15.105:0/1680545803 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f043c019020 con 0x7f044c147c80 2024-04-15T15:19:50.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.083+0000 7f04437fe640 1 -- 172.21.15.105:0/1680545803 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f043c004460 con 0x7f044c147c80 2024-04-15T15:19:50.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.083+0000 7f04437fe640 1 -- 172.21.15.105:0/1680545803 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f043c003850 con 0x7f044c147c80 2024-04-15T15:19:50.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.083+0000 7f04514c5640 1 -- 172.21.15.105:0/1680545803 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f044c15d150 con 0x7f044c147c80 2024-04-15T15:19:50.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.083+0000 7f04514c5640 1 -- 172.21.15.105:0/1680545803 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f044c15d710 con 0x7f044c147c80 2024-04-15T15:19:50.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.084+0000 7f04514c5640 1 -- 172.21.15.105:0/1680545803 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f044c069150 con 0x7f044c147c80 2024-04-15T15:19:50.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.084+0000 7f04437fe640 1 -- 172.21.15.105:0/1680545803 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f043c01c020 con 0x7f044c147c80 2024-04-15T15:19:50.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.084+0000 7f04437fe640 1 --2- 172.21.15.105:0/1680545803 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f042c06c920 0x7f042c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:50.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.085+0000 7f044affd640 1 --2- 172.21.15.105:0/1680545803 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f042c06c920 0x7f042c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:50.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.085+0000 7f04437fe640 1 -- 172.21.15.105:0/1680545803 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f043c08bc90 con 0x7f044c147c80 2024-04-15T15:19:50.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.085+0000 7f044affd640 1 --2- 172.21.15.105:0/1680545803 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f042c06c920 0x7f042c06ede0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0444034000 tx=0x7f0444007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:50.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.089+0000 7f04437fe640 1 -- 172.21.15.105:0/1680545803 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f043c023030 con 0x7f044c147c80 2024-04-15T15:19:50.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.264+0000 7f04514c5640 1 -- 172.21.15.105:0/1680545803 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f044c06bca0 con 0x7f044c147c80 2024-04-15T15:19:50.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.265+0000 7f04437fe640 1 -- 172.21.15.105:0/1680545803 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16653 (secure 0 0 0) 0x7f043c05a150 con 0x7f044c147c80 2024-04-15T15:19:50.268 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:19:50.268 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":20,"fsid":"09e4d2a4-a324-4793-af77-644b25e520e1","created":"2024-04-15T15:19:01.524543+0000","modified":"2024-04-15T15:19:27.414890+0000","last_up_change":"2024-04-15T15:19:12.825815+0000","last_in_change":"2024-04-15T15:19:03.788126+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:19:13.748585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:19:23.789287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:19:24.722250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6f6292b-2d7e-4aa0-9d1c-6ddd0f97287c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6801","nonce":2873040054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6803","nonce":2873040054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6807","nonce":2873040054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6805","nonce":2873040054}]},"public_addr":"172.21.15.105:6801/2873040054","cluster_addr":"172.21.15.105:6803/2873040054","heartbeat_back_addr":"172.21.15.105:6807/2873040054","heartbeat_front_addr":"172.21.15.105:6805/2873040054","state":["exists","up"]},{"osd":1,"uuid":"8900966a-b393-41e3-8f93-e5a43bd23622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6818","nonce":2494994479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6821","nonce":2494994479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6828","nonce":2494994479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6823","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6825","nonce":2494994479}]},"public_addr":"172.21.15.105:6818/2494994479","cluster_addr":"172.21.15.105:6821/2494994479","heartbeat_back_addr":"172.21.15.105:6828/2494994479","heartbeat_front_addr":"172.21.15.105:6825/2494994479","state":["exists","up"]},{"osd":2,"uuid":"81a5dc7a-9325-4ff5-941d-a2d5fba21f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6809","nonce":3798408699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6811","nonce":3798408699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6815","nonce":3798408699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6813","nonce":3798408699}]},"public_addr":"172.21.15.105:6809/3798408699","cluster_addr":"172.21.15.105:6811/3798408699","heartbeat_back_addr":"172.21.15.105:6815/3798408699","heartbeat_front_addr":"172.21.15.105:6813/3798408699","state":["exists","up"]},{"osd":3,"uuid":"df766ed2-5c71-4805-96f4-3d4970edc960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6819","nonce":1920720186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6824","nonce":1920720186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6831","nonce":1920720186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6827","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6829","nonce":1920720186}]},"public_addr":"172.21.15.105:6819/1920720186","cluster_addr":"172.21.15.105:6824/1920720186","heartbeat_back_addr":"172.21.15.105:6831/1920720186","heartbeat_front_addr":"172.21.15.105:6829/1920720186","state":["exists","up"]},{"osd":4,"uuid":"e1039e03-28b8-49df-b951-c91930b93c9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6810","nonce":3516494456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6813","nonce":3516494456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6820","nonce":3516494456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6817","nonce":3516494456}]},"public_addr":"172.21.15.154:6810/3516494456","cluster_addr":"172.21.15.154:6813/3516494456","heartbeat_back_addr":"172.21.15.154:6820/3516494456","heartbeat_front_addr":"172.21.15.154:6817/3516494456","state":["exists","up"]},{"osd":5,"uuid":"a9f5eaf1-66ff-405a-a9e4-ec3340812a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6801","nonce":540367751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6803","nonce":540367751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6807","nonce":540367751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6805","nonce":540367751}]},"public_addr":"172.21.15.154:6801/540367751","cluster_addr":"172.21.15.154:6803/540367751","heartbeat_back_addr":"172.21.15.154:6807/540367751","heartbeat_front_addr":"172.21.15.154:6805/540367751","state":["exists","up"]},{"osd":6,"uuid":"267d02ea-675f-428f-aae8-e0375e4d4feb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6825","nonce":1550829005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6827","nonce":1550829005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6831","nonce":1550829005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6829","nonce":1550829005}]},"public_addr":"172.21.15.154:6825/1550829005","cluster_addr":"172.21.15.154:6827/1550829005","heartbeat_back_addr":"172.21.15.154:6831/1550829005","heartbeat_front_addr":"172.21.15.154:6829/1550829005","state":["exists","up"]},{"osd":7,"uuid":"759ac6cc-896a-423c-885f-9fe90d8dcb14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6809","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6811","nonce":3920089903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6816","nonce":3920089903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6823","nonce":3920089903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6819","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6821","nonce":3920089903}]},"public_addr":"172.21.15.154:6811/3920089903","cluster_addr":"172.21.15.154:6816/3920089903","heartbeat_back_addr":"172.21.15.154:6823/3920089903","heartbeat_front_addr":"172.21.15.154:6821/3920089903","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.949509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.984072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.907278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.947391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.975447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.998896+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:09.028739+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.894119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:19:50.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.267+0000 7f04514c5640 1 -- 172.21.15.105:0/1680545803 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f042c06c920 msgr2=0x7f042c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:50.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.267+0000 7f04514c5640 1 --2- 172.21.15.105:0/1680545803 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f042c06c920 0x7f042c06ede0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0444034000 tx=0x7f0444007000 comp rx=0 tx=0).stop 2024-04-15T15:19:50.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.267+0000 7f04514c5640 1 -- 172.21.15.105:0/1680545803 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f044c147c80 msgr2=0x7f044c148100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:50.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.267+0000 7f04514c5640 1 --2- 172.21.15.105:0/1680545803 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f044c147c80 0x7f044c148100 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f044c072eb0 tx=0x7f043c011920 comp rx=0 tx=0).stop 2024-04-15T15:19:50.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.267+0000 7f04514c5640 1 -- 172.21.15.105:0/1680545803 shutdown_connections 2024-04-15T15:19:50.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.267+0000 7f04514c5640 1 --2- 172.21.15.105:0/1680545803 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f042c06c920 0x7f042c06ede0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:50.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.267+0000 7f04514c5640 1 --2- 172.21.15.105:0/1680545803 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f044c147c80 0x7f044c148100 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:50.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.267+0000 7f04514c5640 1 --2- 172.21.15.105:0/1680545803 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f044c0b1660 0x7f044c0b1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:50.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.267+0000 7f04514c5640 1 --2- 172.21.15.105:0/1680545803 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f04380039b0 0x7f044c1618e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:50.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.267+0000 7f04514c5640 1 -- 172.21.15.105:0/1680545803 >> 172.21.15.105:0/1680545803 conn(0x7f044c09aa40 msgr2=0x7f044c15a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:50.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.267+0000 7f04514c5640 1 -- 172.21.15.105:0/1680545803 shutdown_connections 2024-04-15T15:19:50.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.267+0000 7f04514c5640 1 -- 172.21.15.105:0/1680545803 wait complete. 2024-04-15T15:19:50.281 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:19:13.748585+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-15T15:19:23.789287+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-15T15:19:24.722250+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:19:50.281 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:19:50.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.384+0000 7fd677d75640 1 Processor -- start 2024-04-15T15:19:50.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.384+0000 7fd677d75640 1 -- start start 2024-04-15T15:19:50.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.385+0000 7fd677d75640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd670144c90 0x7fd6700b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:50.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.385+0000 7fd677d75640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd67006ef90 0x7fd67006f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:50.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.385+0000 7fd677d75640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd6700cd1d0 0x7fd6700cd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:50.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.385+0000 7fd677d75640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fd6700b3e50 con 0x7fd6700cd1d0 2024-04-15T15:19:50.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.385+0000 7fd677d75640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fd67006f910 con 0x7fd670144c90 2024-04-15T15:19:50.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.385+0000 7fd677d75640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fd6700b40a0 con 0x7fd67006ef90 2024-04-15T15:19:50.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.385+0000 7fd676312640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd6700cd1d0 0x7fd6700cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:50.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.385+0000 7fd676312640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd6700cd1d0 0x7fd6700cd600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51714/0 (socket says 172.21.15.105:51714) 2024-04-15T15:19:50.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.385+0000 7fd676312640 1 -- 172.21.15.105:0/1341501739 learned_addr learned my addr 172.21.15.105:0/1341501739 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:50.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.385+0000 7fd675b11640 1 --2- 172.21.15.105:0/1341501739 >> v2:172.21.15.154:3300/0 conn(0x7fd670144c90 0x7fd6700b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:50.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.385+0000 7fd675310640 1 --2- 172.21.15.105:0/1341501739 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd67006ef90 0x7fd67006f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:50.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.385+0000 7fd676312640 1 -- 172.21.15.105:0/1341501739 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd67006ef90 msgr2=0x7fd67006f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:50.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.385+0000 7fd676312640 1 --2- 172.21.15.105:0/1341501739 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd67006ef90 0x7fd67006f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:50.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.385+0000 7fd676312640 1 -- 172.21.15.105:0/1341501739 >> v2:172.21.15.154:3300/0 conn(0x7fd670144c90 msgr2=0x7fd6700b4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:50.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.385+0000 7fd676312640 1 --2- 172.21.15.105:0/1341501739 >> v2:172.21.15.154:3300/0 conn(0x7fd670144c90 0x7fd6700b4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:50.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.385+0000 7fd676312640 1 -- 172.21.15.105:0/1341501739 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6700b4380 con 0x7fd6700cd1d0 2024-04-15T15:19:50.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.385+0000 7fd675b11640 1 --2- 172.21.15.105:0/1341501739 >> v2:172.21.15.154:3300/0 conn(0x7fd670144c90 0x7fd6700b4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:19:50.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.386+0000 7fd676312640 1 --2- 172.21.15.105:0/1341501739 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6700cd1d0 0x7fd6700cd600 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd67006c8b0 tx=0x7fd66802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cee0c73cd30ad83 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:50.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.386+0000 7fd674b0f640 1 -- 172.21.15.105:0/1341501739 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66802e070 con 0x7fd6700cd1d0 2024-04-15T15:19:50.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.386+0000 7fd674b0f640 1 -- 172.21.15.105:0/1341501739 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd668037070 con 0x7fd6700cd1d0 2024-04-15T15:19:50.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.386+0000 7fd674b0f640 1 -- 172.21.15.105:0/1341501739 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd668031a00 con 0x7fd6700cd1d0 2024-04-15T15:19:50.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.387+0000 7fd677d75640 1 -- 172.21.15.105:0/1341501739 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6700cd1d0 msgr2=0x7fd6700cd600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:50.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.387+0000 7fd677d75640 1 --2- 172.21.15.105:0/1341501739 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6700cd1d0 0x7fd6700cd600 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd67006c8b0 tx=0x7fd66802b880 comp rx=0 tx=0).stop 2024-04-15T15:19:50.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.387+0000 7fd677d75640 1 -- 172.21.15.105:0/1341501739 shutdown_connections 2024-04-15T15:19:50.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.387+0000 7fd677d75640 1 --2- 172.21.15.105:0/1341501739 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6700cd1d0 0x7fd6700cd600 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:50.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.387+0000 7fd677d75640 1 --2- 172.21.15.105:0/1341501739 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd67006ef90 0x7fd67006f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:50.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.387+0000 7fd677d75640 1 --2- 172.21.15.105:0/1341501739 >> v2:172.21.15.154:3300/0 conn(0x7fd670144c90 0x7fd6700b4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:50.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.387+0000 7fd677d75640 1 -- 172.21.15.105:0/1341501739 >> 172.21.15.105:0/1341501739 conn(0x7fd670067d50 msgr2=0x7fd67006b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:50.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.387+0000 7fd677d75640 1 -- 172.21.15.105:0/1341501739 shutdown_connections 2024-04-15T15:19:50.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.387+0000 7fd677d75640 1 -- 172.21.15.105:0/1341501739 wait complete. 2024-04-15T15:19:50.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.388+0000 7fd677d75640 1 Processor -- start 2024-04-15T15:19:50.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.388+0000 7fd677d75640 1 -- start start 2024-04-15T15:19:50.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.389+0000 7fd677d75640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd67006ef90 0x7fd670139c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:50.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.389+0000 7fd677d75640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6700cd1d0 0x7fd670140fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:50.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.389+0000 7fd677d75640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd670144c90 0x7fd670137fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:50.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.389+0000 7fd677d75640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd6700cdb40 con 0x7fd670144c90 2024-04-15T15:19:50.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.389+0000 7fd677d75640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd6700b4940 con 0x7fd6700cd1d0 2024-04-15T15:19:50.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.389+0000 7fd677d75640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fd6700cdcc0 con 0x7fd67006ef90 2024-04-15T15:19:50.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.389+0000 7fd676312640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd670144c90 0x7fd670137fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:50.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.389+0000 7fd676312640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd670144c90 0x7fd670137fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51728/0 (socket says 172.21.15.105:51728) 2024-04-15T15:19:50.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.389+0000 7fd676312640 1 -- 172.21.15.105:0/2027795293 learned_addr learned my addr 172.21.15.105:0/2027795293 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:50.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.389+0000 7fd675b11640 1 --2- 172.21.15.105:0/2027795293 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd67006ef90 0x7fd670139c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:50.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.389+0000 7fd675310640 1 --2- 172.21.15.105:0/2027795293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6700cd1d0 0x7fd670140fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:50.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.389+0000 7fd676312640 1 -- 172.21.15.105:0/2027795293 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd67006ef90 msgr2=0x7fd670139c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:50.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.389+0000 7fd676312640 1 --2- 172.21.15.105:0/2027795293 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd67006ef90 0x7fd670139c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:50.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.389+0000 7fd676312640 1 -- 172.21.15.105:0/2027795293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6700cd1d0 msgr2=0x7fd670140fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:50.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.389+0000 7fd676312640 1 --2- 172.21.15.105:0/2027795293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6700cd1d0 0x7fd670140fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:50.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.389+0000 7fd676312640 1 -- 172.21.15.105:0/2027795293 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd670138be0 con 0x7fd670144c90 2024-04-15T15:19:50.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.389+0000 7fd675310640 1 --2- 172.21.15.105:0/2027795293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6700cd1d0 0x7fd670140fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:19:50.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.389+0000 7fd676312640 1 --2- 172.21.15.105:0/2027795293 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd670144c90 0x7fd670137fd0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd6680090b0 tx=0x7fd6680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:50.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.389+0000 7fd6667fc640 1 -- 172.21.15.105:0/2027795293 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd668032670 con 0x7fd670144c90 2024-04-15T15:19:50.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.389+0000 7fd6667fc640 1 -- 172.21.15.105:0/2027795293 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd668032830 con 0x7fd670144c90 2024-04-15T15:19:50.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.389+0000 7fd677d75640 1 -- 172.21.15.105:0/2027795293 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd670138ec0 con 0x7fd670144c90 2024-04-15T15:19:50.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.389+0000 7fd6667fc640 1 -- 172.21.15.105:0/2027795293 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd668032c20 con 0x7fd670144c90 2024-04-15T15:19:50.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.389+0000 7fd677d75640 1 -- 172.21.15.105:0/2027795293 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd670139480 con 0x7fd670144c90 2024-04-15T15:19:50.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.391+0000 7fd677d75640 1 -- 172.21.15.105:0/2027795293 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd670068ab0 con 0x7fd670144c90 2024-04-15T15:19:50.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.391+0000 7fd6667fc640 1 -- 172.21.15.105:0/2027795293 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd6680453f0 con 0x7fd670144c90 2024-04-15T15:19:50.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.391+0000 7fd6667fc640 1 --2- 172.21.15.105:0/2027795293 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fd65406c8f0 0x7fd65406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:50.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.391+0000 7fd6667fc640 1 -- 172.21.15.105:0/2027795293 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd6680ad9f0 con 0x7fd670144c90 2024-04-15T15:19:50.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.391+0000 7fd675b11640 1 --2- 172.21.15.105:0/2027795293 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fd65406c8f0 0x7fd65406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:50.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.392+0000 7fd675b11640 1 --2- 172.21.15.105:0/2027795293 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fd65406c8f0 0x7fd65406edb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd66c009fd0 tx=0x7fd66c009870 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:50.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.396+0000 7fd6667fc640 1 -- 172.21.15.105:0/2027795293 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd668040070 con 0x7fd670144c90 2024-04-15T15:19:50.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.577+0000 7fd677d75640 1 -- 172.21.15.105:0/2027795293 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd670144070 con 0x7fd670144c90 2024-04-15T15:19:50.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.577+0000 7fd6667fc640 1 -- 172.21.15.105:0/2027795293 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fd66807be20 con 0x7fd670144c90 2024-04-15T15:19:50.581 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-04-15T15:19:50.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.579+0000 7fd677d75640 1 -- 172.21.15.105:0/2027795293 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fd65406c8f0 msgr2=0x7fd65406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:50.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.579+0000 7fd677d75640 1 --2- 172.21.15.105:0/2027795293 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fd65406c8f0 0x7fd65406edb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd66c009fd0 tx=0x7fd66c009870 comp rx=0 tx=0).stop 2024-04-15T15:19:50.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.579+0000 7fd677d75640 1 -- 172.21.15.105:0/2027795293 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd670144c90 msgr2=0x7fd670137fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:50.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.579+0000 7fd677d75640 1 --2- 172.21.15.105:0/2027795293 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd670144c90 0x7fd670137fd0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd6680090b0 tx=0x7fd6680046c0 comp rx=0 tx=0).stop 2024-04-15T15:19:50.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.579+0000 7fd677d75640 1 -- 172.21.15.105:0/2027795293 shutdown_connections 2024-04-15T15:19:50.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.579+0000 7fd677d75640 1 --2- 172.21.15.105:0/2027795293 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fd65406c8f0 0x7fd65406edb0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:50.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.579+0000 7fd677d75640 1 --2- 172.21.15.105:0/2027795293 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd670144c90 0x7fd670137fd0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:50.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.579+0000 7fd677d75640 1 --2- 172.21.15.105:0/2027795293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6700cd1d0 0x7fd670140fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:50.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.579+0000 7fd677d75640 1 --2- 172.21.15.105:0/2027795293 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd67006ef90 0x7fd670139c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:50.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.579+0000 7fd677d75640 1 -- 172.21.15.105:0/2027795293 >> 172.21.15.105:0/2027795293 conn(0x7fd670067d50 msgr2=0x7fd6700b21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:50.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.580+0000 7fd677d75640 1 -- 172.21.15.105:0/2027795293 shutdown_connections 2024-04-15T15:19:50.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.580+0000 7fd677d75640 1 -- 172.21.15.105:0/2027795293 wait complete. 2024-04-15T15:19:50.593 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-15T15:19:50.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.696+0000 7ffa03270640 1 Processor -- start 2024-04-15T15:19:50.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.696+0000 7ffa03270640 1 -- start start 2024-04-15T15:19:50.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.697+0000 7ffa03270640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff9fc0d07c0 0x7ff9fc06b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:50.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.697+0000 7ffa03270640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff9fc0cd5a0 0x7ff9fc0d0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:50.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.697+0000 7ffa03270640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9fc0cf150 con 0x7ff9fc0ce520 2024-04-15T15:19:50.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.697+0000 7ffa03270640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7ff9fc0d10e0 con 0x7ff9fc0cd5a0 2024-04-15T15:19:50.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.697+0000 7ffa03270640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7ff9fc06c030 con 0x7ff9fc0d07c0 2024-04-15T15:19:50.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.697+0000 7ffa0080b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff9fc0cd5a0 0x7ff9fc0d0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:50.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.697+0000 7ffa0080b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff9fc0cd5a0 0x7ff9fc0d0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46722/0 (socket says 172.21.15.105:46722) 2024-04-15T15:19:50.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.697+0000 7ffa0080b640 1 -- 172.21.15.105:0/1251862104 learned_addr learned my addr 172.21.15.105:0/1251862104 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:50.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.697+0000 7ffa0100c640 1 --2- 172.21.15.105:0/1251862104 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff9fc0d07c0 0x7ff9fc06b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:50.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.697+0000 7ffa0080b640 1 -- 172.21.15.105:0/1251862104 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff9fc0d07c0 msgr2=0x7ff9fc06b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:50.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.697+0000 7ffa0080b640 1 --2- 172.21.15.105:0/1251862104 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff9fc0d07c0 0x7ff9fc06b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:50.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.697+0000 7ffa0080b640 1 -- 172.21.15.105:0/1251862104 >> v1:172.21.15.154:6789/0 conn(0x7ff9fc0ce520 legacy=0x7ff9fc0cba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:50.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.697+0000 7ffa0080b640 1 -- 172.21.15.105:0/1251862104 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9fc06c310 con 0x7ff9fc0cd5a0 2024-04-15T15:19:50.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.697+0000 7ffa0100c640 1 --2- 172.21.15.105:0/1251862104 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff9fc0d07c0 0x7ff9fc06b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:50.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.698+0000 7ffa0080b640 1 --2- 172.21.15.105:0/1251862104 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9fc0cd5a0 0x7ff9fc0d0ba0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec009870 tx=0x7ff9ec02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1b700520412d12f server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:50.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.698+0000 7ff9f3fff640 1 -- 172.21.15.105:0/1251862104 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec035020 con 0x7ff9fc0cd5a0 2024-04-15T15:19:50.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.698+0000 7ff9f3fff640 1 -- 172.21.15.105:0/1251862104 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9ec031640 con 0x7ff9fc0cd5a0 2024-04-15T15:19:50.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.698+0000 7ff9f3fff640 1 -- 172.21.15.105:0/1251862104 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec02d040 con 0x7ff9fc0cd5a0 2024-04-15T15:19:50.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.699+0000 7ffa03270640 1 -- 172.21.15.105:0/1251862104 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9fc0cd5a0 msgr2=0x7ff9fc0d0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:50.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.699+0000 7ffa03270640 1 --2- 172.21.15.105:0/1251862104 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9fc0cd5a0 0x7ff9fc0d0ba0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec009870 tx=0x7ff9ec02b8f0 comp rx=0 tx=0).stop 2024-04-15T15:19:50.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.699+0000 7ffa03270640 1 -- 172.21.15.105:0/1251862104 shutdown_connections 2024-04-15T15:19:50.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.699+0000 7ffa03270640 1 --2- 172.21.15.105:0/1251862104 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9fc0cd5a0 0x7ff9fc0d0ba0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:50.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.699+0000 7ffa03270640 1 --2- 172.21.15.105:0/1251862104 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff9fc0d07c0 0x7ff9fc06b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:50.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.699+0000 7ffa03270640 1 -- 172.21.15.105:0/1251862104 >> 172.21.15.105:0/1251862104 conn(0x7ff9fc0b2fc0 msgr2=0x7ff9fc0b33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:50.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.699+0000 7ffa03270640 1 -- 172.21.15.105:0/1251862104 shutdown_connections 2024-04-15T15:19:50.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.699+0000 7ffa03270640 1 -- 172.21.15.105:0/1251862104 wait complete. 2024-04-15T15:19:50.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.700+0000 7ffa03270640 1 Processor -- start 2024-04-15T15:19:50.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.700+0000 7ffa03270640 1 -- start start 2024-04-15T15:19:50.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.701+0000 7ffa03270640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9fc0cd5a0 0x7ff9fc15e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:50.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.701+0000 7ffa03270640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff9fc0ce520 0x7ff9fc0c99e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:50.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.701+0000 7ffa03270640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9fc0d07c0 0x7ff9fc1f51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:50.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.701+0000 7ffa03270640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff9fc0ce000 con 0x7ff9fc0cd5a0 2024-04-15T15:19:50.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.701+0000 7ffa03270640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff9fc0cdd20 con 0x7ff9fc0d07c0 2024-04-15T15:19:50.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.701+0000 7ffa03270640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7ff9fc0cc190 con 0x7ff9fc0ce520 2024-04-15T15:19:50.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.701+0000 7ffa0100c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9fc0cd5a0 0x7ff9fc15e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:50.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.701+0000 7ffa0100c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9fc0cd5a0 0x7ff9fc15e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46736/0 (socket says 172.21.15.105:46736) 2024-04-15T15:19:50.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.701+0000 7ffa0100c640 1 -- 172.21.15.105:0/4134897203 learned_addr learned my addr 172.21.15.105:0/4134897203 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:50.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.701+0000 7ffa0180d640 1 --2- 172.21.15.105:0/4134897203 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9fc0d07c0 0x7ff9fc1f51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:50.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.701+0000 7ffa0080b640 1 --2- 172.21.15.105:0/4134897203 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff9fc0ce520 0x7ff9fc0c99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:50.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.701+0000 7ffa0100c640 1 -- 172.21.15.105:0/4134897203 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff9fc0ce520 msgr2=0x7ff9fc0c99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:50.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.701+0000 7ffa0100c640 1 --2- 172.21.15.105:0/4134897203 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff9fc0ce520 0x7ff9fc0c99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:50.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.701+0000 7ffa0100c640 1 -- 172.21.15.105:0/4134897203 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9fc0d07c0 msgr2=0x7ff9fc1f51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:50.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.701+0000 7ffa0100c640 1 --2- 172.21.15.105:0/4134897203 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9fc0d07c0 0x7ff9fc1f51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:50.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.701+0000 7ffa0100c640 1 -- 172.21.15.105:0/4134897203 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9fc1f5730 con 0x7ff9fc0cd5a0 2024-04-15T15:19:50.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.701+0000 7ffa0180d640 1 --2- 172.21.15.105:0/4134897203 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9fc0d07c0 0x7ff9fc1f51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:50.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.701+0000 7ffa0080b640 1 --2- 172.21.15.105:0/4134897203 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff9fc0ce520 0x7ff9fc0c99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:50.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.701+0000 7ffa0100c640 1 --2- 172.21.15.105:0/4134897203 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9fc0cd5a0 0x7ff9fc15e8f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff9f800ce30 tx=0x7ff9f800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:50.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.701+0000 7ff9f1ffb640 1 -- 172.21.15.105:0/4134897203 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f800f890 con 0x7ff9fc0cd5a0 2024-04-15T15:19:50.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.701+0000 7ff9f1ffb640 1 -- 172.21.15.105:0/4134897203 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9f800fa30 con 0x7ff9fc0cd5a0 2024-04-15T15:19:50.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.701+0000 7ffa03270640 1 -- 172.21.15.105:0/4134897203 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9fc15d850 con 0x7ff9fc0cd5a0 2024-04-15T15:19:50.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.701+0000 7ff9f1ffb640 1 -- 172.21.15.105:0/4134897203 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f800fe00 con 0x7ff9fc0cd5a0 2024-04-15T15:19:50.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.702+0000 7ffa03270640 1 -- 172.21.15.105:0/4134897203 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9fc15de10 con 0x7ff9fc0cd5a0 2024-04-15T15:19:50.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.703+0000 7ffa03270640 1 -- 172.21.15.105:0/4134897203 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9fc068a40 con 0x7ff9fc0cd5a0 2024-04-15T15:19:50.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.703+0000 7ff9f1ffb640 1 -- 172.21.15.105:0/4134897203 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff9f8018400 con 0x7ff9fc0cd5a0 2024-04-15T15:19:50.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.703+0000 7ff9f1ffb640 1 --2- 172.21.15.105:0/4134897203 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff9e006c920 0x7ff9e006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:50.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.703+0000 7ff9f1ffb640 1 -- 172.21.15.105:0/4134897203 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff9f808c8d0 con 0x7ff9fc0cd5a0 2024-04-15T15:19:50.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.703+0000 7ffa0080b640 1 --2- 172.21.15.105:0/4134897203 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff9e006c920 0x7ff9e006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:50.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.704+0000 7ffa0080b640 1 --2- 172.21.15.105:0/4134897203 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff9e006c920 0x7ff9e006ede0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec036000 tx=0x7ff9ec008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:50.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.708+0000 7ff9f1ffb640 1 -- 172.21.15.105:0/4134897203 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff9fc1f5730 con 0x7ff9fc0cd5a0 2024-04-15T15:19:50.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.882+0000 7ffa03270640 1 -- 172.21.15.105:0/4134897203 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff9fc0b0000 con 0x7ff9fc0cd5a0 2024-04-15T15:19:50.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.883+0000 7ff9f1ffb640 1 -- 172.21.15.105:0/4134897203 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7ff9f8014030 con 0x7ff9fc0cd5a0 2024-04-15T15:19:50.886 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-04-15T15:19:50.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.885+0000 7ffa03270640 1 -- 172.21.15.105:0/4134897203 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff9e006c920 msgr2=0x7ff9e006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:50.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.885+0000 7ffa03270640 1 --2- 172.21.15.105:0/4134897203 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff9e006c920 0x7ff9e006ede0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec036000 tx=0x7ff9ec008030 comp rx=0 tx=0).stop 2024-04-15T15:19:50.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.885+0000 7ffa03270640 1 -- 172.21.15.105:0/4134897203 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9fc0cd5a0 msgr2=0x7ff9fc15e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:50.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.885+0000 7ffa03270640 1 --2- 172.21.15.105:0/4134897203 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9fc0cd5a0 0x7ff9fc15e8f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff9f800ce30 tx=0x7ff9f800a650 comp rx=0 tx=0).stop 2024-04-15T15:19:50.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.885+0000 7ffa03270640 1 -- 172.21.15.105:0/4134897203 shutdown_connections 2024-04-15T15:19:50.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.885+0000 7ffa03270640 1 --2- 172.21.15.105:0/4134897203 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff9fc0ce520 0x7ff9fc0c99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:50.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.885+0000 7ffa03270640 1 --2- 172.21.15.105:0/4134897203 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff9e006c920 0x7ff9e006ede0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:50.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.885+0000 7ffa03270640 1 --2- 172.21.15.105:0/4134897203 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9fc0d07c0 0x7ff9fc1f51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:50.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.885+0000 7ffa03270640 1 --2- 172.21.15.105:0/4134897203 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9fc0cd5a0 0x7ff9fc15e8f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:50.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.885+0000 7ffa03270640 1 -- 172.21.15.105:0/4134897203 >> 172.21.15.105:0/4134897203 conn(0x7ff9fc0b2fc0 msgr2=0x7ff9fc0ae2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:50.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.885+0000 7ffa03270640 1 -- 172.21.15.105:0/4134897203 shutdown_connections 2024-04-15T15:19:50.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:50.885+0000 7ffa03270640 1 -- 172.21.15.105:0/4134897203 wait complete. 2024-04-15T15:19:50.899 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-15T15:19:51.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.003+0000 7f81525d4640 1 Processor -- start 2024-04-15T15:19:51.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.003+0000 7f81525d4640 1 -- start start 2024-04-15T15:19:51.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.004+0000 7f81525d4640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f814c15fd00 0x7f814c0b2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:51.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.004+0000 7f81525d4640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f814c0b1920 0x7f814c06c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:51.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.004+0000 7f81525d4640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f814c0b2860 con 0x7f814c0cd060 2024-04-15T15:19:51.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.004+0000 7f81525d4640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f814c1600e0 con 0x7f814c15fd00 2024-04-15T15:19:51.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.004+0000 7f81525d4640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f814c06bdd0 con 0x7f814c0b1920 2024-04-15T15:19:51.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.004+0000 7f8150b71640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f814c0cd060 0x7f814c0cd4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:57326/0 (socket says 172.21.15.105:57326) 2024-04-15T15:19:51.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.004+0000 7f8150b71640 1 -- 172.21.15.105:0/2607605027 learned_addr learned my addr 172.21.15.105:0/2607605027 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:19:51.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.004+0000 7f814affd640 1 -- 172.21.15.105:0/2607605027 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1042204294 0 0) 0x7f814c0b2860 con 0x7f814c0cd060 2024-04-15T15:19:51.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.004+0000 7f814affd640 1 -- 172.21.15.105:0/2607605027 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8134003520 con 0x7f814c0cd060 2024-04-15T15:19:51.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.004+0000 7f814bfff640 1 --2- 172.21.15.105:0/2607605027 >> v2:172.21.15.154:3300/0 conn(0x7f814c15fd00 0x7f814c0b2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:51.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.004+0000 7f814b7fe640 1 --2- 172.21.15.105:0/2607605027 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f814c0b1920 0x7f814c06c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:51.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.004+0000 7f814affd640 1 -- 172.21.15.105:0/2607605027 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1657687851 0 0) 0x7f8134003520 con 0x7f814c0cd060 2024-04-15T15:19:51.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.004+0000 7f814affd640 1 -- 172.21.15.105:0/2607605027 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f814c0b1920 msgr2=0x7f814c06c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:51.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.004+0000 7f814affd640 1 --2- 172.21.15.105:0/2607605027 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f814c0b1920 0x7f814c06c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:51.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.004+0000 7f814affd640 1 -- 172.21.15.105:0/2607605027 >> v2:172.21.15.154:3300/0 conn(0x7f814c15fd00 msgr2=0x7f814c0b2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:51.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.004+0000 7f814affd640 1 --2- 172.21.15.105:0/2607605027 >> v2:172.21.15.154:3300/0 conn(0x7f814c15fd00 0x7f814c0b2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:51.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.004+0000 7f814affd640 1 -- 172.21.15.105:0/2607605027 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f814c0b2b20 con 0x7f814c0cd060 2024-04-15T15:19:51.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.004+0000 7f814affd640 1 -- 172.21.15.105:0/2607605027 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7f81440019f0 con 0x7f814c0cd060 2024-04-15T15:19:51.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.004+0000 7f814affd640 1 -- 172.21.15.105:0/2607605027 >> v1:172.21.15.105:6789/0 conn(0x7f814c0cd060 legacy=0x7f814c0cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:51.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.004+0000 7f814affd640 1 --2- 172.21.15.105:0/2607605027 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f81340039b0 0x7f8134003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:51.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.004+0000 7f814affd640 1 --2- 172.21.15.105:0/2607605027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81340043a0 0x7f8134004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:51.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.005+0000 7f814affd640 1 --2- 172.21.15.105:0/2607605027 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8134004fa0 0x7f813400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:51.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.005+0000 7f814affd640 1 -- 172.21.15.105:0/2607605027 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f814c1600e0 con 0x7f81340043a0 2024-04-15T15:19:51.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.005+0000 7f814affd640 1 -- 172.21.15.105:0/2607605027 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f814c06bdd0 con 0x7f8134004fa0 2024-04-15T15:19:51.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.005+0000 7f814affd640 1 -- 172.21.15.105:0/2607605027 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f813400bf30 con 0x7f81340039b0 2024-04-15T15:19:51.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.005+0000 7f814b7fe640 1 --2- 172.21.15.105:0/2607605027 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f814c0b1920 0x7f814c06c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:19:51.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.005+0000 7f814b7fe640 1 --2- 172.21.15.105:0/2607605027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81340043a0 0x7f8134004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:51.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.005+0000 7f814bfff640 1 --2- 172.21.15.105:0/2607605027 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f81340039b0 0x7f8134003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:51.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.005+0000 7f8150b71640 1 --2- 172.21.15.105:0/2607605027 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8134004fa0 0x7f813400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:51.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.005+0000 7f814b7fe640 1 -- 172.21.15.105:0/2607605027 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f81340039b0 msgr2=0x7f8134003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:51.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.005+0000 7f814b7fe640 1 --2- 172.21.15.105:0/2607605027 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f81340039b0 0x7f8134003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:51.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.005+0000 7f814b7fe640 1 -- 172.21.15.105:0/2607605027 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8134004fa0 msgr2=0x7f813400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:51.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.005+0000 7f814b7fe640 1 --2- 172.21.15.105:0/2607605027 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8134004fa0 0x7f813400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:51.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.005+0000 7f814b7fe640 1 -- 172.21.15.105:0/2607605027 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8134003520 con 0x7f81340043a0 2024-04-15T15:19:51.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.006+0000 7f814b7fe640 1 --2- 172.21.15.105:0/2607605027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81340043a0 0x7f8134004880 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8140006fd0 tx=0x7f8134005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5982e56ddb6b19c server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:51.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.006+0000 7f814affd640 1 -- 172.21.15.105:0/2607605027 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f814000f040 con 0x7f81340043a0 2024-04-15T15:19:51.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.006+0000 7f814affd640 1 -- 172.21.15.105:0/2607605027 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8140033600 con 0x7f81340043a0 2024-04-15T15:19:51.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.006+0000 7f814affd640 1 -- 172.21.15.105:0/2607605027 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81400339d0 con 0x7f81340043a0 2024-04-15T15:19:51.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.006+0000 7f81525d4640 1 -- 172.21.15.105:0/2607605027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81340043a0 msgr2=0x7f8134004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:51.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.006+0000 7f81525d4640 1 --2- 172.21.15.105:0/2607605027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81340043a0 0x7f8134004880 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8140006fd0 tx=0x7f8134005830 comp rx=0 tx=0).stop 2024-04-15T15:19:51.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.006+0000 7f8150b71640 1 -- 172.21.15.105:0/2607605027 reap_dead start 2024-04-15T15:19:51.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.006+0000 7f81525d4640 1 -- 172.21.15.105:0/2607605027 shutdown_connections 2024-04-15T15:19:51.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.006+0000 7f81525d4640 1 -- 172.21.15.105:0/2607605027 >> 172.21.15.105:0/2607605027 conn(0x7f814c09aa40 msgr2=0x7f814c0b30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:51.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.006+0000 7f81525d4640 1 -- 172.21.15.105:0/2607605027 shutdown_connections 2024-04-15T15:19:51.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.006+0000 7f81525d4640 1 -- 172.21.15.105:0/2607605027 wait complete. 2024-04-15T15:19:51.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.008+0000 7f81525d4640 1 Processor -- start 2024-04-15T15:19:51.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.008+0000 7f81525d4640 1 -- start start 2024-04-15T15:19:51.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.008+0000 7f81525d4640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f81340043a0 0x7f814c152850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:51.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.008+0000 7f81525d4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f814c0b5ba0 0x7f814c0b4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:51.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.008+0000 7f81525d4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f814c0c6b90 0x7f814c15e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:51.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.008+0000 7f81525d4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f814c0b1d60 con 0x7f814c0c6b90 2024-04-15T15:19:51.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.008+0000 7f81525d4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f814c0b2e00 con 0x7f814c0b5ba0 2024-04-15T15:19:51.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.008+0000 7f81525d4640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f814c06c260 con 0x7f81340043a0 2024-04-15T15:19:51.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.009+0000 7f8150b71640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f814c0c6b90 0x7f814c15e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:51.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.009+0000 7f8150b71640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f814c0c6b90 0x7f814c15e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46744/0 (socket says 172.21.15.105:46744) 2024-04-15T15:19:51.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.009+0000 7f8150b71640 1 -- 172.21.15.105:0/1929671294 learned_addr learned my addr 172.21.15.105:0/1929671294 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:51.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.009+0000 7f814bfff640 1 --2- 172.21.15.105:0/1929671294 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f81340043a0 0x7f814c152850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:51.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.009+0000 7f814b7fe640 1 --2- 172.21.15.105:0/1929671294 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f814c0b5ba0 0x7f814c0b4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:51.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.009+0000 7f8150b71640 1 -- 172.21.15.105:0/1929671294 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f81340043a0 msgr2=0x7f814c152850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:51.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.009+0000 7f8150b71640 1 --2- 172.21.15.105:0/1929671294 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f81340043a0 0x7f814c152850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:51.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.009+0000 7f8150b71640 1 -- 172.21.15.105:0/1929671294 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f814c0b5ba0 msgr2=0x7f814c0b4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:51.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.009+0000 7f8150b71640 1 --2- 172.21.15.105:0/1929671294 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f814c0b5ba0 0x7f814c0b4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:51.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.009+0000 7f8150b71640 1 -- 172.21.15.105:0/1929671294 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f814c152d90 con 0x7f814c0c6b90 2024-04-15T15:19:51.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.009+0000 7f8150b71640 1 --2- 172.21.15.105:0/1929671294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f814c0c6b90 0x7f814c15e4a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8144007600 tx=0x7f8144007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:51.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.009+0000 7f8148ff9640 1 -- 172.21.15.105:0/1929671294 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f814400aaf0 con 0x7f814c0c6b90 2024-04-15T15:19:51.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.009+0000 7f8148ff9640 1 -- 172.21.15.105:0/1929671294 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8144018070 con 0x7f814c0c6b90 2024-04-15T15:19:51.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.009+0000 7f81525d4640 1 -- 172.21.15.105:0/1929671294 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f814c0b4b00 con 0x7f814c0c6b90 2024-04-15T15:19:51.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.009+0000 7f8148ff9640 1 -- 172.21.15.105:0/1929671294 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81440133f0 con 0x7f814c0c6b90 2024-04-15T15:19:51.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.009+0000 7f81525d4640 1 -- 172.21.15.105:0/1929671294 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f814c0b50c0 con 0x7f814c0c6b90 2024-04-15T15:19:51.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.010+0000 7f81525d4640 1 -- 172.21.15.105:0/1929671294 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f814c069150 con 0x7f814c0c6b90 2024-04-15T15:19:51.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.011+0000 7f8148ff9640 1 -- 172.21.15.105:0/1929671294 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8144013590 con 0x7f814c0c6b90 2024-04-15T15:19:51.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.011+0000 7f8148ff9640 1 --2- 172.21.15.105:0/1929671294 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f813806c920 0x7f813806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:51.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.011+0000 7f8148ff9640 1 -- 172.21.15.105:0/1929671294 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f814408c010 con 0x7f814c0c6b90 2024-04-15T15:19:51.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.011+0000 7f814bfff640 1 --2- 172.21.15.105:0/1929671294 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f813806c920 0x7f813806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:51.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.011+0000 7f814bfff640 1 --2- 172.21.15.105:0/1929671294 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f813806c920 0x7f813806ede0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f813c009bc0 tx=0x7f813c009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:51.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.015+0000 7f8148ff9640 1 -- 172.21.15.105:0/1929671294 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f814405a440 con 0x7f814c0c6b90 2024-04-15T15:19:51.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.190+0000 7f81525d4640 1 -- 172.21.15.105:0/1929671294 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f814c06f320 con 0x7f814c0c6b90 2024-04-15T15:19:51.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.190+0000 7f8148ff9640 1 -- 172.21.15.105:0/1929671294 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f8144059d40 con 0x7f814c0c6b90 2024-04-15T15:19:51.193 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-04-15T15:19:51.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.192+0000 7f81525d4640 1 -- 172.21.15.105:0/1929671294 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f813806c920 msgr2=0x7f813806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:51.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.192+0000 7f81525d4640 1 --2- 172.21.15.105:0/1929671294 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f813806c920 0x7f813806ede0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f813c009bc0 tx=0x7f813c009190 comp rx=0 tx=0).stop 2024-04-15T15:19:51.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.192+0000 7f81525d4640 1 -- 172.21.15.105:0/1929671294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f814c0c6b90 msgr2=0x7f814c15e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:51.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.192+0000 7f81525d4640 1 --2- 172.21.15.105:0/1929671294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f814c0c6b90 0x7f814c15e4a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8144007600 tx=0x7f8144007a00 comp rx=0 tx=0).stop 2024-04-15T15:19:51.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.192+0000 7f81525d4640 1 -- 172.21.15.105:0/1929671294 shutdown_connections 2024-04-15T15:19:51.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.192+0000 7f81525d4640 1 --2- 172.21.15.105:0/1929671294 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f813806c920 0x7f813806ede0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:51.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.192+0000 7f81525d4640 1 --2- 172.21.15.105:0/1929671294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f814c0c6b90 0x7f814c15e4a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:51.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.192+0000 7f81525d4640 1 --2- 172.21.15.105:0/1929671294 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f814c0b5ba0 0x7f814c0b4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:51.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.192+0000 7f81525d4640 1 --2- 172.21.15.105:0/1929671294 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f81340043a0 0x7f814c152850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:51.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.192+0000 7f81525d4640 1 -- 172.21.15.105:0/1929671294 >> 172.21.15.105:0/1929671294 conn(0x7f814c09aa40 msgr2=0x7f814c06f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:51.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.192+0000 7f81525d4640 1 -- 172.21.15.105:0/1929671294 shutdown_connections 2024-04-15T15:19:51.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.192+0000 7f81525d4640 1 -- 172.21.15.105:0/1929671294 wait complete. 2024-04-15T15:19:51.206 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:19:51.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.307+0000 7f748590e640 1 Processor -- start 2024-04-15T15:19:51.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.307+0000 7f748590e640 1 -- start start 2024-04-15T15:19:51.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.308+0000 7f748590e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f748015eb60 0x7f74800d11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:51.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.308+0000 7f748590e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f74800d1710 0x7f7480162630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:51.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.308+0000 7f748590e640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f748015ef40 con 0x7f7480162b70 2024-04-15T15:19:51.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.308+0000 7f748590e640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f74800d1b60 con 0x7f74800d1710 2024-04-15T15:19:51.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.308+0000 7f748590e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f7480163540 con 0x7f748015eb60 2024-04-15T15:19:51.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.308+0000 7f7477fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f74800d1710 0x7f7480162630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:51.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.308+0000 7f747f7fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f7480162b70 0x7f74800b6240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:57342/0 (socket says 172.21.15.105:57342) 2024-04-15T15:19:51.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.308+0000 7f747f7fe640 1 -- 172.21.15.105:0/915163267 learned_addr learned my addr 172.21.15.105:0/915163267 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:19:51.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.308+0000 7f747effd640 1 --2- 172.21.15.105:0/915163267 >> v2:172.21.15.154:3300/0 conn(0x7f748015eb60 0x7f74800d11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:51.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.308+0000 7f7477fff640 1 -- 172.21.15.105:0/915163267 >> v2:172.21.15.154:3300/0 conn(0x7f748015eb60 msgr2=0x7f74800d11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:51.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.308+0000 7f7477fff640 1 --2- 172.21.15.105:0/915163267 >> v2:172.21.15.154:3300/0 conn(0x7f748015eb60 0x7f74800d11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:51.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.308+0000 7f7477fff640 1 -- 172.21.15.105:0/915163267 >> v1:172.21.15.105:6789/0 conn(0x7f7480162b70 legacy=0x7f74800b6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:51.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.308+0000 7f7477fff640 1 -- 172.21.15.105:0/915163267 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7480163260 con 0x7f74800d1710 2024-04-15T15:19:51.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.308+0000 7f747e7fc640 1 -- 172.21.15.105:0/915163267 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1936750283 0 0) 0x7f748015ef40 con 0x7f7480162b70 2024-04-15T15:19:51.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.309+0000 7f7477fff640 1 --2- 172.21.15.105:0/915163267 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74800d1710 0x7f7480162630 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f74800d5880 tx=0x7f746802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=190b0ba2e4262175 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:51.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.309+0000 7f747e7fc640 1 -- 172.21.15.105:0/915163267 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7468035020 con 0x7f74800d1710 2024-04-15T15:19:51.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.309+0000 7f747e7fc640 1 -- 172.21.15.105:0/915163267 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7468031830 con 0x7f74800d1710 2024-04-15T15:19:51.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.309+0000 7f747e7fc640 1 -- 172.21.15.105:0/915163267 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7468031c00 con 0x7f74800d1710 2024-04-15T15:19:51.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.310+0000 7f748590e640 1 -- 172.21.15.105:0/915163267 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74800d1710 msgr2=0x7f7480162630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:51.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.310+0000 7f748590e640 1 --2- 172.21.15.105:0/915163267 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74800d1710 0x7f7480162630 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f74800d5880 tx=0x7f746802b8d0 comp rx=0 tx=0).stop 2024-04-15T15:19:51.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.310+0000 7f748590e640 1 -- 172.21.15.105:0/915163267 shutdown_connections 2024-04-15T15:19:51.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.310+0000 7f748590e640 1 --2- 172.21.15.105:0/915163267 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74800d1710 0x7f7480162630 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:51.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.310+0000 7f748590e640 1 --2- 172.21.15.105:0/915163267 >> v2:172.21.15.154:3300/0 conn(0x7f748015eb60 0x7f74800d11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:51.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.310+0000 7f748590e640 1 -- 172.21.15.105:0/915163267 >> 172.21.15.105:0/915163267 conn(0x7f7480067720 msgr2=0x7f7480067b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:51.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.310+0000 7f748590e640 1 -- 172.21.15.105:0/915163267 shutdown_connections 2024-04-15T15:19:51.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.310+0000 7f748590e640 1 -- 172.21.15.105:0/915163267 wait complete. 2024-04-15T15:19:51.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.311+0000 7f748590e640 1 Processor -- start 2024-04-15T15:19:51.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.311+0000 7f748590e640 1 -- start start 2024-04-15T15:19:51.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.312+0000 7f748590e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74800d1710 0x7f74801f4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:51.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.312+0000 7f748590e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f748015eb60 0x7f74801615c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:51.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.312+0000 7f748590e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7480162b70 0x7f748014a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:51.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.312+0000 7f748590e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f74800d2c50 con 0x7f74800d1710 2024-04-15T15:19:51.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.312+0000 7f748590e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f74800d2fc0 con 0x7f748015eb60 2024-04-15T15:19:51.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.312+0000 7f748590e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f7480162f80 con 0x7f7480162b70 2024-04-15T15:19:51.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.312+0000 7f747effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74800d1710 0x7f74801f4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:51.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.312+0000 7f747effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74800d1710 0x7f74801f4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46760/0 (socket says 172.21.15.105:46760) 2024-04-15T15:19:51.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.312+0000 7f747effd640 1 -- 172.21.15.105:0/1002851026 learned_addr learned my addr 172.21.15.105:0/1002851026 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:51.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.312+0000 7f7477fff640 1 --2- 172.21.15.105:0/1002851026 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f748015eb60 0x7f74801615c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:51.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.312+0000 7f747f7fe640 1 --2- 172.21.15.105:0/1002851026 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7480162b70 0x7f748014a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:51.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.312+0000 7f747effd640 1 -- 172.21.15.105:0/1002851026 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7480162b70 msgr2=0x7f748014a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:51.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.312+0000 7f747effd640 1 --2- 172.21.15.105:0/1002851026 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7480162b70 0x7f748014a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:51.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.312+0000 7f747effd640 1 -- 172.21.15.105:0/1002851026 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f748015eb60 msgr2=0x7f74801615c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:51.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.312+0000 7f747effd640 1 --2- 172.21.15.105:0/1002851026 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f748015eb60 0x7f74801615c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:51.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.312+0000 7f747effd640 1 -- 172.21.15.105:0/1002851026 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74801f52c0 con 0x7f74800d1710 2024-04-15T15:19:51.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.312+0000 7f747f7fe640 1 --2- 172.21.15.105:0/1002851026 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7480162b70 0x7f748014a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:19:51.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.312+0000 7f747effd640 1 --2- 172.21.15.105:0/1002851026 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74800d1710 0x7f74801f4d80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f747000d830 tx=0x7f747000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:51.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.312+0000 7f747cff9640 1 -- 172.21.15.105:0/1002851026 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747000b6c0 con 0x7f74800d1710 2024-04-15T15:19:51.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.312+0000 7f747cff9640 1 -- 172.21.15.105:0/1002851026 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7470019070 con 0x7f74800d1710 2024-04-15T15:19:51.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.312+0000 7f748590e640 1 -- 172.21.15.105:0/1002851026 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7480160520 con 0x7f74800d1710 2024-04-15T15:19:51.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.312+0000 7f747cff9640 1 -- 172.21.15.105:0/1002851026 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747000ba80 con 0x7f74800d1710 2024-04-15T15:19:51.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.312+0000 7f748590e640 1 -- 172.21.15.105:0/1002851026 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7480160ae0 con 0x7f74800d1710 2024-04-15T15:19:51.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.313+0000 7f748590e640 1 -- 172.21.15.105:0/1002851026 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7480068a40 con 0x7f74800d1710 2024-04-15T15:19:51.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.314+0000 7f747cff9640 1 -- 172.21.15.105:0/1002851026 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f747001e460 con 0x7f74800d1710 2024-04-15T15:19:51.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.314+0000 7f747cff9640 1 --2- 172.21.15.105:0/1002851026 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f746006c920 0x7f746006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:51.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.314+0000 7f7477fff640 1 --2- 172.21.15.105:0/1002851026 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f746006c920 0x7f746006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:51.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.315+0000 7f747cff9640 1 -- 172.21.15.105:0/1002851026 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f747008c090 con 0x7f74800d1710 2024-04-15T15:19:51.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.315+0000 7f7477fff640 1 --2- 172.21.15.105:0/1002851026 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f746006c920 0x7f746006ede0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7468036000 tx=0x7f7468008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:51.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.320+0000 7f747cff9640 1 -- 172.21.15.105:0/1002851026 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f74801f52c0 con 0x7f74800d1710 2024-04-15T15:19:51.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.494+0000 7f748590e640 1 -- 172.21.15.105:0/1002851026 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f74800d23f0 con 0x7f74800d1710 2024-04-15T15:19:51.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.495+0000 7f747cff9640 1 -- 172.21.15.105:0/1002851026 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16653 (secure 0 0 0) 0x7f747005a550 con 0x7f74800d1710 2024-04-15T15:19:51.498 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:19:51.499 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":20,"fsid":"09e4d2a4-a324-4793-af77-644b25e520e1","created":"2024-04-15T15:19:01.524543+0000","modified":"2024-04-15T15:19:27.414890+0000","last_up_change":"2024-04-15T15:19:12.825815+0000","last_in_change":"2024-04-15T15:19:03.788126+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:19:13.748585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:19:23.789287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:19:24.722250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6f6292b-2d7e-4aa0-9d1c-6ddd0f97287c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6801","nonce":2873040054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6803","nonce":2873040054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6807","nonce":2873040054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6805","nonce":2873040054}]},"public_addr":"172.21.15.105:6801/2873040054","cluster_addr":"172.21.15.105:6803/2873040054","heartbeat_back_addr":"172.21.15.105:6807/2873040054","heartbeat_front_addr":"172.21.15.105:6805/2873040054","state":["exists","up"]},{"osd":1,"uuid":"8900966a-b393-41e3-8f93-e5a43bd23622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6818","nonce":2494994479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6821","nonce":2494994479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6828","nonce":2494994479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6823","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6825","nonce":2494994479}]},"public_addr":"172.21.15.105:6818/2494994479","cluster_addr":"172.21.15.105:6821/2494994479","heartbeat_back_addr":"172.21.15.105:6828/2494994479","heartbeat_front_addr":"172.21.15.105:6825/2494994479","state":["exists","up"]},{"osd":2,"uuid":"81a5dc7a-9325-4ff5-941d-a2d5fba21f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6809","nonce":3798408699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6811","nonce":3798408699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6815","nonce":3798408699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6813","nonce":3798408699}]},"public_addr":"172.21.15.105:6809/3798408699","cluster_addr":"172.21.15.105:6811/3798408699","heartbeat_back_addr":"172.21.15.105:6815/3798408699","heartbeat_front_addr":"172.21.15.105:6813/3798408699","state":["exists","up"]},{"osd":3,"uuid":"df766ed2-5c71-4805-96f4-3d4970edc960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6819","nonce":1920720186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6824","nonce":1920720186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6831","nonce":1920720186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6827","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6829","nonce":1920720186}]},"public_addr":"172.21.15.105:6819/1920720186","cluster_addr":"172.21.15.105:6824/1920720186","heartbeat_back_addr":"172.21.15.105:6831/1920720186","heartbeat_front_addr":"172.21.15.105:6829/1920720186","state":["exists","up"]},{"osd":4,"uuid":"e1039e03-28b8-49df-b951-c91930b93c9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6810","nonce":3516494456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6813","nonce":3516494456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6820","nonce":3516494456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6817","nonce":3516494456}]},"public_addr":"172.21.15.154:6810/3516494456","cluster_addr":"172.21.15.154:6813/3516494456","heartbeat_back_addr":"172.21.15.154:6820/3516494456","heartbeat_front_addr":"172.21.15.154:6817/3516494456","state":["exists","up"]},{"osd":5,"uuid":"a9f5eaf1-66ff-405a-a9e4-ec3340812a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6801","nonce":540367751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6803","nonce":540367751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6807","nonce":540367751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6805","nonce":540367751}]},"public_addr":"172.21.15.154:6801/540367751","cluster_addr":"172.21.15.154:6803/540367751","heartbeat_back_addr":"172.21.15.154:6807/540367751","heartbeat_front_addr":"172.21.15.154:6805/540367751","state":["exists","up"]},{"osd":6,"uuid":"267d02ea-675f-428f-aae8-e0375e4d4feb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6825","nonce":1550829005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6827","nonce":1550829005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6831","nonce":1550829005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6829","nonce":1550829005}]},"public_addr":"172.21.15.154:6825/1550829005","cluster_addr":"172.21.15.154:6827/1550829005","heartbeat_back_addr":"172.21.15.154:6831/1550829005","heartbeat_front_addr":"172.21.15.154:6829/1550829005","state":["exists","up"]},{"osd":7,"uuid":"759ac6cc-896a-423c-885f-9fe90d8dcb14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6809","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6811","nonce":3920089903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6816","nonce":3920089903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6823","nonce":3920089903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6819","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6821","nonce":3920089903}]},"public_addr":"172.21.15.154:6811/3920089903","cluster_addr":"172.21.15.154:6816/3920089903","heartbeat_back_addr":"172.21.15.154:6823/3920089903","heartbeat_front_addr":"172.21.15.154:6821/3920089903","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.949509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.984072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.907278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.947391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.975447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.998896+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:09.028739+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.894119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:19:51.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.497+0000 7f748590e640 1 -- 172.21.15.105:0/1002851026 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f746006c920 msgr2=0x7f746006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:51.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.497+0000 7f748590e640 1 --2- 172.21.15.105:0/1002851026 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f746006c920 0x7f746006ede0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7468036000 tx=0x7f7468008030 comp rx=0 tx=0).stop 2024-04-15T15:19:51.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.497+0000 7f748590e640 1 -- 172.21.15.105:0/1002851026 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74800d1710 msgr2=0x7f74801f4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:51.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.497+0000 7f748590e640 1 --2- 172.21.15.105:0/1002851026 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74800d1710 0x7f74801f4d80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f747000d830 tx=0x7f747000dcf0 comp rx=0 tx=0).stop 2024-04-15T15:19:51.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.497+0000 7f748590e640 1 -- 172.21.15.105:0/1002851026 shutdown_connections 2024-04-15T15:19:51.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.497+0000 7f748590e640 1 --2- 172.21.15.105:0/1002851026 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7480162b70 0x7f748014a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:51.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.497+0000 7f748590e640 1 --2- 172.21.15.105:0/1002851026 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f746006c920 0x7f746006ede0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:51.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.497+0000 7f748590e640 1 --2- 172.21.15.105:0/1002851026 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f748015eb60 0x7f74801615c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:51.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.497+0000 7f748590e640 1 --2- 172.21.15.105:0/1002851026 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74800d1710 0x7f74801f4d80 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:51.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.497+0000 7f748590e640 1 -- 172.21.15.105:0/1002851026 >> 172.21.15.105:0/1002851026 conn(0x7f7480067720 msgr2=0x7f7480067b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:51.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.498+0000 7f748590e640 1 -- 172.21.15.105:0/1002851026 shutdown_connections 2024-04-15T15:19:51.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.498+0000 7f748590e640 1 -- 172.21.15.105:0/1002851026 wait complete. 2024-04-15T15:19:51.512 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:19:13.748585+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-15T15:19:23.789287+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-15T15:19:24.722250+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:19:51.512 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:19:51.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.615+0000 7f9627e93640 1 Processor -- start 2024-04-15T15:19:51.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.616+0000 7f9627e93640 1 -- start start 2024-04-15T15:19:51.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.616+0000 7f9627e93640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9620144c90 0x7f96200b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:51.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.616+0000 7f9627e93640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96200cd8e0 0x7f962006f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:51.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.616+0000 7f9627e93640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96200b40a0 con 0x7f962006ef90 2024-04-15T15:19:51.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.616+0000 7f9627e93640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f962006f940 con 0x7f9620144c90 2024-04-15T15:19:51.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.616+0000 7f9627e93640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f96200cdcf0 con 0x7f96200cd8e0 2024-04-15T15:19:51.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.617+0000 7f962542e640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f962006ef90 0x7f96200cd1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:57348/0 (socket says 172.21.15.105:57348) 2024-04-15T15:19:51.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.617+0000 7f962542e640 1 -- 172.21.15.105:0/564741375 learned_addr learned my addr 172.21.15.105:0/564741375 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:19:51.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.617+0000 7f9625c2f640 1 --2- 172.21.15.105:0/564741375 >> v2:172.21.15.154:3300/0 conn(0x7f9620144c90 0x7f96200b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:51.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.617+0000 7f9626430640 1 --2- 172.21.15.105:0/564741375 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96200cd8e0 0x7f962006f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:51.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.617+0000 7f9624c2d640 1 -- 172.21.15.105:0/564741375 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2300305148 0 0) 0x7f96200b40a0 con 0x7f962006ef90 2024-04-15T15:19:51.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.617+0000 7f9624c2d640 1 -- 172.21.15.105:0/564741375 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f960c003520 con 0x7f962006ef90 2024-04-15T15:19:51.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.617+0000 7f9624c2d640 1 -- 172.21.15.105:0/564741375 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3778198447 0 0) 0x7f960c003520 con 0x7f962006ef90 2024-04-15T15:19:51.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.617+0000 7f9624c2d640 1 -- 172.21.15.105:0/564741375 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96200cd8e0 msgr2=0x7f962006f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:51.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.617+0000 7f9624c2d640 1 --2- 172.21.15.105:0/564741375 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96200cd8e0 0x7f962006f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:51.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.617+0000 7f9624c2d640 1 -- 172.21.15.105:0/564741375 >> v2:172.21.15.154:3300/0 conn(0x7f9620144c90 msgr2=0x7f96200b4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:51.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.617+0000 7f9624c2d640 1 --2- 172.21.15.105:0/564741375 >> v2:172.21.15.154:3300/0 conn(0x7f9620144c90 0x7f96200b4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:51.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.617+0000 7f9624c2d640 1 -- 172.21.15.105:0/564741375 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96200b4380 con 0x7f962006ef90 2024-04-15T15:19:51.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.617+0000 7f9624c2d640 1 -- 172.21.15.105:0/564741375 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7f9610001670 con 0x7f962006ef90 2024-04-15T15:19:51.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.617+0000 7f9624c2d640 1 -- 172.21.15.105:0/564741375 >> v1:172.21.15.105:6789/0 conn(0x7f962006ef90 legacy=0x7f96200cd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:51.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.617+0000 7f9624c2d640 1 --2- 172.21.15.105:0/564741375 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f960c0039b0 0x7f960c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:51.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.617+0000 7f9624c2d640 1 --2- 172.21.15.105:0/564741375 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f960c004800 0x7f960c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:51.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.617+0000 7f9624c2d640 1 --2- 172.21.15.105:0/564741375 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96200cd8e0 0x7f960c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:51.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.617+0000 7f9624c2d640 1 -- 172.21.15.105:0/564741375 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f962006f940 con 0x7f960c0039b0 2024-04-15T15:19:51.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.617+0000 7f9624c2d640 1 -- 172.21.15.105:0/564741375 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f960c00b9c0 con 0x7f960c004800 2024-04-15T15:19:51.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.617+0000 7f9624c2d640 1 -- 172.21.15.105:0/564741375 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f960c00bb40 con 0x7f96200cd8e0 2024-04-15T15:19:51.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.617+0000 7f9625c2f640 1 --2- 172.21.15.105:0/564741375 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f960c0039b0 0x7f960c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:51.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.617+0000 7f962542e640 1 --2- 172.21.15.105:0/564741375 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f960c004800 0x7f960c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:51.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.617+0000 7f9626430640 1 --2- 172.21.15.105:0/564741375 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96200cd8e0 0x7f960c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:51.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.618+0000 7f9625c2f640 1 -- 172.21.15.105:0/564741375 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96200cd8e0 msgr2=0x7f960c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:51.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.618+0000 7f9625c2f640 1 --2- 172.21.15.105:0/564741375 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96200cd8e0 0x7f960c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:51.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.618+0000 7f9625c2f640 1 -- 172.21.15.105:0/564741375 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f960c004800 msgr2=0x7f960c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:51.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.618+0000 7f9625c2f640 1 --2- 172.21.15.105:0/564741375 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f960c004800 0x7f960c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:51.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.618+0000 7f9625c2f640 1 -- 172.21.15.105:0/564741375 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f960c003520 con 0x7f960c0039b0 2024-04-15T15:19:51.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.618+0000 7f9626430640 1 --2- 172.21.15.105:0/564741375 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96200cd8e0 0x7f960c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:19:51.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.618+0000 7f9625c2f640 1 --2- 172.21.15.105:0/564741375 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f960c0039b0 0x7f960c003dd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f961c00bd90 tx=0x7f960c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=981f4eabfe516b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:51.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.619+0000 7f9624c2d640 1 -- 172.21.15.105:0/564741375 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961c00b930 con 0x7f960c0039b0 2024-04-15T15:19:51.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.619+0000 7f9624c2d640 1 -- 172.21.15.105:0/564741375 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f961c0306f0 con 0x7f960c0039b0 2024-04-15T15:19:51.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.619+0000 7f9624c2d640 1 -- 172.21.15.105:0/564741375 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961c030ac0 con 0x7f960c0039b0 2024-04-15T15:19:51.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.619+0000 7f9627e93640 1 -- 172.21.15.105:0/564741375 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f960c0039b0 msgr2=0x7f960c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:51.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.619+0000 7f9627e93640 1 --2- 172.21.15.105:0/564741375 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f960c0039b0 0x7f960c003dd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f961c00bd90 tx=0x7f960c005830 comp rx=0 tx=0).stop 2024-04-15T15:19:51.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.619+0000 7f9626430640 1 -- 172.21.15.105:0/564741375 reap_dead start 2024-04-15T15:19:51.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.619+0000 7f9627e93640 1 -- 172.21.15.105:0/564741375 shutdown_connections 2024-04-15T15:19:51.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.619+0000 7f9627e93640 1 -- 172.21.15.105:0/564741375 >> 172.21.15.105:0/564741375 conn(0x7f9620067d50 msgr2=0x7f962006b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:51.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.619+0000 7f9627e93640 1 -- 172.21.15.105:0/564741375 shutdown_connections 2024-04-15T15:19:51.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.619+0000 7f9627e93640 1 -- 172.21.15.105:0/564741375 wait complete. 2024-04-15T15:19:51.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.620+0000 7f9627e93640 1 Processor -- start 2024-04-15T15:19:51.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.620+0000 7f9627e93640 1 -- start start 2024-04-15T15:19:51.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.621+0000 7f9627e93640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f960c0039b0 0x7f9620150f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:51.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.621+0000 7f9627e93640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f962015c5a0 0x7f9620138450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:51.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.621+0000 7f9627e93640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9620139570 0x7f96201399f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:51.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.621+0000 7f9627e93640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f96200b4940 con 0x7f9620139570 2024-04-15T15:19:51.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.621+0000 7f9627e93640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f96200b3e50 con 0x7f962015c5a0 2024-04-15T15:19:51.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.621+0000 7f9627e93640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f9620145070 con 0x7f960c0039b0 2024-04-15T15:19:51.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.621+0000 7f9626430640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9620139570 0x7f96201399f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:51.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.621+0000 7f9626430640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9620139570 0x7f96201399f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46774/0 (socket says 172.21.15.105:46774) 2024-04-15T15:19:51.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.621+0000 7f9626430640 1 -- 172.21.15.105:0/3942166816 learned_addr learned my addr 172.21.15.105:0/3942166816 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:51.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.621+0000 7f9626430640 1 -- 172.21.15.105:0/3942166816 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f960c0039b0 msgr2=0x7f9620150f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:51.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.621+0000 7f9626430640 1 --2- 172.21.15.105:0/3942166816 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f960c0039b0 0x7f9620150f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:51.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.621+0000 7f962542e640 1 --2- 172.21.15.105:0/3942166816 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f962015c5a0 0x7f9620138450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:51.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.621+0000 7f9626430640 1 -- 172.21.15.105:0/3942166816 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f962015c5a0 msgr2=0x7f9620138450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:51.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.622+0000 7f9626430640 1 --2- 172.21.15.105:0/3942166816 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f962015c5a0 0x7f9620138450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:51.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.622+0000 7f9626430640 1 -- 172.21.15.105:0/3942166816 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f962015ca00 con 0x7f9620139570 2024-04-15T15:19:51.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.622+0000 7f962542e640 1 --2- 172.21.15.105:0/3942166816 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f962015c5a0 0x7f9620138450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:51.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.622+0000 7f9626430640 1 --2- 172.21.15.105:0/3942166816 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9620139570 0x7f96201399f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9620073300 tx=0x7f9618010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:51.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.622+0000 7f96167fc640 1 -- 172.21.15.105:0/3942166816 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9618003990 con 0x7f9620139570 2024-04-15T15:19:51.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.622+0000 7f96167fc640 1 -- 172.21.15.105:0/3942166816 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9618003b50 con 0x7f9620139570 2024-04-15T15:19:51.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.622+0000 7f96167fc640 1 -- 172.21.15.105:0/3942166816 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9618004020 con 0x7f9620139570 2024-04-15T15:19:51.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.622+0000 7f9627e93640 1 -- 172.21.15.105:0/3942166816 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9620138990 con 0x7f9620139570 2024-04-15T15:19:51.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.622+0000 7f9627e93640 1 -- 172.21.15.105:0/3942166816 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f962015cc80 con 0x7f9620139570 2024-04-15T15:19:51.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.623+0000 7f9627e93640 1 -- 172.21.15.105:0/3942166816 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9620068ab0 con 0x7f9620139570 2024-04-15T15:19:51.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.624+0000 7f96167fc640 1 -- 172.21.15.105:0/3942166816 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f961801e030 con 0x7f9620139570 2024-04-15T15:19:51.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.624+0000 7f96167fc640 1 --2- 172.21.15.105:0/3942166816 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f960406c8f0 0x7f960406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:51.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.624+0000 7f9625c2f640 1 --2- 172.21.15.105:0/3942166816 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f960406c8f0 0x7f960406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:51.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.624+0000 7f96167fc640 1 -- 172.21.15.105:0/3942166816 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f961808c310 con 0x7f9620139570 2024-04-15T15:19:51.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.625+0000 7f9625c2f640 1 --2- 172.21.15.105:0/3942166816 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f960406c8f0 0x7f960406edb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f961c039000 tx=0x7f961c033000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:51.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.629+0000 7f96167fc640 1 -- 172.21.15.105:0/3942166816 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9618025020 con 0x7f9620139570 2024-04-15T15:19:51.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.805+0000 7f9627e93640 1 -- 172.21.15.105:0/3942166816 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f962013b5d0 con 0x7f9620139570 2024-04-15T15:19:51.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.805+0000 7f96167fc640 1 -- 172.21.15.105:0/3942166816 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f962015ca00 con 0x7f9620139570 2024-04-15T15:19:51.808 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-04-15T15:19:51.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.807+0000 7f9627e93640 1 -- 172.21.15.105:0/3942166816 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f960406c8f0 msgr2=0x7f960406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:51.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.807+0000 7f9627e93640 1 --2- 172.21.15.105:0/3942166816 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f960406c8f0 0x7f960406edb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f961c039000 tx=0x7f961c033000 comp rx=0 tx=0).stop 2024-04-15T15:19:51.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.807+0000 7f9627e93640 1 -- 172.21.15.105:0/3942166816 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9620139570 msgr2=0x7f96201399f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:51.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.807+0000 7f9627e93640 1 --2- 172.21.15.105:0/3942166816 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9620139570 0x7f96201399f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9620073300 tx=0x7f9618010590 comp rx=0 tx=0).stop 2024-04-15T15:19:51.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.807+0000 7f9627e93640 1 -- 172.21.15.105:0/3942166816 shutdown_connections 2024-04-15T15:19:51.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.807+0000 7f9627e93640 1 --2- 172.21.15.105:0/3942166816 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f960406c8f0 0x7f960406edb0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:51.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.807+0000 7f9627e93640 1 --2- 172.21.15.105:0/3942166816 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9620139570 0x7f96201399f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:51.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.807+0000 7f9627e93640 1 --2- 172.21.15.105:0/3942166816 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f962015c5a0 0x7f9620138450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:51.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.807+0000 7f9627e93640 1 --2- 172.21.15.105:0/3942166816 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f960c0039b0 0x7f9620150f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:51.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.807+0000 7f9627e93640 1 -- 172.21.15.105:0/3942166816 >> 172.21.15.105:0/3942166816 conn(0x7f9620067d50 msgr2=0x7f9620136750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:51.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.807+0000 7f9627e93640 1 -- 172.21.15.105:0/3942166816 shutdown_connections 2024-04-15T15:19:51.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.807+0000 7f9627e93640 1 -- 172.21.15.105:0/3942166816 wait complete. 2024-04-15T15:19:51.821 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-15T15:19:51.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.924+0000 7f7cc7ab1640 1 Processor -- start 2024-04-15T15:19:51.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.925+0000 7f7cc7ab1640 1 -- start start 2024-04-15T15:19:51.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.925+0000 7f7cc7ab1640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7cc015ee90 0x7f7cc0070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:51.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.925+0000 7f7cc7ab1640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7cc00cec90 con 0x7f7cc00cfbb0 2024-04-15T15:19:51.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.925+0000 7f7cc7ab1640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7cc015f270 con 0x7f7cc00cf720 2024-04-15T15:19:51.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.925+0000 7f7cc7ab1640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f7cc006fd20 con 0x7f7cc015ee90 2024-04-15T15:19:51.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.925+0000 7f7cc604e640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f7cc00cfbb0 0x7f7cc00ce580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:57364/0 (socket says 172.21.15.105:57364) 2024-04-15T15:19:51.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.925+0000 7f7cc604e640 1 -- 172.21.15.105:0/1259808098 learned_addr learned my addr 172.21.15.105:0/1259808098 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:19:51.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.926+0000 7f7cc484b640 1 -- 172.21.15.105:0/1259808098 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2033845582 0 0) 0x7f7cc00cec90 con 0x7f7cc00cfbb0 2024-04-15T15:19:51.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.926+0000 7f7cc584d640 1 --2- 172.21.15.105:0/1259808098 >> v2:172.21.15.154:3300/0 conn(0x7f7cc015ee90 0x7f7cc0070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:51.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.926+0000 7f7cc484b640 1 -- 172.21.15.105:0/1259808098 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7cac003520 con 0x7f7cc00cfbb0 2024-04-15T15:19:51.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.926+0000 7f7cc484b640 1 -- 172.21.15.105:0/1259808098 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2677558273 0 0) 0x7f7cac003520 con 0x7f7cc00cfbb0 2024-04-15T15:19:51.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.926+0000 7f7cc484b640 1 -- 172.21.15.105:0/1259808098 >> v2:172.21.15.154:3300/0 conn(0x7f7cc015ee90 msgr2=0x7f7cc0070300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:51.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.926+0000 7f7cc484b640 1 --2- 172.21.15.105:0/1259808098 >> v2:172.21.15.154:3300/0 conn(0x7f7cc015ee90 0x7f7cc0070300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:51.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.926+0000 7f7cc484b640 1 -- 172.21.15.105:0/1259808098 >> v1:172.21.15.154:6789/0 conn(0x7f7cc00cf720 legacy=0x7f7cc00cde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:51.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.926+0000 7f7cc484b640 1 -- 172.21.15.105:0/1259808098 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cc0070000 con 0x7f7cc00cfbb0 2024-04-15T15:19:51.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.926+0000 7f7cc484b640 1 -- 172.21.15.105:0/1259808098 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7f7cb8001670 con 0x7f7cc00cfbb0 2024-04-15T15:19:51.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.926+0000 7f7cc484b640 1 -- 172.21.15.105:0/1259808098 >> v1:172.21.15.105:6789/0 conn(0x7f7cc00cfbb0 legacy=0x7f7cc00ce580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:51.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.926+0000 7f7cc484b640 1 --2- 172.21.15.105:0/1259808098 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7cac0039b0 0x7f7cac003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:51.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.926+0000 7f7cc484b640 1 --2- 172.21.15.105:0/1259808098 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7cac004800 0x7f7cac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:51.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.926+0000 7f7cc484b640 1 --2- 172.21.15.105:0/1259808098 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7cac0051c0 0x7f7cac00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:51.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.926+0000 7f7cc484b640 1 -- 172.21.15.105:0/1259808098 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc006fd20 con 0x7f7cac0051c0 2024-04-15T15:19:51.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.926+0000 7f7cc484b640 1 -- 172.21.15.105:0/1259808098 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7cac00c060 con 0x7f7cac0039b0 2024-04-15T15:19:51.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.926+0000 7f7cc484b640 1 -- 172.21.15.105:0/1259808098 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f7cac00c260 con 0x7f7cac004800 2024-04-15T15:19:51.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.926+0000 7f7cc604e640 1 --2- 172.21.15.105:0/1259808098 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7cac0051c0 0x7f7cac00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:51.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.926+0000 7f7cc584d640 1 --2- 172.21.15.105:0/1259808098 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7cac0039b0 0x7f7cac003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:51.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.926+0000 7f7cc504c640 1 --2- 172.21.15.105:0/1259808098 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7cac004800 0x7f7cac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:51.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.926+0000 7f7cc604e640 1 -- 172.21.15.105:0/1259808098 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7cac004800 msgr2=0x7f7cac004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:51.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.926+0000 7f7cc604e640 1 --2- 172.21.15.105:0/1259808098 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7cac004800 0x7f7cac004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:51.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.926+0000 7f7cc604e640 1 -- 172.21.15.105:0/1259808098 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7cac0039b0 msgr2=0x7f7cac003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:51.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.926+0000 7f7cc604e640 1 --2- 172.21.15.105:0/1259808098 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7cac0039b0 0x7f7cac003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:51.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.927+0000 7f7cc604e640 1 -- 172.21.15.105:0/1259808098 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cac003520 con 0x7f7cac0051c0 2024-04-15T15:19:51.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.927+0000 7f7cc604e640 1 -- 172.21.15.105:0/1259808098 reap_dead start 2024-04-15T15:19:51.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.927+0000 7f7cc604e640 1 --2- 172.21.15.105:0/1259808098 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7cac0051c0 0x7f7cac00b9c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7cb800c210 tx=0x7f7cac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35a49f8fbf3d89f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:51.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.927+0000 7f7cc484b640 1 -- 172.21.15.105:0/1259808098 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb8007d30 con 0x7f7cac0051c0 2024-04-15T15:19:51.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.927+0000 7f7cc484b640 1 -- 172.21.15.105:0/1259808098 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cb8030070 con 0x7f7cac0051c0 2024-04-15T15:19:51.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.927+0000 7f7cc484b640 1 -- 172.21.15.105:0/1259808098 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb802cd60 con 0x7f7cac0051c0 2024-04-15T15:19:51.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.928+0000 7f7cc7ab1640 1 -- 172.21.15.105:0/1259808098 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7cac0051c0 msgr2=0x7f7cac00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:51.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.928+0000 7f7cc7ab1640 1 --2- 172.21.15.105:0/1259808098 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7cac0051c0 0x7f7cac00b9c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7cb800c210 tx=0x7f7cac005830 comp rx=0 tx=0).stop 2024-04-15T15:19:51.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.928+0000 7f7cc7ab1640 1 -- 172.21.15.105:0/1259808098 shutdown_connections 2024-04-15T15:19:51.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.928+0000 7f7cc7ab1640 1 --2- 172.21.15.105:0/1259808098 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7cac0051c0 0x7f7cac00b9c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:51.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.928+0000 7f7cc7ab1640 1 -- 172.21.15.105:0/1259808098 >> 172.21.15.105:0/1259808098 conn(0x7f7cc00b3030 msgr2=0x7f7cc00b3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:51.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.928+0000 7f7cc7ab1640 1 -- 172.21.15.105:0/1259808098 shutdown_connections 2024-04-15T15:19:51.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.928+0000 7f7cc7ab1640 1 -- 172.21.15.105:0/1259808098 wait complete. 2024-04-15T15:19:51.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.929+0000 7f7cc7ab1640 1 Processor -- start 2024-04-15T15:19:51.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.929+0000 7f7cc7ab1640 1 -- start start 2024-04-15T15:19:51.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.930+0000 7f7cc7ab1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7cac0051c0 0x7f7cc014b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:51.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.930+0000 7f7cc7ab1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7cc00c7cf0 0x7f7cc00ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:51.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.930+0000 7f7cc7ab1640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7cc00cd040 0x7f7cc014bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:51.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.930+0000 7f7cc7ab1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc00cffc0 con 0x7f7cac0051c0 2024-04-15T15:19:51.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.930+0000 7f7cc7ab1640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc00cc4e0 con 0x7f7cc00c7cf0 2024-04-15T15:19:51.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.930+0000 7f7cc7ab1640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f7cc00cbe20 con 0x7f7cc00cd040 2024-04-15T15:19:51.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.930+0000 7f7cc584d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7cac0051c0 0x7f7cc014b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:51.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.930+0000 7f7cc584d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7cac0051c0 0x7f7cc014b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46792/0 (socket says 172.21.15.105:46792) 2024-04-15T15:19:51.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.930+0000 7f7cc584d640 1 -- 172.21.15.105:0/2608607412 learned_addr learned my addr 172.21.15.105:0/2608607412 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:51.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.930+0000 7f7cc504c640 1 --2- 172.21.15.105:0/2608607412 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7cc00c7cf0 0x7f7cc00ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:51.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.930+0000 7f7cc584d640 1 -- 172.21.15.105:0/2608607412 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7cc00cd040 msgr2=0x7f7cc014bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:51.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.930+0000 7f7cc584d640 1 --2- 172.21.15.105:0/2608607412 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7cc00cd040 0x7f7cc014bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:51.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.930+0000 7f7cc584d640 1 -- 172.21.15.105:0/2608607412 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7cc00c7cf0 msgr2=0x7f7cc00ccb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:51.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.930+0000 7f7cc584d640 1 --2- 172.21.15.105:0/2608607412 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7cc00c7cf0 0x7f7cc00ccb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:51.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.930+0000 7f7cc584d640 1 -- 172.21.15.105:0/2608607412 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cc014b990 con 0x7f7cac0051c0 2024-04-15T15:19:51.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.930+0000 7f7cc584d640 1 --2- 172.21.15.105:0/2608607412 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7cac0051c0 0x7f7cc014b450 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc009f90 tx=0x7f7cbc00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:51.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.930+0000 7f7cb67fc640 1 -- 172.21.15.105:0/2608607412 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cbc015430 con 0x7f7cac0051c0 2024-04-15T15:19:51.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.930+0000 7f7cb67fc640 1 -- 172.21.15.105:0/2608607412 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cbc0155d0 con 0x7f7cac0051c0 2024-04-15T15:19:51.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.930+0000 7f7cc7ab1640 1 -- 172.21.15.105:0/2608607412 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cc014ebf0 con 0x7f7cac0051c0 2024-04-15T15:19:51.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.931+0000 7f7cb67fc640 1 -- 172.21.15.105:0/2608607412 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cbc0159c0 con 0x7f7cac0051c0 2024-04-15T15:19:51.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.931+0000 7f7cc7ab1640 1 -- 172.21.15.105:0/2608607412 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cc00c7210 con 0x7f7cac0051c0 2024-04-15T15:19:51.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.932+0000 7f7cc7ab1640 1 -- 172.21.15.105:0/2608607412 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cc0069150 con 0x7f7cac0051c0 2024-04-15T15:19:51.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.932+0000 7f7cb67fc640 1 -- 172.21.15.105:0/2608607412 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7cbc015b60 con 0x7f7cac0051c0 2024-04-15T15:19:51.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.932+0000 7f7cb67fc640 1 --2- 172.21.15.105:0/2608607412 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f7ca406c8f0 0x7f7ca406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:51.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.932+0000 7f7cb67fc640 1 -- 172.21.15.105:0/2608607412 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7cbc08cc50 con 0x7f7cac0051c0 2024-04-15T15:19:51.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.932+0000 7f7cc504c640 1 --2- 172.21.15.105:0/2608607412 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f7ca406c8f0 0x7f7ca406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:51.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.933+0000 7f7cc504c640 1 --2- 172.21.15.105:0/2608607412 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f7ca406c8f0 0x7f7ca406edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0000c00 tx=0x7f7cb0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:51.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:51.937+0000 7f7cb67fc640 1 -- 172.21.15.105:0/2608607412 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7cbc092020 con 0x7f7cac0051c0 2024-04-15T15:19:52.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.111+0000 7f7cc7ab1640 1 -- 172.21.15.105:0/2608607412 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7cc00cdc50 con 0x7f7cac0051c0 2024-04-15T15:19:52.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.112+0000 7f7cb67fc640 1 -- 172.21.15.105:0/2608607412 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f7cc014b990 con 0x7f7cac0051c0 2024-04-15T15:19:52.114 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-04-15T15:19:52.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.113+0000 7f7cc7ab1640 1 -- 172.21.15.105:0/2608607412 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f7ca406c8f0 msgr2=0x7f7ca406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:52.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.113+0000 7f7cc7ab1640 1 --2- 172.21.15.105:0/2608607412 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f7ca406c8f0 0x7f7ca406edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0000c00 tx=0x7f7cb0009040 comp rx=0 tx=0).stop 2024-04-15T15:19:52.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.113+0000 7f7cc7ab1640 1 -- 172.21.15.105:0/2608607412 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7cac0051c0 msgr2=0x7f7cc014b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:52.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.113+0000 7f7cc7ab1640 1 --2- 172.21.15.105:0/2608607412 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7cac0051c0 0x7f7cc014b450 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc009f90 tx=0x7f7cbc00f580 comp rx=0 tx=0).stop 2024-04-15T15:19:52.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.113+0000 7f7cc7ab1640 1 -- 172.21.15.105:0/2608607412 shutdown_connections 2024-04-15T15:19:52.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.113+0000 7f7cc7ab1640 1 --2- 172.21.15.105:0/2608607412 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7cc00cd040 0x7f7cc014bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:52.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.113+0000 7f7cc7ab1640 1 --2- 172.21.15.105:0/2608607412 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f7ca406c8f0 0x7f7ca406edb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:52.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.113+0000 7f7cc7ab1640 1 --2- 172.21.15.105:0/2608607412 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7cc00c7cf0 0x7f7cc00ccb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:52.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.113+0000 7f7cc7ab1640 1 --2- 172.21.15.105:0/2608607412 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7cac0051c0 0x7f7cc014b450 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:52.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.113+0000 7f7cc7ab1640 1 -- 172.21.15.105:0/2608607412 >> 172.21.15.105:0/2608607412 conn(0x7f7cc00b3030 msgr2=0x7f7cc00b0520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:52.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.113+0000 7f7cc7ab1640 1 -- 172.21.15.105:0/2608607412 shutdown_connections 2024-04-15T15:19:52.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.113+0000 7f7cc7ab1640 1 -- 172.21.15.105:0/2608607412 wait complete. 2024-04-15T15:19:52.127 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-15T15:19:52.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.231+0000 7f393919d640 1 Processor -- start 2024-04-15T15:19:52.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.231+0000 7f393919d640 1 -- start start 2024-04-15T15:19:52.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.232+0000 7f393919d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f393415ea10 0x7f39340b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:52.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.232+0000 7f393919d640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f393415edf0 con 0x7f393406c0d0 2024-04-15T15:19:52.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.232+0000 7f393919d640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39340b6490 con 0x7f39340734b0 2024-04-15T15:19:52.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.232+0000 7f393919d640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f39340b0ae0 con 0x7f393415ea10 2024-04-15T15:19:52.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.232+0000 7f3932d9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f393415ea10 0x7f39340b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:52.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.232+0000 7f393359e640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f393406c0d0 0x7f39340b1250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:57374/0 (socket says 172.21.15.105:57374) 2024-04-15T15:19:52.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.232+0000 7f3932d9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f393415ea10 0x7f39340b2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46806/0 (socket says 172.21.15.105:46806) 2024-04-15T15:19:52.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.232+0000 7f393359e640 1 -- 172.21.15.105:0/1086447506 learned_addr learned my addr 172.21.15.105:0/1086447506 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:19:52.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.232+0000 7f3932d9d640 1 -- 172.21.15.105:0/1086447506 >> v1:172.21.15.154:6789/0 conn(0x7f39340734b0 legacy=0x7f393406b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:52.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.232+0000 7f3932d9d640 1 -- 172.21.15.105:0/1086447506 >> v1:172.21.15.105:6789/0 conn(0x7f393406c0d0 legacy=0x7f39340b1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:52.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.232+0000 7f3932d9d640 1 -- 172.21.15.105:0/1086447506 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39340b6770 con 0x7f393415ea10 2024-04-15T15:19:52.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.233+0000 7f3932d9d640 1 --2- 172.21.15.105:0/1086447506 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f393415ea10 0x7f39340b2a80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f39340b0c60 tx=0x7f392402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2a6dcffa755f882 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:52.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.233+0000 7f3931d9b640 1 -- 172.21.15.105:0/1086447506 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3924034070 con 0x7f393415ea10 2024-04-15T15:19:52.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.233+0000 7f3931d9b640 1 -- 172.21.15.105:0/1086447506 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3924030610 con 0x7f393415ea10 2024-04-15T15:19:52.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.233+0000 7f3931d9b640 1 -- 172.21.15.105:0/1086447506 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39240309e0 con 0x7f393415ea10 2024-04-15T15:19:52.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.234+0000 7f393919d640 1 -- 172.21.15.105:0/1086447506 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f393415ea10 msgr2=0x7f39340b2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:52.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.234+0000 7f393919d640 1 --2- 172.21.15.105:0/1086447506 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f393415ea10 0x7f39340b2a80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f39340b0c60 tx=0x7f392402b960 comp rx=0 tx=0).stop 2024-04-15T15:19:52.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.234+0000 7f393919d640 1 -- 172.21.15.105:0/1086447506 shutdown_connections 2024-04-15T15:19:52.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.234+0000 7f393919d640 1 --2- 172.21.15.105:0/1086447506 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f393415ea10 0x7f39340b2a80 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:52.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.234+0000 7f393919d640 1 -- 172.21.15.105:0/1086447506 >> 172.21.15.105:0/1086447506 conn(0x7f39340b6b10 msgr2=0x7f393409aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:52.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.234+0000 7f393919d640 1 -- 172.21.15.105:0/1086447506 shutdown_connections 2024-04-15T15:19:52.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.234+0000 7f393919d640 1 -- 172.21.15.105:0/1086447506 wait complete. 2024-04-15T15:19:52.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.235+0000 7f393919d640 1 Processor -- start 2024-04-15T15:19:52.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.235+0000 7f393919d640 1 -- start start 2024-04-15T15:19:52.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.236+0000 7f393919d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f393406c0d0 0x7f393415a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:52.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.236+0000 7f393919d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39340734b0 0x7f3934142dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:52.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.236+0000 7f393919d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f393415ea10 0x7f3934143310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:52.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.236+0000 7f393919d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f39340b6140 con 0x7f393415ea10 2024-04-15T15:19:52.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.236+0000 7f393919d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f39340b0dc0 con 0x7f39340734b0 2024-04-15T15:19:52.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.236+0000 7f393919d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f39340ae3b0 con 0x7f393406c0d0 2024-04-15T15:19:52.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.236+0000 7f393359e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f393415ea10 0x7f3934143310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:52.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.236+0000 7f393359e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f393415ea10 0x7f3934143310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46808/0 (socket says 172.21.15.105:46808) 2024-04-15T15:19:52.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.236+0000 7f393359e640 1 -- 172.21.15.105:0/1186064516 learned_addr learned my addr 172.21.15.105:0/1186064516 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:52.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.237+0000 7f3932d9d640 1 --2- 172.21.15.105:0/1186064516 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f393406c0d0 0x7f393415a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:52.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.237+0000 7f393359e640 1 -- 172.21.15.105:0/1186064516 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f393406c0d0 msgr2=0x7f393415a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:52.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.237+0000 7f393259c640 1 --2- 172.21.15.105:0/1186064516 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39340734b0 0x7f3934142dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:52.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.237+0000 7f393359e640 1 --2- 172.21.15.105:0/1186064516 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f393406c0d0 0x7f393415a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:52.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.237+0000 7f393359e640 1 -- 172.21.15.105:0/1186064516 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39340734b0 msgr2=0x7f3934142dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:52.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.237+0000 7f393359e640 1 --2- 172.21.15.105:0/1186064516 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39340734b0 0x7f3934142dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:52.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.237+0000 7f393359e640 1 -- 172.21.15.105:0/1186064516 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3934152690 con 0x7f393415ea10 2024-04-15T15:19:52.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.237+0000 7f3932d9d640 1 --2- 172.21.15.105:0/1186064516 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f393406c0d0 0x7f393415a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:52.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.237+0000 7f393259c640 1 --2- 172.21.15.105:0/1186064516 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39340734b0 0x7f3934142dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:52.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.237+0000 7f393359e640 1 --2- 172.21.15.105:0/1186064516 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f393415ea10 0x7f3934143310 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f392c00ca20 tx=0x7f392c00cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:52.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.237+0000 7f391b7fe640 1 -- 172.21.15.105:0/1186064516 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392c017070 con 0x7f393415ea10 2024-04-15T15:19:52.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.237+0000 7f391b7fe640 1 -- 172.21.15.105:0/1186064516 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f392c00ab70 con 0x7f393415ea10 2024-04-15T15:19:52.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.237+0000 7f393919d640 1 -- 172.21.15.105:0/1186064516 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3934152970 con 0x7f393415ea10 2024-04-15T15:19:52.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.237+0000 7f391b7fe640 1 -- 172.21.15.105:0/1186064516 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392c012400 con 0x7f393415ea10 2024-04-15T15:19:52.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.237+0000 7f393919d640 1 -- 172.21.15.105:0/1186064516 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3934159d40 con 0x7f393415ea10 2024-04-15T15:19:52.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.238+0000 7f393919d640 1 -- 172.21.15.105:0/1186064516 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3934159850 con 0x7f393415ea10 2024-04-15T15:19:52.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.238+0000 7f391b7fe640 1 -- 172.21.15.105:0/1186064516 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f392c01a070 con 0x7f393415ea10 2024-04-15T15:19:52.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.239+0000 7f391b7fe640 1 --2- 172.21.15.105:0/1186064516 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f392006c8f0 0x7f392006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:52.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.239+0000 7f391b7fe640 1 -- 172.21.15.105:0/1186064516 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f392c08b6b0 con 0x7f393415ea10 2024-04-15T15:19:52.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.239+0000 7f3932d9d640 1 --2- 172.21.15.105:0/1186064516 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f392006c8f0 0x7f392006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:52.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.239+0000 7f3932d9d640 1 --2- 172.21.15.105:0/1186064516 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f392006c8f0 0x7f392006edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3924036000 tx=0x7f392403b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:52.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.243+0000 7f391b7fe640 1 -- 172.21.15.105:0/1186064516 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f392c059490 con 0x7f393415ea10 2024-04-15T15:19:52.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.418+0000 7f393919d640 1 -- 172.21.15.105:0/1186064516 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3934068ab0 con 0x7f393415ea10 2024-04-15T15:19:52.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.418+0000 7f391b7fe640 1 -- 172.21.15.105:0/1186064516 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f392c05d790 con 0x7f393415ea10 2024-04-15T15:19:52.421 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-04-15T15:19:52.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.421+0000 7f393919d640 1 -- 172.21.15.105:0/1186064516 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f392006c8f0 msgr2=0x7f392006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:52.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.421+0000 7f393919d640 1 --2- 172.21.15.105:0/1186064516 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f392006c8f0 0x7f392006edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3924036000 tx=0x7f392403b040 comp rx=0 tx=0).stop 2024-04-15T15:19:52.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.421+0000 7f393919d640 1 -- 172.21.15.105:0/1186064516 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f393415ea10 msgr2=0x7f3934143310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:52.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.421+0000 7f393919d640 1 --2- 172.21.15.105:0/1186064516 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f393415ea10 0x7f3934143310 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f392c00ca20 tx=0x7f392c00cee0 comp rx=0 tx=0).stop 2024-04-15T15:19:52.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.421+0000 7f393919d640 1 -- 172.21.15.105:0/1186064516 shutdown_connections 2024-04-15T15:19:52.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.421+0000 7f393919d640 1 --2- 172.21.15.105:0/1186064516 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f392006c8f0 0x7f392006edb0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:52.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.421+0000 7f393919d640 1 --2- 172.21.15.105:0/1186064516 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f393415ea10 0x7f3934143310 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:52.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.421+0000 7f393919d640 1 --2- 172.21.15.105:0/1186064516 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39340734b0 0x7f3934142dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:52.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.421+0000 7f393919d640 1 --2- 172.21.15.105:0/1186064516 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f393406c0d0 0x7f393415a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:52.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.421+0000 7f393919d640 1 -- 172.21.15.105:0/1186064516 >> 172.21.15.105:0/1186064516 conn(0x7f39340b6b10 msgr2=0x7f39340b1250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:52.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.421+0000 7f393919d640 1 -- 172.21.15.105:0/1186064516 shutdown_connections 2024-04-15T15:19:52.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.421+0000 7f393919d640 1 -- 172.21.15.105:0/1186064516 wait complete. 2024-04-15T15:19:52.445 INFO:tasks.cephfs_test_runner:Starting test: test_quiesce_blocked (tasks.cephfs.test_quiesce.TestQuiesce) That a file with ceph.quiesce.block is not quiesced. 2024-04-15T15:19:52.445 INFO:tasks.cephfs_test_runner:test_quiesce_blocked (tasks.cephfs.test_quiesce.TestQuiesce) 2024-04-15T15:19:52.445 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:19:52.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.546+0000 7f16f84c1640 1 Processor -- start 2024-04-15T15:19:52.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.546+0000 7f16f84c1640 1 -- start start 2024-04-15T15:19:52.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.547+0000 7f16f84c1640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f16f00b3c80 0x7f16f0169ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:52.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.547+0000 7f16f84c1640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16f00c8270 con 0x7f16f00d0540 2024-04-15T15:19:52.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.547+0000 7f16f84c1640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16f00d0920 con 0x7f16f006c0d0 2024-04-15T15:19:52.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.547+0000 7f16f84c1640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f16f00c7cb0 con 0x7f16f00b3c80 2024-04-15T15:19:52.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.547+0000 7f16f625d640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f16f00d0540 0x7f16f006b9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:57386/0 (socket says 172.21.15.105:57386) 2024-04-15T15:19:52.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.547+0000 7f16f625d640 1 -- 172.21.15.105:0/2323980484 learned_addr learned my addr 172.21.15.105:0/2323980484 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:19:52.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.547+0000 7f16f525b640 1 -- 172.21.15.105:0/2323980484 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3406840707 0 0) 0x7f16f00c8270 con 0x7f16f00d0540 2024-04-15T15:19:52.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.547+0000 7f16f6a5e640 1 --2- 172.21.15.105:0/2323980484 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f16f00b3c80 0x7f16f0169ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:52.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.547+0000 7f16f525b640 1 -- 172.21.15.105:0/2323980484 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f16dc003520 con 0x7f16f00d0540 2024-04-15T15:19:52.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.547+0000 7f16f525b640 1 -- 172.21.15.105:0/2323980484 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 409146932 0 0) 0x7f16dc003520 con 0x7f16f00d0540 2024-04-15T15:19:52.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.547+0000 7f16f525b640 1 -- 172.21.15.105:0/2323980484 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f16f00b3c80 msgr2=0x7f16f0169ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:52.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.547+0000 7f16f525b640 1 --2- 172.21.15.105:0/2323980484 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f16f00b3c80 0x7f16f0169ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:52.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.547+0000 7f16f525b640 1 -- 172.21.15.105:0/2323980484 >> v1:172.21.15.154:6789/0 conn(0x7f16f006c0d0 legacy=0x7f16f0166570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:52.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.547+0000 7f16f525b640 1 -- 172.21.15.105:0/2323980484 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16f00c7f90 con 0x7f16f00d0540 2024-04-15T15:19:52.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.547+0000 7f16f525b640 1 -- 172.21.15.105:0/2323980484 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7f16ec0019f0 con 0x7f16f00d0540 2024-04-15T15:19:52.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.547+0000 7f16f525b640 1 -- 172.21.15.105:0/2323980484 >> v1:172.21.15.105:6789/0 conn(0x7f16f00d0540 legacy=0x7f16f006b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:52.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.547+0000 7f16f525b640 1 --2- 172.21.15.105:0/2323980484 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f16dc0039e0 0x7f16dc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:52.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.547+0000 7f16f525b640 1 --2- 172.21.15.105:0/2323980484 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f16dc004800 0x7f16dc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:52.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.547+0000 7f16f525b640 1 --2- 172.21.15.105:0/2323980484 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f16f00b3c80 0x7f16dc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:52.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.548+0000 7f16f525b640 1 -- 172.21.15.105:0/2323980484 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f16f00c7cb0 con 0x7f16dc0039e0 2024-04-15T15:19:52.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.548+0000 7f16f525b640 1 -- 172.21.15.105:0/2323980484 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f16dc00baa0 con 0x7f16dc004800 2024-04-15T15:19:52.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.548+0000 7f16f525b640 1 -- 172.21.15.105:0/2323980484 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f16dc00bca0 con 0x7f16f00b3c80 2024-04-15T15:19:52.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.548+0000 7f16f625d640 1 --2- 172.21.15.105:0/2323980484 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f16dc0039e0 0x7f16dc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:52.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.548+0000 7f16f6a5e640 1 --2- 172.21.15.105:0/2323980484 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f16f00b3c80 0x7f16dc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:52.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.548+0000 7f16f5a5c640 1 --2- 172.21.15.105:0/2323980484 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f16dc004800 0x7f16dc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:52.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.548+0000 7f16f625d640 1 -- 172.21.15.105:0/2323980484 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f16f00b3c80 msgr2=0x7f16dc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:52.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.548+0000 7f16f625d640 1 --2- 172.21.15.105:0/2323980484 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f16f00b3c80 0x7f16dc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:52.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.548+0000 7f16f625d640 1 -- 172.21.15.105:0/2323980484 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f16dc004800 msgr2=0x7f16dc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:52.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.548+0000 7f16f625d640 1 --2- 172.21.15.105:0/2323980484 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f16dc004800 0x7f16dc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:52.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.548+0000 7f16f625d640 1 -- 172.21.15.105:0/2323980484 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16dc003520 con 0x7f16dc0039e0 2024-04-15T15:19:52.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.548+0000 7f16f5a5c640 1 --2- 172.21.15.105:0/2323980484 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f16dc004800 0x7f16dc004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:19:52.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.549+0000 7f16f625d640 1 --2- 172.21.15.105:0/2323980484 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f16dc0039e0 0x7f16dc003e00 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f16ec002230 tx=0x7f16dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6d1f3c988740196 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:52.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.549+0000 7f16f525b640 1 -- 172.21.15.105:0/2323980484 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ec00a4a0 con 0x7f16dc0039e0 2024-04-15T15:19:52.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.549+0000 7f16f525b640 1 -- 172.21.15.105:0/2323980484 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16ec034650 con 0x7f16dc0039e0 2024-04-15T15:19:52.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.549+0000 7f16f525b640 1 -- 172.21.15.105:0/2323980484 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ec034a40 con 0x7f16dc0039e0 2024-04-15T15:19:52.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.549+0000 7f16f84c1640 1 -- 172.21.15.105:0/2323980484 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f16dc0039e0 msgr2=0x7f16dc003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:52.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.549+0000 7f16f84c1640 1 --2- 172.21.15.105:0/2323980484 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f16dc0039e0 0x7f16dc003e00 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f16ec002230 tx=0x7f16dc005830 comp rx=0 tx=0).stop 2024-04-15T15:19:52.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.549+0000 7f16f6a5e640 1 -- 172.21.15.105:0/2323980484 reap_dead start 2024-04-15T15:19:52.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.549+0000 7f16f84c1640 1 -- 172.21.15.105:0/2323980484 shutdown_connections 2024-04-15T15:19:52.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.549+0000 7f16f84c1640 1 -- 172.21.15.105:0/2323980484 >> 172.21.15.105:0/2323980484 conn(0x7f16f009aa40 msgr2=0x7f16f00b30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:52.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.549+0000 7f16f84c1640 1 -- 172.21.15.105:0/2323980484 shutdown_connections 2024-04-15T15:19:52.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.549+0000 7f16f84c1640 1 -- 172.21.15.105:0/2323980484 wait complete. 2024-04-15T15:19:52.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.550+0000 7f16f84c1640 1 Processor -- start 2024-04-15T15:19:52.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.551+0000 7f16f84c1640 1 -- start start 2024-04-15T15:19:52.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.551+0000 7f16f84c1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f16dc0039e0 0x7f16f014d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:52.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.551+0000 7f16f84c1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f16f009b490 0x7f16f009b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:52.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.551+0000 7f16f84c1640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f16f00aefb0 0x7f16f00af430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:52.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.551+0000 7f16f84c1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f16f0070e40 con 0x7f16dc0039e0 2024-04-15T15:19:52.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.551+0000 7f16f84c1640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f16f006f180 con 0x7f16f009b490 2024-04-15T15:19:52.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.551+0000 7f16f84c1640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f16f0070fc0 con 0x7f16f00aefb0 2024-04-15T15:19:52.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.551+0000 7f16f625d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f16dc0039e0 0x7f16f014d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:52.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.551+0000 7f16f625d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f16dc0039e0 0x7f16f014d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46816/0 (socket says 172.21.15.105:46816) 2024-04-15T15:19:52.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.551+0000 7f16f625d640 1 -- 172.21.15.105:0/2359122484 learned_addr learned my addr 172.21.15.105:0/2359122484 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:52.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.552+0000 7f16f5a5c640 1 --2- 172.21.15.105:0/2359122484 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f16f009b490 0x7f16f009b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:52.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.552+0000 7f16f625d640 1 -- 172.21.15.105:0/2359122484 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f16f00aefb0 msgr2=0x7f16f00af430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:52.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.552+0000 7f16f625d640 1 --2- 172.21.15.105:0/2359122484 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f16f00aefb0 0x7f16f00af430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:52.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.552+0000 7f16f625d640 1 -- 172.21.15.105:0/2359122484 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f16f009b490 msgr2=0x7f16f009b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:52.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.552+0000 7f16f625d640 1 --2- 172.21.15.105:0/2359122484 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f16f009b490 0x7f16f009b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:52.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.552+0000 7f16f625d640 1 -- 172.21.15.105:0/2359122484 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16f009e250 con 0x7f16dc0039e0 2024-04-15T15:19:52.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.552+0000 7f16f625d640 1 --2- 172.21.15.105:0/2359122484 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f16dc0039e0 0x7f16f014d1c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f16ec00a940 tx=0x7f16ec001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:52.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.552+0000 7f16e6ffd640 1 -- 172.21.15.105:0/2359122484 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ec043020 con 0x7f16dc0039e0 2024-04-15T15:19:52.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.552+0000 7f16e6ffd640 1 -- 172.21.15.105:0/2359122484 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16ec00a480 con 0x7f16dc0039e0 2024-04-15T15:19:52.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.552+0000 7f16f84c1640 1 -- 172.21.15.105:0/2359122484 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16f009e530 con 0x7f16dc0039e0 2024-04-15T15:19:52.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.552+0000 7f16e6ffd640 1 -- 172.21.15.105:0/2359122484 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ec034a00 con 0x7f16dc0039e0 2024-04-15T15:19:52.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.552+0000 7f16f84c1640 1 -- 172.21.15.105:0/2359122484 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16f009eaf0 con 0x7f16dc0039e0 2024-04-15T15:19:52.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.553+0000 7f16f84c1640 1 -- 172.21.15.105:0/2359122484 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16f0068a40 con 0x7f16dc0039e0 2024-04-15T15:19:52.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.554+0000 7f16e6ffd640 1 -- 172.21.15.105:0/2359122484 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f16ec046020 con 0x7f16dc0039e0 2024-04-15T15:19:52.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.554+0000 7f16e6ffd640 1 --2- 172.21.15.105:0/2359122484 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f16d406c8f0 0x7f16d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:52.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.554+0000 7f16e6ffd640 1 -- 172.21.15.105:0/2359122484 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f16ec0ae060 con 0x7f16dc0039e0 2024-04-15T15:19:52.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.554+0000 7f16f5a5c640 1 --2- 172.21.15.105:0/2359122484 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f16d406c8f0 0x7f16d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:52.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.554+0000 7f16f5a5c640 1 --2- 172.21.15.105:0/2359122484 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f16d406c8f0 0x7f16d406edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f16e0007fd0 tx=0x7f16e0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:52.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.558+0000 7f16e6ffd640 1 -- 172.21.15.105:0/2359122484 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f16ec04d030 con 0x7f16dc0039e0 2024-04-15T15:19:52.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.734+0000 7f16f84c1640 1 -- 172.21.15.105:0/2359122484 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f16f006b9c0 con 0x7f16dc0039e0 2024-04-15T15:19:52.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.735+0000 7f16e6ffd640 1 -- 172.21.15.105:0/2359122484 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16653 (secure 0 0 0) 0x7f16ec07c400 con 0x7f16dc0039e0 2024-04-15T15:19:52.737 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:19:52.737 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":20,"fsid":"09e4d2a4-a324-4793-af77-644b25e520e1","created":"2024-04-15T15:19:01.524543+0000","modified":"2024-04-15T15:19:27.414890+0000","last_up_change":"2024-04-15T15:19:12.825815+0000","last_in_change":"2024-04-15T15:19:03.788126+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:19:13.748585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:19:23.789287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:19:24.722250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6f6292b-2d7e-4aa0-9d1c-6ddd0f97287c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6801","nonce":2873040054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6803","nonce":2873040054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6807","nonce":2873040054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6805","nonce":2873040054}]},"public_addr":"172.21.15.105:6801/2873040054","cluster_addr":"172.21.15.105:6803/2873040054","heartbeat_back_addr":"172.21.15.105:6807/2873040054","heartbeat_front_addr":"172.21.15.105:6805/2873040054","state":["exists","up"]},{"osd":1,"uuid":"8900966a-b393-41e3-8f93-e5a43bd23622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6818","nonce":2494994479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6821","nonce":2494994479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6828","nonce":2494994479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6823","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6825","nonce":2494994479}]},"public_addr":"172.21.15.105:6818/2494994479","cluster_addr":"172.21.15.105:6821/2494994479","heartbeat_back_addr":"172.21.15.105:6828/2494994479","heartbeat_front_addr":"172.21.15.105:6825/2494994479","state":["exists","up"]},{"osd":2,"uuid":"81a5dc7a-9325-4ff5-941d-a2d5fba21f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6809","nonce":3798408699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6811","nonce":3798408699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6815","nonce":3798408699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6813","nonce":3798408699}]},"public_addr":"172.21.15.105:6809/3798408699","cluster_addr":"172.21.15.105:6811/3798408699","heartbeat_back_addr":"172.21.15.105:6815/3798408699","heartbeat_front_addr":"172.21.15.105:6813/3798408699","state":["exists","up"]},{"osd":3,"uuid":"df766ed2-5c71-4805-96f4-3d4970edc960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6819","nonce":1920720186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6824","nonce":1920720186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6831","nonce":1920720186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6827","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6829","nonce":1920720186}]},"public_addr":"172.21.15.105:6819/1920720186","cluster_addr":"172.21.15.105:6824/1920720186","heartbeat_back_addr":"172.21.15.105:6831/1920720186","heartbeat_front_addr":"172.21.15.105:6829/1920720186","state":["exists","up"]},{"osd":4,"uuid":"e1039e03-28b8-49df-b951-c91930b93c9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6810","nonce":3516494456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6813","nonce":3516494456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6820","nonce":3516494456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6817","nonce":3516494456}]},"public_addr":"172.21.15.154:6810/3516494456","cluster_addr":"172.21.15.154:6813/3516494456","heartbeat_back_addr":"172.21.15.154:6820/3516494456","heartbeat_front_addr":"172.21.15.154:6817/3516494456","state":["exists","up"]},{"osd":5,"uuid":"a9f5eaf1-66ff-405a-a9e4-ec3340812a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6801","nonce":540367751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6803","nonce":540367751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6807","nonce":540367751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6805","nonce":540367751}]},"public_addr":"172.21.15.154:6801/540367751","cluster_addr":"172.21.15.154:6803/540367751","heartbeat_back_addr":"172.21.15.154:6807/540367751","heartbeat_front_addr":"172.21.15.154:6805/540367751","state":["exists","up"]},{"osd":6,"uuid":"267d02ea-675f-428f-aae8-e0375e4d4feb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6825","nonce":1550829005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6827","nonce":1550829005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6831","nonce":1550829005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6829","nonce":1550829005}]},"public_addr":"172.21.15.154:6825/1550829005","cluster_addr":"172.21.15.154:6827/1550829005","heartbeat_back_addr":"172.21.15.154:6831/1550829005","heartbeat_front_addr":"172.21.15.154:6829/1550829005","state":["exists","up"]},{"osd":7,"uuid":"759ac6cc-896a-423c-885f-9fe90d8dcb14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6809","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6811","nonce":3920089903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6816","nonce":3920089903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6823","nonce":3920089903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6819","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6821","nonce":3920089903}]},"public_addr":"172.21.15.154:6811/3920089903","cluster_addr":"172.21.15.154:6816/3920089903","heartbeat_back_addr":"172.21.15.154:6823/3920089903","heartbeat_front_addr":"172.21.15.154:6821/3920089903","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.949509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.984072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.907278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.947391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.975447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.998896+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:09.028739+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.894119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:19:52.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.737+0000 7f16f84c1640 1 -- 172.21.15.105:0/2359122484 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f16d406c8f0 msgr2=0x7f16d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:52.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.737+0000 7f16f84c1640 1 --2- 172.21.15.105:0/2359122484 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f16d406c8f0 0x7f16d406edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f16e0007fd0 tx=0x7f16e0009040 comp rx=0 tx=0).stop 2024-04-15T15:19:52.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.737+0000 7f16f84c1640 1 -- 172.21.15.105:0/2359122484 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f16dc0039e0 msgr2=0x7f16f014d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:52.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.737+0000 7f16f84c1640 1 --2- 172.21.15.105:0/2359122484 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f16dc0039e0 0x7f16f014d1c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f16ec00a940 tx=0x7f16ec001dc0 comp rx=0 tx=0).stop 2024-04-15T15:19:52.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.737+0000 7f16f84c1640 1 -- 172.21.15.105:0/2359122484 shutdown_connections 2024-04-15T15:19:52.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.737+0000 7f16f84c1640 1 --2- 172.21.15.105:0/2359122484 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f16f00aefb0 0x7f16f00af430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:52.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.737+0000 7f16f84c1640 1 --2- 172.21.15.105:0/2359122484 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f16d406c8f0 0x7f16d406edb0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:52.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.737+0000 7f16f84c1640 1 --2- 172.21.15.105:0/2359122484 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f16f009b490 0x7f16f009b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:52.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.737+0000 7f16f84c1640 1 --2- 172.21.15.105:0/2359122484 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f16dc0039e0 0x7f16f014d1c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:52.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.737+0000 7f16f84c1640 1 -- 172.21.15.105:0/2359122484 >> 172.21.15.105:0/2359122484 conn(0x7f16f009aa40 msgr2=0x7f16f00b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:52.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.737+0000 7f16f84c1640 1 -- 172.21.15.105:0/2359122484 shutdown_connections 2024-04-15T15:19:52.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.737+0000 7f16f84c1640 1 -- 172.21.15.105:0/2359122484 wait complete. 2024-04-15T15:19:52.750 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:19:13.748585+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-15T15:19:23.789287+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-15T15:19:24.722250+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:19:52.750 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:19:52.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.854+0000 7f8efb088640 1 Processor -- start 2024-04-15T15:19:52.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.854+0000 7f8efb088640 1 -- start start 2024-04-15T15:19:52.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.854+0000 7f8efb088640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8ef40b1730 0x7f8ef409b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:52.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.854+0000 7f8efb088640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ef40b1cf0 con 0x7f8ef40d1db0 2024-04-15T15:19:52.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.854+0000 7f8efb088640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ef406cac0 con 0x7f8ef415ee90 2024-04-15T15:19:52.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.854+0000 7f8efb088640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f8ef409b990 con 0x7f8ef40b1730 2024-04-15T15:19:52.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.855+0000 7f8eebfff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f8ef40d1db0 0x7f8ef4070530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:57392/0 (socket says 172.21.15.105:57392) 2024-04-15T15:19:52.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.855+0000 7f8eebfff640 1 -- 172.21.15.105:0/348122737 learned_addr learned my addr 172.21.15.105:0/348122737 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:19:52.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.855+0000 7f8ef9625640 1 --2- 172.21.15.105:0/348122737 >> v2:172.21.15.154:3300/0 conn(0x7f8ef40b1730 0x7f8ef409b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:52.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.855+0000 7f8eeb7fe640 1 -- 172.21.15.105:0/348122737 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3444986281 0 0) 0x7f8ef40b1cf0 con 0x7f8ef40d1db0 2024-04-15T15:19:52.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.855+0000 7f8eeb7fe640 1 -- 172.21.15.105:0/348122737 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8edc003520 con 0x7f8ef40d1db0 2024-04-15T15:19:52.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.855+0000 7f8eeb7fe640 1 -- 172.21.15.105:0/348122737 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 610911203 0 0) 0x7f8ef406cac0 con 0x7f8ef415ee90 2024-04-15T15:19:52.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.855+0000 7f8eeb7fe640 1 -- 172.21.15.105:0/348122737 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8ef40b1cf0 con 0x7f8ef415ee90 2024-04-15T15:19:52.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.855+0000 7f8eeb7fe640 1 -- 172.21.15.105:0/348122737 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3590706956 0 0) 0x7f8edc003520 con 0x7f8ef40d1db0 2024-04-15T15:19:52.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.855+0000 7f8ef9625640 1 -- 172.21.15.105:0/348122737 >> v1:172.21.15.154:6789/0 conn(0x7f8ef415ee90 legacy=0x7f8ef40d16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:52.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.855+0000 7f8ef9625640 1 -- 172.21.15.105:0/348122737 >> v1:172.21.15.105:6789/0 conn(0x7f8ef40d1db0 legacy=0x7f8ef4070530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:52.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.855+0000 7f8ef9625640 1 -- 172.21.15.105:0/348122737 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ef406cda0 con 0x7f8ef40b1730 2024-04-15T15:19:52.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.856+0000 7f8ef9625640 1 --2- 172.21.15.105:0/348122737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ef40b1730 0x7f8ef409b450 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8eec009d90 tx=0x7f8eec02be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a04c50b3635d9e5b server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:52.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.856+0000 7f8eeb7fe640 1 -- 172.21.15.105:0/348122737 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eec039070 con 0x7f8ef40b1730 2024-04-15T15:19:52.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.856+0000 7f8eeb7fe640 1 -- 172.21.15.105:0/348122737 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8eec02c940 con 0x7f8ef40b1730 2024-04-15T15:19:52.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.856+0000 7f8eeb7fe640 1 -- 172.21.15.105:0/348122737 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee4001670 con 0x7f8ef40b1730 2024-04-15T15:19:52.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.857+0000 7f8efb088640 1 -- 172.21.15.105:0/348122737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ef40b1730 msgr2=0x7f8ef409b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:52.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.857+0000 7f8efb088640 1 --2- 172.21.15.105:0/348122737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ef40b1730 0x7f8ef409b450 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8eec009d90 tx=0x7f8eec02be10 comp rx=0 tx=0).stop 2024-04-15T15:19:52.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.857+0000 7f8efb088640 1 -- 172.21.15.105:0/348122737 shutdown_connections 2024-04-15T15:19:52.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.857+0000 7f8efb088640 1 --2- 172.21.15.105:0/348122737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ef40b1730 0x7f8ef409b450 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:52.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.857+0000 7f8efb088640 1 -- 172.21.15.105:0/348122737 >> 172.21.15.105:0/348122737 conn(0x7f8ef40b4070 msgr2=0x7f8ef40b4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:52.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.857+0000 7f8efb088640 1 -- 172.21.15.105:0/348122737 shutdown_connections 2024-04-15T15:19:52.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.857+0000 7f8efb088640 1 -- 172.21.15.105:0/348122737 wait complete. 2024-04-15T15:19:52.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.858+0000 7f8efb088640 1 Processor -- start 2024-04-15T15:19:52.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.858+0000 7f8efb088640 1 -- start start 2024-04-15T15:19:52.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.859+0000 7f8efb088640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ef40b1730 0x7f8ef40c8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:52.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.859+0000 7f8efb088640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ef40d1db0 0x7f8ef415c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:52.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.859+0000 7f8efb088640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ef415ee90 0x7f8ef41513e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:52.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.859+0000 7f8efb088640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef406d880 con 0x7f8ef415ee90 2024-04-15T15:19:52.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.859+0000 7f8efb088640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef406d3f0 con 0x7f8ef40b1730 2024-04-15T15:19:52.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.859+0000 7f8efb088640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f8ef406da00 con 0x7f8ef40d1db0 2024-04-15T15:19:52.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.859+0000 7f8ef9625640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ef415ee90 0x7f8ef41513e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:52.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.859+0000 7f8ef9625640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ef415ee90 0x7f8ef41513e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46832/0 (socket says 172.21.15.105:46832) 2024-04-15T15:19:52.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.859+0000 7f8ef9625640 1 -- 172.21.15.105:0/2214984701 learned_addr learned my addr 172.21.15.105:0/2214984701 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:52.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.859+0000 7f8eebfff640 1 --2- 172.21.15.105:0/2214984701 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ef40d1db0 0x7f8ef415c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:52.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.859+0000 7f8ef8e24640 1 --2- 172.21.15.105:0/2214984701 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ef40b1730 0x7f8ef40c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:52.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.859+0000 7f8ef9625640 1 -- 172.21.15.105:0/2214984701 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ef40d1db0 msgr2=0x7f8ef415c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:52.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.859+0000 7f8ef9625640 1 --2- 172.21.15.105:0/2214984701 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ef40d1db0 0x7f8ef415c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:52.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.859+0000 7f8ef9625640 1 -- 172.21.15.105:0/2214984701 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ef40b1730 msgr2=0x7f8ef40c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:52.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.859+0000 7f8ef9625640 1 --2- 172.21.15.105:0/2214984701 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ef40b1730 0x7f8ef40c8930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:52.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.859+0000 7f8ef9625640 1 -- 172.21.15.105:0/2214984701 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ef40c7430 con 0x7f8ef415ee90 2024-04-15T15:19:52.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.859+0000 7f8ef9625640 1 --2- 172.21.15.105:0/2214984701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ef415ee90 0x7f8ef41513e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8eec009d90 tx=0x7f8eec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:52.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.859+0000 7f8ee97fa640 1 -- 172.21.15.105:0/2214984701 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eec039050 con 0x7f8ef415ee90 2024-04-15T15:19:52.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.859+0000 7f8ee97fa640 1 -- 172.21.15.105:0/2214984701 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8eec03a040 con 0x7f8ef415ee90 2024-04-15T15:19:52.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.859+0000 7f8ee97fa640 1 -- 172.21.15.105:0/2214984701 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eec009450 con 0x7f8ef415ee90 2024-04-15T15:19:52.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.859+0000 7f8efb088640 1 -- 172.21.15.105:0/2214984701 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ef40c7710 con 0x7f8ef415ee90 2024-04-15T15:19:52.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.860+0000 7f8efb088640 1 -- 172.21.15.105:0/2214984701 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ef40c7cd0 con 0x7f8ef415ee90 2024-04-15T15:19:52.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.860+0000 7f8efb088640 1 -- 172.21.15.105:0/2214984701 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ef4069150 con 0x7f8ef415ee90 2024-04-15T15:19:52.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.861+0000 7f8ee97fa640 1 -- 172.21.15.105:0/2214984701 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8eec047020 con 0x7f8ef415ee90 2024-04-15T15:19:52.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.861+0000 7f8ee97fa640 1 --2- 172.21.15.105:0/2214984701 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f8ee006c8f0 0x7f8ee006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:52.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.861+0000 7f8ee97fa640 1 -- 172.21.15.105:0/2214984701 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8eec0ada40 con 0x7f8ef415ee90 2024-04-15T15:19:52.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.861+0000 7f8ef8e24640 1 --2- 172.21.15.105:0/2214984701 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f8ee006c8f0 0x7f8ee006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:52.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.862+0000 7f8ef8e24640 1 --2- 172.21.15.105:0/2214984701 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f8ee006c8f0 0x7f8ee006edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8ef00010e0 tx=0x7f8ef0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:52.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:52.866+0000 7f8ee97fa640 1 -- 172.21.15.105:0/2214984701 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8eec04e020 con 0x7f8ef415ee90 2024-04-15T15:19:53.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.041+0000 7f8efb088640 1 -- 172.21.15.105:0/2214984701 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8ef40d1b90 con 0x7f8ef415ee90 2024-04-15T15:19:53.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.042+0000 7f8ee97fa640 1 -- 172.21.15.105:0/2214984701 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f8eec07be70 con 0x7f8ef415ee90 2024-04-15T15:19:53.044 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-04-15T15:19:53.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.044+0000 7f8efb088640 1 -- 172.21.15.105:0/2214984701 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f8ee006c8f0 msgr2=0x7f8ee006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:53.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.044+0000 7f8efb088640 1 --2- 172.21.15.105:0/2214984701 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f8ee006c8f0 0x7f8ee006edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8ef00010e0 tx=0x7f8ef0009040 comp rx=0 tx=0).stop 2024-04-15T15:19:53.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.044+0000 7f8efb088640 1 -- 172.21.15.105:0/2214984701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ef415ee90 msgr2=0x7f8ef41513e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:53.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.044+0000 7f8efb088640 1 --2- 172.21.15.105:0/2214984701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ef415ee90 0x7f8ef41513e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8eec009d90 tx=0x7f8eec004a40 comp rx=0 tx=0).stop 2024-04-15T15:19:53.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.044+0000 7f8efb088640 1 -- 172.21.15.105:0/2214984701 shutdown_connections 2024-04-15T15:19:53.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.044+0000 7f8efb088640 1 --2- 172.21.15.105:0/2214984701 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ef40d1db0 0x7f8ef415c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:53.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.044+0000 7f8efb088640 1 --2- 172.21.15.105:0/2214984701 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f8ee006c8f0 0x7f8ee006edb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:53.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.044+0000 7f8efb088640 1 --2- 172.21.15.105:0/2214984701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ef415ee90 0x7f8ef41513e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:53.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.044+0000 7f8efb088640 1 --2- 172.21.15.105:0/2214984701 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ef40b1730 0x7f8ef40c8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:53.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.044+0000 7f8efb088640 1 -- 172.21.15.105:0/2214984701 >> 172.21.15.105:0/2214984701 conn(0x7f8ef40b4070 msgr2=0x7f8ef415def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:53.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.044+0000 7f8efb088640 1 -- 172.21.15.105:0/2214984701 shutdown_connections 2024-04-15T15:19:53.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.044+0000 7f8efb088640 1 -- 172.21.15.105:0/2214984701 wait complete. 2024-04-15T15:19:53.057 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-15T15:19:53.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.161+0000 7f35537e9640 1 Processor -- start 2024-04-15T15:19:53.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.161+0000 7f35537e9640 1 -- start start 2024-04-15T15:19:53.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.162+0000 7f35537e9640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f354c15ff10 0x7f354c0ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:53.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.162+0000 7f35537e9640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f354c09b150 0x7f354c0b1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:53.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.162+0000 7f35537e9640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f354c0d1730 con 0x7f354c0ce9c0 2024-04-15T15:19:53.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.162+0000 7f35537e9640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f354c0cd1e0 con 0x7f354c09b150 2024-04-15T15:19:53.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.162+0000 7f35537e9640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f354c0cd480 con 0x7f354c15ff10 2024-04-15T15:19:53.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.162+0000 7f3551d86640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f354c09b150 0x7f354c0b1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:53.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.162+0000 7f3551d86640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f354c09b150 0x7f354c0b1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46848/0 (socket says 172.21.15.105:46848) 2024-04-15T15:19:53.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.162+0000 7f3551d86640 1 -- 172.21.15.105:0/154265169 learned_addr learned my addr 172.21.15.105:0/154265169 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:53.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.162+0000 7f3551585640 1 --2- 172.21.15.105:0/154265169 >> v2:172.21.15.154:3300/0 conn(0x7f354c15ff10 0x7f354c0ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:53.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.163+0000 7f3551d86640 1 -- 172.21.15.105:0/154265169 >> v2:172.21.15.154:3300/0 conn(0x7f354c15ff10 msgr2=0x7f354c0ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:53.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.163+0000 7f3551d86640 1 --2- 172.21.15.105:0/154265169 >> v2:172.21.15.154:3300/0 conn(0x7f354c15ff10 0x7f354c0ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:53.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.163+0000 7f3551d86640 1 -- 172.21.15.105:0/154265169 >> v1:172.21.15.154:6789/0 conn(0x7f354c0ce9c0 legacy=0x7f354c09aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:53.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.163+0000 7f3551d86640 1 -- 172.21.15.105:0/154265169 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f354c0cd760 con 0x7f354c09b150 2024-04-15T15:19:53.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.163+0000 7f3551d86640 1 --2- 172.21.15.105:0/154265169 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f354c09b150 0x7f354c0b1fe0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f354c073380 tx=0x7f354402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69949e0b91d7465e server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:53.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.164+0000 7f3543fff640 1 -- 172.21.15.105:0/154265169 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3544034070 con 0x7f354c09b150 2024-04-15T15:19:53.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.164+0000 7f3543fff640 1 -- 172.21.15.105:0/154265169 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3544030860 con 0x7f354c09b150 2024-04-15T15:19:53.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.164+0000 7f3543fff640 1 -- 172.21.15.105:0/154265169 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3544030c30 con 0x7f354c09b150 2024-04-15T15:19:53.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.164+0000 7f35537e9640 1 -- 172.21.15.105:0/154265169 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f354c09b150 msgr2=0x7f354c0b1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:53.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.164+0000 7f35537e9640 1 --2- 172.21.15.105:0/154265169 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f354c09b150 0x7f354c0b1fe0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f354c073380 tx=0x7f354402b9a0 comp rx=0 tx=0).stop 2024-04-15T15:19:53.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.164+0000 7f35537e9640 1 -- 172.21.15.105:0/154265169 shutdown_connections 2024-04-15T15:19:53.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.164+0000 7f35537e9640 1 --2- 172.21.15.105:0/154265169 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f354c09b150 0x7f354c0b1fe0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:53.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.164+0000 7f35537e9640 1 --2- 172.21.15.105:0/154265169 >> v2:172.21.15.154:3300/0 conn(0x7f354c15ff10 0x7f354c0ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:53.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.164+0000 7f35537e9640 1 -- 172.21.15.105:0/154265169 >> 172.21.15.105:0/154265169 conn(0x7f354c0b4170 msgr2=0x7f354c0b4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:53.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.164+0000 7f35537e9640 1 -- 172.21.15.105:0/154265169 shutdown_connections 2024-04-15T15:19:53.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.164+0000 7f35537e9640 1 -- 172.21.15.105:0/154265169 wait complete. 2024-04-15T15:19:53.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.165+0000 7f35537e9640 1 Processor -- start 2024-04-15T15:19:53.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.165+0000 7f35537e9640 1 -- start start 2024-04-15T15:19:53.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.166+0000 7f35537e9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f354c09b150 0x7f354c1e5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:53.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.166+0000 7f35537e9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f354c0ce9c0 0x7f354c14f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:53.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.166+0000 7f35537e9640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f354c15ff10 0x7f354c1f2840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:53.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.166+0000 7f35537e9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f354c0d0580 con 0x7f354c09b150 2024-04-15T15:19:53.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.166+0000 7f35537e9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f354c0cdd20 con 0x7f354c0ce9c0 2024-04-15T15:19:53.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.166+0000 7f35537e9640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f354c0d0700 con 0x7f354c15ff10 2024-04-15T15:19:53.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.166+0000 7f3551585640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f354c09b150 0x7f354c1e5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:53.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.166+0000 7f3551585640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f354c09b150 0x7f354c1e5210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46850/0 (socket says 172.21.15.105:46850) 2024-04-15T15:19:53.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.166+0000 7f3551585640 1 -- 172.21.15.105:0/2511832236 learned_addr learned my addr 172.21.15.105:0/2511832236 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:53.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.166+0000 7f3551d86640 1 --2- 172.21.15.105:0/2511832236 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f354c15ff10 0x7f354c1f2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:53.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.166+0000 7f3550d84640 1 --2- 172.21.15.105:0/2511832236 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f354c0ce9c0 0x7f354c14f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:53.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.166+0000 7f3551585640 1 -- 172.21.15.105:0/2511832236 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f354c15ff10 msgr2=0x7f354c1f2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:53.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.166+0000 7f3551585640 1 --2- 172.21.15.105:0/2511832236 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f354c15ff10 0x7f354c1f2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:53.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.166+0000 7f3551585640 1 -- 172.21.15.105:0/2511832236 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f354c0ce9c0 msgr2=0x7f354c14f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:53.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.166+0000 7f3551585640 1 --2- 172.21.15.105:0/2511832236 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f354c0ce9c0 0x7f354c14f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:53.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.166+0000 7f3551585640 1 -- 172.21.15.105:0/2511832236 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f354c0b6220 con 0x7f354c09b150 2024-04-15T15:19:53.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.167+0000 7f3551585640 1 --2- 172.21.15.105:0/2511832236 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f354c09b150 0x7f354c1e5210 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3548007ef0 tx=0x7f354800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:53.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.167+0000 7f3541ffb640 1 -- 172.21.15.105:0/2511832236 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3548014620 con 0x7f354c09b150 2024-04-15T15:19:53.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.167+0000 7f3541ffb640 1 -- 172.21.15.105:0/2511832236 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3548019070 con 0x7f354c09b150 2024-04-15T15:19:53.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.167+0000 7f35537e9640 1 -- 172.21.15.105:0/2511832236 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f354c0b6500 con 0x7f354c09b150 2024-04-15T15:19:53.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.167+0000 7f3541ffb640 1 -- 172.21.15.105:0/2511832236 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3548014a00 con 0x7f354c09b150 2024-04-15T15:19:53.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.167+0000 7f35537e9640 1 -- 172.21.15.105:0/2511832236 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f354c0b6da0 con 0x7f354c09b150 2024-04-15T15:19:53.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.168+0000 7f35537e9640 1 -- 172.21.15.105:0/2511832236 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f354c068a40 con 0x7f354c09b150 2024-04-15T15:19:53.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.168+0000 7f3541ffb640 1 -- 172.21.15.105:0/2511832236 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3548014ba0 con 0x7f354c09b150 2024-04-15T15:19:53.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.169+0000 7f3541ffb640 1 --2- 172.21.15.105:0/2511832236 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f353006c8f0 0x7f353006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:53.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.169+0000 7f3541ffb640 1 -- 172.21.15.105:0/2511832236 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f354808c490 con 0x7f354c09b150 2024-04-15T15:19:53.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.169+0000 7f3550d84640 1 --2- 172.21.15.105:0/2511832236 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f353006c8f0 0x7f353006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:53.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.169+0000 7f3550d84640 1 --2- 172.21.15.105:0/2511832236 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f353006c8f0 0x7f353006edb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f353c007fd0 tx=0x7f353c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:53.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.173+0000 7f3541ffb640 1 -- 172.21.15.105:0/2511832236 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f354805a8c0 con 0x7f354c09b150 2024-04-15T15:19:53.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.347+0000 7f35537e9640 1 -- 172.21.15.105:0/2511832236 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f354c0b8600 con 0x7f354c09b150 2024-04-15T15:19:53.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.348+0000 7f3541ffb640 1 -- 172.21.15.105:0/2511832236 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f354805a1c0 con 0x7f354c09b150 2024-04-15T15:19:53.350 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-04-15T15:19:53.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.350+0000 7f35537e9640 1 -- 172.21.15.105:0/2511832236 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f353006c8f0 msgr2=0x7f353006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:53.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.350+0000 7f35537e9640 1 --2- 172.21.15.105:0/2511832236 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f353006c8f0 0x7f353006edb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f353c007fd0 tx=0x7f353c009040 comp rx=0 tx=0).stop 2024-04-15T15:19:53.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.350+0000 7f35537e9640 1 -- 172.21.15.105:0/2511832236 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f354c09b150 msgr2=0x7f354c1e5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:53.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.350+0000 7f35537e9640 1 --2- 172.21.15.105:0/2511832236 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f354c09b150 0x7f354c1e5210 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3548007ef0 tx=0x7f354800e4b0 comp rx=0 tx=0).stop 2024-04-15T15:19:53.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.350+0000 7f35537e9640 1 -- 172.21.15.105:0/2511832236 shutdown_connections 2024-04-15T15:19:53.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.350+0000 7f35537e9640 1 --2- 172.21.15.105:0/2511832236 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f354c15ff10 0x7f354c1f2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:53.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.350+0000 7f35537e9640 1 --2- 172.21.15.105:0/2511832236 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f353006c8f0 0x7f353006edb0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:53.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.350+0000 7f35537e9640 1 --2- 172.21.15.105:0/2511832236 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f354c0ce9c0 0x7f354c14f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:53.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.350+0000 7f35537e9640 1 --2- 172.21.15.105:0/2511832236 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f354c09b150 0x7f354c1e5210 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:53.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.350+0000 7f35537e9640 1 -- 172.21.15.105:0/2511832236 >> 172.21.15.105:0/2511832236 conn(0x7f354c0b4170 msgr2=0x7f354c0cf100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:53.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.350+0000 7f35537e9640 1 -- 172.21.15.105:0/2511832236 shutdown_connections 2024-04-15T15:19:53.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.350+0000 7f35537e9640 1 -- 172.21.15.105:0/2511832236 wait complete. 2024-04-15T15:19:53.362 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-15T15:19:53.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.466+0000 7fa2c9ed9640 1 Processor -- start 2024-04-15T15:19:53.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.466+0000 7fa2c9ed9640 1 -- start start 2024-04-15T15:19:53.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.467+0000 7fa2c9ed9640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa2c40b4890 0x7fa2c40b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:53.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.467+0000 7fa2c9ed9640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2c406ecd0 con 0x7fa2c415e9f0 2024-04-15T15:19:53.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.467+0000 7fa2c9ed9640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2c40aee60 con 0x7fa2c40b52d0 2024-04-15T15:19:53.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.467+0000 7fa2c9ed9640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fa2c40af140 con 0x7fa2c40b4890 2024-04-15T15:19:53.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.467+0000 7fa2c2ffd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa2c40b4890 0x7fa2c40b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:53.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.467+0000 7fa2c37fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fa2c415e9f0 0x7fa2c406e5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:57398/0 (socket says 172.21.15.105:57398) 2024-04-15T15:19:53.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.467+0000 7fa2c2ffd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa2c40b4890 0x7fa2c40b4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46864/0 (socket says 172.21.15.105:46864) 2024-04-15T15:19:53.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.467+0000 7fa2c37fe640 1 -- 172.21.15.105:0/3945964664 learned_addr learned my addr 172.21.15.105:0/3945964664 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:19:53.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.467+0000 7fa2c2ffd640 1 -- 172.21.15.105:0/3945964664 >> v1:172.21.15.154:6789/0 conn(0x7fa2c40b52d0 legacy=0x7fa2c40b56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:53.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.467+0000 7fa2c2ffd640 1 -- 172.21.15.105:0/3945964664 >> v1:172.21.15.105:6789/0 conn(0x7fa2c415e9f0 legacy=0x7fa2c406e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:53.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.467+0000 7fa2c2ffd640 1 -- 172.21.15.105:0/3945964664 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2c40af420 con 0x7fa2c40b4890 2024-04-15T15:19:53.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.467+0000 7fa2c27fc640 1 -- 172.21.15.105:0/3945964664 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2840666368 0 0) 0x7fa2c406ecd0 con 0x7fa2c415e9f0 2024-04-15T15:19:53.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.468+0000 7fa2c2ffd640 1 --2- 172.21.15.105:0/3945964664 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2c40b4890 0x7fa2c40b4d20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8009d90 tx=0x7fa2b802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc3877baed9aaf06 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:53.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.468+0000 7fa2c27fc640 1 -- 172.21.15.105:0/3945964664 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b8039070 con 0x7fa2c40b4890 2024-04-15T15:19:53.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.468+0000 7fa2c27fc640 1 -- 172.21.15.105:0/3945964664 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2b802cc60 con 0x7fa2c40b4890 2024-04-15T15:19:53.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.468+0000 7fa2c27fc640 1 -- 172.21.15.105:0/3945964664 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b8034660 con 0x7fa2c40b4890 2024-04-15T15:19:53.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.469+0000 7fa2c9ed9640 1 -- 172.21.15.105:0/3945964664 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2c40b4890 msgr2=0x7fa2c40b4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:53.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.469+0000 7fa2c9ed9640 1 --2- 172.21.15.105:0/3945964664 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2c40b4890 0x7fa2c40b4d20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8009d90 tx=0x7fa2b802beb0 comp rx=0 tx=0).stop 2024-04-15T15:19:53.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.469+0000 7fa2c9ed9640 1 -- 172.21.15.105:0/3945964664 shutdown_connections 2024-04-15T15:19:53.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.469+0000 7fa2c9ed9640 1 --2- 172.21.15.105:0/3945964664 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2c40b4890 0x7fa2c40b4d20 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:53.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.469+0000 7fa2c9ed9640 1 -- 172.21.15.105:0/3945964664 >> 172.21.15.105:0/3945964664 conn(0x7fa2c4073570 msgr2=0x7fa2c406b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:53.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.469+0000 7fa2c9ed9640 1 -- 172.21.15.105:0/3945964664 shutdown_connections 2024-04-15T15:19:53.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.469+0000 7fa2c9ed9640 1 -- 172.21.15.105:0/3945964664 wait complete. 2024-04-15T15:19:53.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.470+0000 7fa2c9ed9640 1 Processor -- start 2024-04-15T15:19:53.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.470+0000 7fa2c9ed9640 1 -- start start 2024-04-15T15:19:53.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.471+0000 7fa2c9ed9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2c40b4890 0x7fa2c40b1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:53.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.471+0000 7fa2c9ed9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2c40b52d0 0x7fa2c415b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:53.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.471+0000 7fa2c9ed9640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa2c415e9f0 0x7fa2c41462f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:53.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.471+0000 7fa2c9ed9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c40aff00 con 0x7fa2c40b4890 2024-04-15T15:19:53.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.471+0000 7fa2c9ed9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c4067940 con 0x7fa2c40b52d0 2024-04-15T15:19:53.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.471+0000 7fa2c9ed9640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fa2c40b0080 con 0x7fa2c415e9f0 2024-04-15T15:19:53.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.471+0000 7fa2c37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2c40b4890 0x7fa2c40b1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:53.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.471+0000 7fa2c37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2c40b4890 0x7fa2c40b1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46866/0 (socket says 172.21.15.105:46866) 2024-04-15T15:19:53.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.471+0000 7fa2c37fe640 1 -- 172.21.15.105:0/2654177889 learned_addr learned my addr 172.21.15.105:0/2654177889 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:53.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.471+0000 7fa2c2ffd640 1 --2- 172.21.15.105:0/2654177889 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2c40b52d0 0x7fa2c415b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:53.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.471+0000 7fa2c3fff640 1 --2- 172.21.15.105:0/2654177889 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa2c415e9f0 0x7fa2c41462f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:53.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.471+0000 7fa2c37fe640 1 -- 172.21.15.105:0/2654177889 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa2c415e9f0 msgr2=0x7fa2c41462f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:53.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.471+0000 7fa2c37fe640 1 --2- 172.21.15.105:0/2654177889 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa2c415e9f0 0x7fa2c41462f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:53.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.471+0000 7fa2c37fe640 1 -- 172.21.15.105:0/2654177889 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2c40b52d0 msgr2=0x7fa2c415b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:53.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.471+0000 7fa2c37fe640 1 --2- 172.21.15.105:0/2654177889 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2c40b52d0 0x7fa2c415b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:53.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.471+0000 7fa2c37fe640 1 -- 172.21.15.105:0/2654177889 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2c414dc60 con 0x7fa2c40b4890 2024-04-15T15:19:53.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.471+0000 7fa2c3fff640 1 --2- 172.21.15.105:0/2654177889 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa2c415e9f0 0x7fa2c41462f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:19:53.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.471+0000 7fa2c2ffd640 1 --2- 172.21.15.105:0/2654177889 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2c40b52d0 0x7fa2c415b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:19:53.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.472+0000 7fa2c37fe640 1 --2- 172.21.15.105:0/2654177889 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2c40b4890 0x7fa2c40b1e00 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa2b400c3f0 tx=0x7fa2b400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:53.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.472+0000 7fa2abfff640 1 -- 172.21.15.105:0/2654177889 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b40074e0 con 0x7fa2c40b4890 2024-04-15T15:19:53.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.472+0000 7fa2abfff640 1 -- 172.21.15.105:0/2654177889 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2b40047e0 con 0x7fa2c40b4890 2024-04-15T15:19:53.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.472+0000 7fa2abfff640 1 -- 172.21.15.105:0/2654177889 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b4004c10 con 0x7fa2c40b4890 2024-04-15T15:19:53.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.472+0000 7fa2c9ed9640 1 -- 172.21.15.105:0/2654177889 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2c414df40 con 0x7fa2c40b4890 2024-04-15T15:19:53.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.472+0000 7fa2c9ed9640 1 -- 172.21.15.105:0/2654177889 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2c414e500 con 0x7fa2c40b4890 2024-04-15T15:19:53.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.473+0000 7fa2c9ed9640 1 -- 172.21.15.105:0/2654177889 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2c4068ab0 con 0x7fa2c40b4890 2024-04-15T15:19:53.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.473+0000 7fa2abfff640 1 -- 172.21.15.105:0/2654177889 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa2b4011070 con 0x7fa2c40b4890 2024-04-15T15:19:53.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.474+0000 7fa2abfff640 1 --2- 172.21.15.105:0/2654177889 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fa2b006c920 0x7fa2b006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:53.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.474+0000 7fa2c2ffd640 1 --2- 172.21.15.105:0/2654177889 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fa2b006c920 0x7fa2b006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:53.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.474+0000 7fa2abfff640 1 -- 172.21.15.105:0/2654177889 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa2b408b760 con 0x7fa2c40b4890 2024-04-15T15:19:53.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.474+0000 7fa2c2ffd640 1 --2- 172.21.15.105:0/2654177889 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fa2b006c920 0x7fa2b006ede0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8036000 tx=0x7fa2b8030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:53.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.478+0000 7fa2abfff640 1 -- 172.21.15.105:0/2654177889 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa2b401a2b0 con 0x7fa2c40b4890 2024-04-15T15:19:53.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.655+0000 7fa2c9ed9640 1 -- 172.21.15.105:0/2654177889 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa2c40b0620 con 0x7fa2c40b4890 2024-04-15T15:19:53.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.656+0000 7fa2abfff640 1 -- 172.21.15.105:0/2654177889 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fa2b4059b90 con 0x7fa2c40b4890 2024-04-15T15:19:53.657 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-04-15T15:19:53.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.657+0000 7fa2c9ed9640 1 -- 172.21.15.105:0/2654177889 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fa2b006c920 msgr2=0x7fa2b006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:53.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.657+0000 7fa2c9ed9640 1 --2- 172.21.15.105:0/2654177889 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fa2b006c920 0x7fa2b006ede0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8036000 tx=0x7fa2b8030000 comp rx=0 tx=0).stop 2024-04-15T15:19:53.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.657+0000 7fa2c9ed9640 1 -- 172.21.15.105:0/2654177889 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2c40b4890 msgr2=0x7fa2c40b1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:53.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.657+0000 7fa2c9ed9640 1 --2- 172.21.15.105:0/2654177889 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2c40b4890 0x7fa2c40b1e00 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa2b400c3f0 tx=0x7fa2b400c8b0 comp rx=0 tx=0).stop 2024-04-15T15:19:53.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.657+0000 7fa2c9ed9640 1 -- 172.21.15.105:0/2654177889 shutdown_connections 2024-04-15T15:19:53.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.657+0000 7fa2c9ed9640 1 --2- 172.21.15.105:0/2654177889 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa2c415e9f0 0x7fa2c41462f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:53.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.657+0000 7fa2c9ed9640 1 --2- 172.21.15.105:0/2654177889 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fa2b006c920 0x7fa2b006ede0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:53.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.657+0000 7fa2c9ed9640 1 --2- 172.21.15.105:0/2654177889 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2c40b52d0 0x7fa2c415b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:53.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.657+0000 7fa2c9ed9640 1 --2- 172.21.15.105:0/2654177889 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2c40b4890 0x7fa2c40b1e00 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:53.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.657+0000 7fa2c9ed9640 1 -- 172.21.15.105:0/2654177889 >> 172.21.15.105:0/2654177889 conn(0x7fa2c4073570 msgr2=0x7fa2c406b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:53.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.658+0000 7fa2c9ed9640 1 -- 172.21.15.105:0/2654177889 shutdown_connections 2024-04-15T15:19:53.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.658+0000 7fa2c9ed9640 1 -- 172.21.15.105:0/2654177889 wait complete. 2024-04-15T15:19:53.671 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_quiesce.TestQuiesce.test_quiesce_blocked' 2024-04-15T15:19:53.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.775+0000 7fcebafe9640 1 Processor -- start 2024-04-15T15:19:53.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.775+0000 7fcebafe9640 1 -- start start 2024-04-15T15:19:53.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.776+0000 7fcebafe9640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fceb413e680 0x7fceb40cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:53.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.776+0000 7fcebafe9640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fceb413ec60 0x7fceb406ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:53.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.776+0000 7fcebafe9640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fceb406f4d0 0x7fceb4071e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:53.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.776+0000 7fcebafe9640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fceb415bc90 con 0x7fceb406f4d0 2024-04-15T15:19:53.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.776+0000 7fcebafe9640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fceb415bf70 con 0x7fceb413e680 2024-04-15T15:19:53.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.776+0000 7fcebafe9640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fceb415c250 con 0x7fceb413ec60 2024-04-15T15:19:53.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.776+0000 7fceb9586640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fceb406f4d0 0x7fceb4071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:53.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.776+0000 7fceb9586640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fceb406f4d0 0x7fceb4071e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46876/0 (socket says 172.21.15.105:46876) 2024-04-15T15:19:53.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.776+0000 7fceb9586640 1 -- 172.21.15.105:0/683851729 learned_addr learned my addr 172.21.15.105:0/683851729 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:53.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.776+0000 7fceabfff640 1 --2- 172.21.15.105:0/683851729 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fceb413ec60 0x7fceb406ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:53.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.776+0000 7fceb8d85640 1 --2- 172.21.15.105:0/683851729 >> v2:172.21.15.154:3300/0 conn(0x7fceb413e680 0x7fceb40cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:53.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.776+0000 7fceb9586640 1 -- 172.21.15.105:0/683851729 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fceb413ec60 msgr2=0x7fceb406ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:53.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.776+0000 7fceb9586640 1 --2- 172.21.15.105:0/683851729 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fceb413ec60 0x7fceb406ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:53.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.776+0000 7fceb9586640 1 -- 172.21.15.105:0/683851729 >> v2:172.21.15.154:3300/0 conn(0x7fceb413e680 msgr2=0x7fceb40cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:53.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.776+0000 7fceb9586640 1 --2- 172.21.15.105:0/683851729 >> v2:172.21.15.154:3300/0 conn(0x7fceb413e680 0x7fceb40cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:53.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.776+0000 7fceb9586640 1 -- 172.21.15.105:0/683851729 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceb415c530 con 0x7fceb406f4d0 2024-04-15T15:19:53.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.777+0000 7fceb9586640 1 --2- 172.21.15.105:0/683851729 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fceb406f4d0 0x7fceb4071e60 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fceb4073070 tx=0x7fceac02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33fe841cc52dc004 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:53.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.777+0000 7fceab7fe640 1 -- 172.21.15.105:0/683851729 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceac02ca70 con 0x7fceb406f4d0 2024-04-15T15:19:53.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.777+0000 7fceab7fe640 1 -- 172.21.15.105:0/683851729 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fceac02cc10 con 0x7fceb406f4d0 2024-04-15T15:19:53.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.777+0000 7fceab7fe640 1 -- 172.21.15.105:0/683851729 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceac0343f0 con 0x7fceb406f4d0 2024-04-15T15:19:53.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.777+0000 7fcebafe9640 1 -- 172.21.15.105:0/683851729 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fceb406f4d0 msgr2=0x7fceb4071e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:53.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.777+0000 7fcebafe9640 1 --2- 172.21.15.105:0/683851729 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fceb406f4d0 0x7fceb4071e60 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fceb4073070 tx=0x7fceac02bed0 comp rx=0 tx=0).stop 2024-04-15T15:19:53.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.778+0000 7fcebafe9640 1 -- 172.21.15.105:0/683851729 shutdown_connections 2024-04-15T15:19:53.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.778+0000 7fcebafe9640 1 --2- 172.21.15.105:0/683851729 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fceb406f4d0 0x7fceb4071e60 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:53.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.778+0000 7fcebafe9640 1 --2- 172.21.15.105:0/683851729 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fceb413ec60 0x7fceb406ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:53.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.778+0000 7fcebafe9640 1 --2- 172.21.15.105:0/683851729 >> v2:172.21.15.154:3300/0 conn(0x7fceb413e680 0x7fceb40cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:53.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.778+0000 7fcebafe9640 1 -- 172.21.15.105:0/683851729 >> 172.21.15.105:0/683851729 conn(0x7fceb4067d50 msgr2=0x7fceb406b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:53.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.778+0000 7fcebafe9640 1 -- 172.21.15.105:0/683851729 shutdown_connections 2024-04-15T15:19:53.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.778+0000 7fcebafe9640 1 -- 172.21.15.105:0/683851729 wait complete. 2024-04-15T15:19:53.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.779+0000 7fcebafe9640 1 Processor -- start 2024-04-15T15:19:53.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.779+0000 7fcebafe9640 1 -- start start 2024-04-15T15:19:53.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.780+0000 7fcebafe9640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fceb406f4d0 0x7fceb4144820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:53.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.780+0000 7fcebafe9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fceb413e680 0x7fceb413d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:53.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.780+0000 7fcebafe9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fceb413ec60 0x7fceb415dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:53.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.780+0000 7fcebafe9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fceb40d0230 con 0x7fceb413ec60 2024-04-15T15:19:53.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.780+0000 7fcebafe9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fceb40d00b0 con 0x7fceb413e680 2024-04-15T15:19:53.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.780+0000 7fcebafe9640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fceb40cf590 con 0x7fceb406f4d0 2024-04-15T15:19:53.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.780+0000 7fceb9586640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fceb413ec60 0x7fceb415dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:53.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.780+0000 7fceb9586640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fceb413ec60 0x7fceb415dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46886/0 (socket says 172.21.15.105:46886) 2024-04-15T15:19:53.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.780+0000 7fceb9586640 1 -- 172.21.15.105:0/3105693084 learned_addr learned my addr 172.21.15.105:0/3105693084 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:53.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.780+0000 7fceabfff640 1 --2- 172.21.15.105:0/3105693084 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fceb413e680 0x7fceb413d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:53.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.780+0000 7fceb8d85640 1 --2- 172.21.15.105:0/3105693084 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fceb406f4d0 0x7fceb4144820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:53.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.780+0000 7fceb9586640 1 -- 172.21.15.105:0/3105693084 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fceb406f4d0 msgr2=0x7fceb4144820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:53.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.780+0000 7fceb9586640 1 --2- 172.21.15.105:0/3105693084 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fceb406f4d0 0x7fceb4144820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:53.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.780+0000 7fceb9586640 1 -- 172.21.15.105:0/3105693084 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fceb413e680 msgr2=0x7fceb413d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:53.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.780+0000 7fceb9586640 1 --2- 172.21.15.105:0/3105693084 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fceb413e680 0x7fceb413d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:53.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.780+0000 7fceb9586640 1 -- 172.21.15.105:0/3105693084 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceb413dbb0 con 0x7fceb413ec60 2024-04-15T15:19:53.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.780+0000 7fceb9586640 1 --2- 172.21.15.105:0/3105693084 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fceb413ec60 0x7fceb415dce0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fceac036040 tx=0x7fceac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:53.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.780+0000 7fcea97fa640 1 -- 172.21.15.105:0/3105693084 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceac043020 con 0x7fceb413ec60 2024-04-15T15:19:53.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.780+0000 7fcea97fa640 1 -- 172.21.15.105:0/3105693084 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fceac03e450 con 0x7fceb413ec60 2024-04-15T15:19:53.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.780+0000 7fcebafe9640 1 -- 172.21.15.105:0/3105693084 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fceb4143a60 con 0x7fceb413ec60 2024-04-15T15:19:53.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.781+0000 7fcea97fa640 1 -- 172.21.15.105:0/3105693084 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceac009510 con 0x7fceb413ec60 2024-04-15T15:19:53.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.781+0000 7fcebafe9640 1 -- 172.21.15.105:0/3105693084 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fceb4144020 con 0x7fceb413ec60 2024-04-15T15:19:53.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.782+0000 7fcebafe9640 1 -- 172.21.15.105:0/3105693084 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fceb4068a40 con 0x7fceb413ec60 2024-04-15T15:19:53.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.782+0000 7fcea97fa640 1 -- 172.21.15.105:0/3105693084 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fceac046020 con 0x7fceb413ec60 2024-04-15T15:19:53.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.783+0000 7fcea97fa640 1 --2- 172.21.15.105:0/3105693084 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fcea006c8f0 0x7fcea006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:53.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.783+0000 7fceb8d85640 1 --2- 172.21.15.105:0/3105693084 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fcea006c8f0 0x7fcea006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:53.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.783+0000 7fcea97fa640 1 -- 172.21.15.105:0/3105693084 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fceac0ae280 con 0x7fceb413ec60 2024-04-15T15:19:53.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.783+0000 7fceb8d85640 1 --2- 172.21.15.105:0/3105693084 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fcea006c8f0 0x7fcea006edb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fceb00079a0 tx=0x7fceb0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:53.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:53.787+0000 7fcea97fa640 1 -- 172.21.15.105:0/3105693084 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fceac04d030 con 0x7fceb413ec60 2024-04-15T15:19:54.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.033+0000 7fcebafe9640 1 -- 172.21.15.105:0/3105693084 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_quiesce.TestQuiesce.test_quiesce_blocked"]} v 0) -- 0x7fceb40b3790 con 0x7fceb413ec60 2024-04-15T15:19:54.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.511+0000 7fcea97fa640 1 -- 172.21.15.105:0/3105693084 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_quiesce.TestQuiesce.test_quiesce_blocked"]}]=0 v49) ==== 140+0+0 (secure 0 0 0) 0x7fceac07c060 con 0x7fceb413ec60 2024-04-15T15:19:54.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.513+0000 7fcebafe9640 1 -- 172.21.15.105:0/3105693084 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fcea006c8f0 msgr2=0x7fcea006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:54.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.513+0000 7fcebafe9640 1 --2- 172.21.15.105:0/3105693084 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fcea006c8f0 0x7fcea006edb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fceb00079a0 tx=0x7fceb0008040 comp rx=0 tx=0).stop 2024-04-15T15:19:54.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.513+0000 7fcebafe9640 1 -- 172.21.15.105:0/3105693084 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fceb413ec60 msgr2=0x7fceb415dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:54.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.513+0000 7fcebafe9640 1 --2- 172.21.15.105:0/3105693084 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fceb413ec60 0x7fceb415dce0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fceac036040 tx=0x7fceac004a40 comp rx=0 tx=0).stop 2024-04-15T15:19:54.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.513+0000 7fcebafe9640 1 -- 172.21.15.105:0/3105693084 shutdown_connections 2024-04-15T15:19:54.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.513+0000 7fcebafe9640 1 --2- 172.21.15.105:0/3105693084 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fcea006c8f0 0x7fcea006edb0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:54.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.513+0000 7fcebafe9640 1 --2- 172.21.15.105:0/3105693084 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fceb413ec60 0x7fceb415dce0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:54.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.513+0000 7fcebafe9640 1 --2- 172.21.15.105:0/3105693084 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fceb413e680 0x7fceb413d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:54.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.513+0000 7fcebafe9640 1 --2- 172.21.15.105:0/3105693084 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fceb406f4d0 0x7fceb4144820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:54.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.513+0000 7fcebafe9640 1 -- 172.21.15.105:0/3105693084 >> 172.21.15.105:0/3105693084 conn(0x7fceb4067d50 msgr2=0x7fceb40b3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:54.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.514+0000 7fcebafe9640 1 -- 172.21.15.105:0/3105693084 shutdown_connections 2024-04-15T15:19:54.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.514+0000 7fcebafe9640 1 -- 172.21.15.105:0/3105693084 wait complete. 2024-04-15T15:19:54.527 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-15T15:19:54.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.631+0000 7f4f59515640 1 Processor -- start 2024-04-15T15:19:54.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.631+0000 7f4f59515640 1 -- start start 2024-04-15T15:19:54.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.632+0000 7f4f59515640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4f5415ee90 0x7f4f5409b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:54.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.632+0000 7f4f59515640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4f540b1730 0x7f4f5406d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:54.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.632+0000 7f4f59515640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f540b1cf0 con 0x7f4f540d16a0 2024-04-15T15:19:54.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.632+0000 7f4f59515640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f4f5409b990 con 0x7f4f5415ee90 2024-04-15T15:19:54.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.632+0000 7f4f59515640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f4f5406ddc0 con 0x7f4f540b1730 2024-04-15T15:19:54.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.633+0000 7f4f52ffd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4f5415ee90 0x7f4f5409b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:54.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.633+0000 7f4f537fe640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f4f540d16a0 0x7f4f54070530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.105:50752/0 (socket says 172.21.15.105:50752) 2024-04-15T15:19:54.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.633+0000 7f4f537fe640 1 -- 172.21.15.105:0/1228855541 learned_addr learned my addr 172.21.15.105:0/1228855541 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:19:54.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.633+0000 7f4f527fc640 1 --2- 172.21.15.105:0/1228855541 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4f540b1730 0x7f4f5406d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:54.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.633+0000 7f4f51ffb640 1 -- 172.21.15.105:0/1228855541 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2731923696 0 0) 0x7f4f540b1cf0 con 0x7f4f540d16a0 2024-04-15T15:19:54.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.633+0000 7f4f51ffb640 1 -- 172.21.15.105:0/1228855541 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4f40003520 con 0x7f4f540d16a0 2024-04-15T15:19:54.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.633+0000 7f4f52ffd640 1 -- 172.21.15.105:0/1228855541 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4f540b1730 msgr2=0x7f4f5406d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:54.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.633+0000 7f4f52ffd640 1 --2- 172.21.15.105:0/1228855541 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4f540b1730 0x7f4f5406d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:54.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.633+0000 7f4f52ffd640 1 -- 172.21.15.105:0/1228855541 >> v1:172.21.15.154:6789/0 conn(0x7f4f540d16a0 legacy=0x7f4f54070530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:54.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.633+0000 7f4f52ffd640 1 -- 172.21.15.105:0/1228855541 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f5406cac0 con 0x7f4f5415ee90 2024-04-15T15:19:54.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.634+0000 7f4f52ffd640 1 --2- 172.21.15.105:0/1228855541 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f5415ee90 0x7f4f5409b450 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4f44009d90 tx=0x7f4f4402bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62c95e7b99902fc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:54.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.634+0000 7f4f51ffb640 1 -- 172.21.15.105:0/1228855541 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f44039070 con 0x7f4f5415ee90 2024-04-15T15:19:54.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.634+0000 7f4f51ffb640 1 -- 172.21.15.105:0/1228855541 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f4402cc50 con 0x7f4f5415ee90 2024-04-15T15:19:54.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.634+0000 7f4f51ffb640 1 -- 172.21.15.105:0/1228855541 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f44034440 con 0x7f4f5415ee90 2024-04-15T15:19:54.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.635+0000 7f4f59515640 1 -- 172.21.15.105:0/1228855541 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f5415ee90 msgr2=0x7f4f5409b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:54.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.635+0000 7f4f59515640 1 --2- 172.21.15.105:0/1228855541 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f5415ee90 0x7f4f5409b450 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4f44009d90 tx=0x7f4f4402bea0 comp rx=0 tx=0).stop 2024-04-15T15:19:54.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.635+0000 7f4f59515640 1 -- 172.21.15.105:0/1228855541 shutdown_connections 2024-04-15T15:19:54.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.635+0000 7f4f59515640 1 --2- 172.21.15.105:0/1228855541 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4f540b1730 0x7f4f5406d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:54.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.635+0000 7f4f59515640 1 --2- 172.21.15.105:0/1228855541 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f5415ee90 0x7f4f5409b450 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:54.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.635+0000 7f4f59515640 1 -- 172.21.15.105:0/1228855541 >> 172.21.15.105:0/1228855541 conn(0x7f4f540b4070 msgr2=0x7f4f540b4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:54.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.635+0000 7f4f59515640 1 -- 172.21.15.105:0/1228855541 shutdown_connections 2024-04-15T15:19:54.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.635+0000 7f4f59515640 1 -- 172.21.15.105:0/1228855541 wait complete. 2024-04-15T15:19:54.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.636+0000 7f4f59515640 1 Processor -- start 2024-04-15T15:19:54.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.636+0000 7f4f59515640 1 -- start start 2024-04-15T15:19:54.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.637+0000 7f4f59515640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4f540b1730 0x7f4f54151980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:54.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.637+0000 7f4f59515640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f540d16a0 0x7f4f540c8210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:54.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.637+0000 7f4f59515640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f5415ee90 0x7f4f540b77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:54.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.637+0000 7f4f59515640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4f5406d080 con 0x7f4f5415ee90 2024-04-15T15:19:54.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.637+0000 7f4f59515640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4f5406d3f0 con 0x7f4f540d16a0 2024-04-15T15:19:54.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.637+0000 7f4f59515640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f4f540d1b30 con 0x7f4f540b1730 2024-04-15T15:19:54.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.637+0000 7f4f537fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f5415ee90 0x7f4f540b77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:54.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.637+0000 7f4f537fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f5415ee90 0x7f4f540b77b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46888/0 (socket says 172.21.15.105:46888) 2024-04-15T15:19:54.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.637+0000 7f4f537fe640 1 -- 172.21.15.105:0/403746181 learned_addr learned my addr 172.21.15.105:0/403746181 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:54.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.637+0000 7f4f52ffd640 1 --2- 172.21.15.105:0/403746181 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4f540b1730 0x7f4f54151980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:54.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.637+0000 7f4f527fc640 1 --2- 172.21.15.105:0/403746181 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f540d16a0 0x7f4f540c8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:54.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.637+0000 7f4f537fe640 1 -- 172.21.15.105:0/403746181 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4f540b1730 msgr2=0x7f4f54151980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:54.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.637+0000 7f4f537fe640 1 --2- 172.21.15.105:0/403746181 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4f540b1730 0x7f4f54151980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:54.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.637+0000 7f4f537fe640 1 -- 172.21.15.105:0/403746181 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f540d16a0 msgr2=0x7f4f540c8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:54.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.637+0000 7f4f537fe640 1 --2- 172.21.15.105:0/403746181 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f540d16a0 0x7f4f540c8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:54.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.637+0000 7f4f537fe640 1 -- 172.21.15.105:0/403746181 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f540b5800 con 0x7f4f5415ee90 2024-04-15T15:19:54.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.637+0000 7f4f527fc640 1 --2- 172.21.15.105:0/403746181 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f540d16a0 0x7f4f540c8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:19:54.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.637+0000 7f4f537fe640 1 --2- 172.21.15.105:0/403746181 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f5415ee90 0x7f4f540b77b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c00c430 tx=0x7f4f4c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:54.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.637+0000 7f4f4b7fe640 1 -- 172.21.15.105:0/403746181 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c017070 con 0x7f4f5415ee90 2024-04-15T15:19:54.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.638+0000 7f4f4b7fe640 1 -- 172.21.15.105:0/403746181 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f4c005ce0 con 0x7f4f5415ee90 2024-04-15T15:19:54.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.638+0000 7f4f59515640 1 -- 172.21.15.105:0/403746181 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f540b5ae0 con 0x7f4f5415ee90 2024-04-15T15:19:54.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.638+0000 7f4f4b7fe640 1 -- 172.21.15.105:0/403746181 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c007310 con 0x7f4f5415ee90 2024-04-15T15:19:54.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.638+0000 7f4f59515640 1 -- 172.21.15.105:0/403746181 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f540b60a0 con 0x7f4f5415ee90 2024-04-15T15:19:54.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.639+0000 7f4f59515640 1 -- 172.21.15.105:0/403746181 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f5415cbb0 con 0x7f4f5415ee90 2024-04-15T15:19:54.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.639+0000 7f4f4b7fe640 1 -- 172.21.15.105:0/403746181 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4f4c01a070 con 0x7f4f5415ee90 2024-04-15T15:19:54.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.639+0000 7f4f4b7fe640 1 --2- 172.21.15.105:0/403746181 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f4f3406c8f0 0x7f4f3406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:54.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.639+0000 7f4f4b7fe640 1 -- 172.21.15.105:0/403746181 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4f4c08af60 con 0x7f4f5415ee90 2024-04-15T15:19:54.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.639+0000 7f4f52ffd640 1 --2- 172.21.15.105:0/403746181 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f4f3406c8f0 0x7f4f3406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:54.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.640+0000 7f4f52ffd640 1 --2- 172.21.15.105:0/403746181 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f4f3406c8f0 0x7f4f3406edb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4f44036000 tx=0x7f4f44030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:54.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.644+0000 7f4f4b7fe640 1 -- 172.21.15.105:0/403746181 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4f4c059420 con 0x7f4f5415ee90 2024-04-15T15:19:54.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.822+0000 7f4f59515640 1 -- 172.21.15.105:0/403746181 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f4f54069150 con 0x7f4f5415ee90 2024-04-15T15:19:54.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.825+0000 7f4f4b7fe640 1 -- 172.21.15.105:0/403746181 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f4f4c058d20 con 0x7f4f5415ee90 2024-04-15T15:19:54.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.827+0000 7f4f59515640 1 -- 172.21.15.105:0/403746181 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f4f3406c8f0 msgr2=0x7f4f3406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:54.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.827+0000 7f4f59515640 1 --2- 172.21.15.105:0/403746181 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f4f3406c8f0 0x7f4f3406edb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4f44036000 tx=0x7f4f44030000 comp rx=0 tx=0).stop 2024-04-15T15:19:54.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.827+0000 7f4f59515640 1 -- 172.21.15.105:0/403746181 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f5415ee90 msgr2=0x7f4f540b77b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:54.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.827+0000 7f4f59515640 1 --2- 172.21.15.105:0/403746181 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f5415ee90 0x7f4f540b77b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c00c430 tx=0x7f4f4c00c8f0 comp rx=0 tx=0).stop 2024-04-15T15:19:54.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.827+0000 7f4f59515640 1 -- 172.21.15.105:0/403746181 shutdown_connections 2024-04-15T15:19:54.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.827+0000 7f4f59515640 1 --2- 172.21.15.105:0/403746181 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f4f3406c8f0 0x7f4f3406edb0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:54.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.827+0000 7f4f59515640 1 --2- 172.21.15.105:0/403746181 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f5415ee90 0x7f4f540b77b0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:54.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.827+0000 7f4f59515640 1 --2- 172.21.15.105:0/403746181 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f540d16a0 0x7f4f540c8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:54.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.827+0000 7f4f59515640 1 --2- 172.21.15.105:0/403746181 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4f540b1730 0x7f4f54151980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:54.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.827+0000 7f4f59515640 1 -- 172.21.15.105:0/403746181 >> 172.21.15.105:0/403746181 conn(0x7f4f540b4070 msgr2=0x7f4f540d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:54.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.827+0000 7f4f59515640 1 -- 172.21.15.105:0/403746181 shutdown_connections 2024-04-15T15:19:54.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:54.827+0000 7f4f59515640 1 -- 172.21.15.105:0/403746181 wait complete. 2024-04-15T15:19:54.840 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-15T15:19:54.840 DEBUG:teuthology.orchestra.run.smithi105:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-15T15:19:54.879 DEBUG:teuthology.orchestra.run.smithi154:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-15T15:19:54.932 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-15T15:19:54.932 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-04-15T15:19:54.932 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:19:54.951 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-04-15T15:19:54.951 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:19:55.007 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:19:55.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.147+0000 7f220b063640 1 Processor -- start 2024-04-15T15:19:55.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.148+0000 7f220b063640 1 -- start start 2024-04-15T15:19:55.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.148+0000 7f220b063640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f220406e060 0x7f22040d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:55.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.148+0000 7f220b063640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f220406d2a0 con 0x7f22040cd360 2024-04-15T15:19:55.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.149+0000 7f220b063640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f220406d580 con 0x7f220415ecf0 2024-04-15T15:19:55.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.149+0000 7f220b063640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f22040d1450 con 0x7f220406e060 2024-04-15T15:19:55.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.149+0000 7f2203fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f220406e060 0x7f22040d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:55.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.149+0000 7f2209600640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f22040cd360 0x7f2204162960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:57414/0 (socket says 172.21.15.105:57414) 2024-04-15T15:19:55.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.149+0000 7f2209600640 1 -- 172.21.15.105:0/1265504955 learned_addr learned my addr 172.21.15.105:0/1265504955 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:19:55.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.149+0000 7f2203fff640 1 -- 172.21.15.105:0/1265504955 >> v1:172.21.15.154:6789/0 conn(0x7f220415ecf0 legacy=0x7f22040ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:55.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.149+0000 7f2203fff640 1 -- 172.21.15.105:0/1265504955 >> v1:172.21.15.105:6789/0 conn(0x7f22040cd360 legacy=0x7f2204162960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:55.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.149+0000 7f2203fff640 1 -- 172.21.15.105:0/1265504955 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f220406d860 con 0x7f220406e060 2024-04-15T15:19:55.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.149+0000 7f22037fe640 1 -- 172.21.15.105:0/1265504955 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 964051622 0 0) 0x7f220406d2a0 con 0x7f22040cd360 2024-04-15T15:19:55.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.150+0000 7f2203fff640 1 --2- 172.21.15.105:0/1265504955 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f220406e060 0x7f22040d0f10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f21f80097a0 tx=0x7f21f802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f15879f009ff27f server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:55.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.150+0000 7f22037fe640 1 -- 172.21.15.105:0/1265504955 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f8035020 con 0x7f220406e060 2024-04-15T15:19:55.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.150+0000 7f22037fe640 1 -- 172.21.15.105:0/1265504955 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21f8031830 con 0x7f220406e060 2024-04-15T15:19:55.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.150+0000 7f22037fe640 1 -- 172.21.15.105:0/1265504955 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f8031c00 con 0x7f220406e060 2024-04-15T15:19:55.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.150+0000 7f220b063640 1 -- 172.21.15.105:0/1265504955 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f220406e060 msgr2=0x7f22040d0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:55.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.150+0000 7f220b063640 1 --2- 172.21.15.105:0/1265504955 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f220406e060 0x7f22040d0f10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f21f80097a0 tx=0x7f21f802b8d0 comp rx=0 tx=0).stop 2024-04-15T15:19:55.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.150+0000 7f220b063640 1 -- 172.21.15.105:0/1265504955 shutdown_connections 2024-04-15T15:19:55.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.150+0000 7f220b063640 1 --2- 172.21.15.105:0/1265504955 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f220406e060 0x7f22040d0f10 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:55.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.151+0000 7f220b063640 1 -- 172.21.15.105:0/1265504955 >> 172.21.15.105:0/1265504955 conn(0x7f22040d2600 msgr2=0x7f22040d2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:55.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.151+0000 7f220b063640 1 -- 172.21.15.105:0/1265504955 shutdown_connections 2024-04-15T15:19:55.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.151+0000 7f220b063640 1 -- 172.21.15.105:0/1265504955 wait complete. 2024-04-15T15:19:55.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.152+0000 7f220b063640 1 Processor -- start 2024-04-15T15:19:55.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.152+0000 7f220b063640 1 -- start start 2024-04-15T15:19:55.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.153+0000 7f220b063640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f220406e060 0x7f220415ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:55.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.153+0000 7f220b063640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f22040cd360 0x7f22040b7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:55.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.153+0000 7f220b063640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f220415ecf0 0x7f22040b7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:55.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.153+0000 7f220b063640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f220415f0d0 con 0x7f220415ecf0 2024-04-15T15:19:55.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.153+0000 7f220b063640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f220406cf90 con 0x7f22040cd360 2024-04-15T15:19:55.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.153+0000 7f220b063640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f220415f250 con 0x7f220406e060 2024-04-15T15:19:55.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.153+0000 7f2209600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f220415ecf0 0x7f22040b7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:55.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.153+0000 7f2209600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f220415ecf0 0x7f22040b7590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46908/0 (socket says 172.21.15.105:46908) 2024-04-15T15:19:55.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.153+0000 7f2209600640 1 -- 172.21.15.105:0/1468259303 learned_addr learned my addr 172.21.15.105:0/1468259303 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:55.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.153+0000 7f2203fff640 1 --2- 172.21.15.105:0/1468259303 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f22040cd360 0x7f22040b7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:55.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.153+0000 7f2208dff640 1 --2- 172.21.15.105:0/1468259303 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f220406e060 0x7f220415ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:55.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.153+0000 7f2209600640 1 -- 172.21.15.105:0/1468259303 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f220406e060 msgr2=0x7f220415ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:55.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.153+0000 7f2209600640 1 --2- 172.21.15.105:0/1468259303 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f220406e060 0x7f220415ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:55.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.153+0000 7f2209600640 1 -- 172.21.15.105:0/1468259303 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f22040cd360 msgr2=0x7f22040b7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:55.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.153+0000 7f2209600640 1 --2- 172.21.15.105:0/1468259303 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f22040cd360 0x7f22040b7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:55.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.153+0000 7f2209600640 1 -- 172.21.15.105:0/1468259303 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22040b6560 con 0x7f220415ecf0 2024-04-15T15:19:55.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.153+0000 7f2208dff640 1 --2- 172.21.15.105:0/1468259303 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f220406e060 0x7f220415ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:19:55.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.153+0000 7f2209600640 1 --2- 172.21.15.105:0/1468259303 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f220415ecf0 0x7f22040b7590 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f21fc00c7d0 tx=0x7f21fc00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:55.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.153+0000 7f22017fa640 1 -- 172.21.15.105:0/1468259303 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21fc018070 con 0x7f220415ecf0 2024-04-15T15:19:55.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.153+0000 7f22017fa640 1 -- 172.21.15.105:0/1468259303 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21fc00a640 con 0x7f220415ecf0 2024-04-15T15:19:55.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.153+0000 7f220b063640 1 -- 172.21.15.105:0/1468259303 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22040c76e0 con 0x7f220415ecf0 2024-04-15T15:19:55.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.153+0000 7f22017fa640 1 -- 172.21.15.105:0/1468259303 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21fc00aa70 con 0x7f220415ecf0 2024-04-15T15:19:55.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.153+0000 7f220b063640 1 -- 172.21.15.105:0/1468259303 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22040c7ca0 con 0x7f220415ecf0 2024-04-15T15:19:55.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.154+0000 7f220b063640 1 -- 172.21.15.105:0/1468259303 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2204068a40 con 0x7f220415ecf0 2024-04-15T15:19:55.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.155+0000 7f22017fa640 1 -- 172.21.15.105:0/1468259303 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f21fc01b070 con 0x7f220415ecf0 2024-04-15T15:19:55.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.155+0000 7f22017fa640 1 --2- 172.21.15.105:0/1468259303 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f21f006c8f0 0x7f21f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:55.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.156+0000 7f2208dff640 1 --2- 172.21.15.105:0/1468259303 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f21f006c8f0 0x7f21f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:55.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.156+0000 7f22017fa640 1 -- 172.21.15.105:0/1468259303 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f21fc08b130 con 0x7f220415ecf0 2024-04-15T15:19:55.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.156+0000 7f2208dff640 1 --2- 172.21.15.105:0/1468259303 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f21f006c8f0 0x7f21f006edb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f21f4003eb0 tx=0x7f21f4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:55.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.160+0000 7f22017fa640 1 -- 172.21.15.105:0/1468259303 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f22040b6560 con 0x7f220415ecf0 2024-04-15T15:19:55.276 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-15T15:19:55.279 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-15T15:19:55.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.339+0000 7f220b063640 1 -- 172.21.15.105:0/1468259303 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f22040b8580 con 0x7f220415ecf0 2024-04-15T15:19:55.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.340+0000 7f22017fa640 1 -- 172.21.15.105:0/1468259303 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16653 (secure 0 0 0) 0x7f21fc058e80 con 0x7f220415ecf0 2024-04-15T15:19:55.343 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:19:55.343 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":20,"fsid":"09e4d2a4-a324-4793-af77-644b25e520e1","created":"2024-04-15T15:19:01.524543+0000","modified":"2024-04-15T15:19:27.414890+0000","last_up_change":"2024-04-15T15:19:12.825815+0000","last_in_change":"2024-04-15T15:19:03.788126+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:19:13.748585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:19:23.789287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:19:24.722250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6f6292b-2d7e-4aa0-9d1c-6ddd0f97287c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6801","nonce":2873040054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6803","nonce":2873040054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6807","nonce":2873040054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6805","nonce":2873040054}]},"public_addr":"172.21.15.105:6801/2873040054","cluster_addr":"172.21.15.105:6803/2873040054","heartbeat_back_addr":"172.21.15.105:6807/2873040054","heartbeat_front_addr":"172.21.15.105:6805/2873040054","state":["exists","up"]},{"osd":1,"uuid":"8900966a-b393-41e3-8f93-e5a43bd23622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6818","nonce":2494994479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6821","nonce":2494994479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6828","nonce":2494994479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6823","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6825","nonce":2494994479}]},"public_addr":"172.21.15.105:6818/2494994479","cluster_addr":"172.21.15.105:6821/2494994479","heartbeat_back_addr":"172.21.15.105:6828/2494994479","heartbeat_front_addr":"172.21.15.105:6825/2494994479","state":["exists","up"]},{"osd":2,"uuid":"81a5dc7a-9325-4ff5-941d-a2d5fba21f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6809","nonce":3798408699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6811","nonce":3798408699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6815","nonce":3798408699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6813","nonce":3798408699}]},"public_addr":"172.21.15.105:6809/3798408699","cluster_addr":"172.21.15.105:6811/3798408699","heartbeat_back_addr":"172.21.15.105:6815/3798408699","heartbeat_front_addr":"172.21.15.105:6813/3798408699","state":["exists","up"]},{"osd":3,"uuid":"df766ed2-5c71-4805-96f4-3d4970edc960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6819","nonce":1920720186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6824","nonce":1920720186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6831","nonce":1920720186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6827","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6829","nonce":1920720186}]},"public_addr":"172.21.15.105:6819/1920720186","cluster_addr":"172.21.15.105:6824/1920720186","heartbeat_back_addr":"172.21.15.105:6831/1920720186","heartbeat_front_addr":"172.21.15.105:6829/1920720186","state":["exists","up"]},{"osd":4,"uuid":"e1039e03-28b8-49df-b951-c91930b93c9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6810","nonce":3516494456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6813","nonce":3516494456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6820","nonce":3516494456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6817","nonce":3516494456}]},"public_addr":"172.21.15.154:6810/3516494456","cluster_addr":"172.21.15.154:6813/3516494456","heartbeat_back_addr":"172.21.15.154:6820/3516494456","heartbeat_front_addr":"172.21.15.154:6817/3516494456","state":["exists","up"]},{"osd":5,"uuid":"a9f5eaf1-66ff-405a-a9e4-ec3340812a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6801","nonce":540367751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6803","nonce":540367751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6807","nonce":540367751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6805","nonce":540367751}]},"public_addr":"172.21.15.154:6801/540367751","cluster_addr":"172.21.15.154:6803/540367751","heartbeat_back_addr":"172.21.15.154:6807/540367751","heartbeat_front_addr":"172.21.15.154:6805/540367751","state":["exists","up"]},{"osd":6,"uuid":"267d02ea-675f-428f-aae8-e0375e4d4feb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6825","nonce":1550829005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6827","nonce":1550829005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6831","nonce":1550829005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6829","nonce":1550829005}]},"public_addr":"172.21.15.154:6825/1550829005","cluster_addr":"172.21.15.154:6827/1550829005","heartbeat_back_addr":"172.21.15.154:6831/1550829005","heartbeat_front_addr":"172.21.15.154:6829/1550829005","state":["exists","up"]},{"osd":7,"uuid":"759ac6cc-896a-423c-885f-9fe90d8dcb14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6809","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6811","nonce":3920089903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6816","nonce":3920089903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6823","nonce":3920089903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6819","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6821","nonce":3920089903}]},"public_addr":"172.21.15.154:6811/3920089903","cluster_addr":"172.21.15.154:6816/3920089903","heartbeat_back_addr":"172.21.15.154:6823/3920089903","heartbeat_front_addr":"172.21.15.154:6821/3920089903","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.949509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.984072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.907278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.947391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.975447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.998896+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:09.028739+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.894119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:19:55.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.342+0000 7f220b063640 1 -- 172.21.15.105:0/1468259303 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f21f006c8f0 msgr2=0x7f21f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:55.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.342+0000 7f220b063640 1 --2- 172.21.15.105:0/1468259303 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f21f006c8f0 0x7f21f006edb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f21f4003eb0 tx=0x7f21f4009040 comp rx=0 tx=0).stop 2024-04-15T15:19:55.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.342+0000 7f220b063640 1 -- 172.21.15.105:0/1468259303 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f220415ecf0 msgr2=0x7f22040b7590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:55.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.342+0000 7f220b063640 1 --2- 172.21.15.105:0/1468259303 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f220415ecf0 0x7f22040b7590 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f21fc00c7d0 tx=0x7f21fc00cc90 comp rx=0 tx=0).stop 2024-04-15T15:19:55.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.342+0000 7f220b063640 1 -- 172.21.15.105:0/1468259303 shutdown_connections 2024-04-15T15:19:55.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.342+0000 7f220b063640 1 --2- 172.21.15.105:0/1468259303 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f21f006c8f0 0x7f21f006edb0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:55.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.342+0000 7f220b063640 1 --2- 172.21.15.105:0/1468259303 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f220415ecf0 0x7f22040b7590 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:55.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.342+0000 7f220b063640 1 --2- 172.21.15.105:0/1468259303 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f22040cd360 0x7f22040b7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:55.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.342+0000 7f220b063640 1 --2- 172.21.15.105:0/1468259303 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f220406e060 0x7f220415ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:55.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.342+0000 7f220b063640 1 -- 172.21.15.105:0/1468259303 >> 172.21.15.105:0/1468259303 conn(0x7f22040d2600 msgr2=0x7f220409b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:55.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.342+0000 7f220b063640 1 -- 172.21.15.105:0/1468259303 shutdown_connections 2024-04-15T15:19:55.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.342+0000 7f220b063640 1 -- 172.21.15.105:0/1468259303 wait complete. 2024-04-15T15:19:55.356 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:19:13.748585+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-15T15:19:23.789287+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-15T15:19:24.722250+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:19:55.357 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:19:55.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.503+0000 7f825ffbd640 1 Processor -- start 2024-04-15T15:19:55.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.503+0000 7f825ffbd640 1 -- start start 2024-04-15T15:19:55.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.504+0000 7f825ffbd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8258072570 0x7f825806de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:55.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.504+0000 7f825ffbd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f825806e3c0 0x7f825815d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:55.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.504+0000 7f825ffbd640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f825806ebb0 con 0x7f8258145f00 2024-04-15T15:19:55.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.504+0000 7f825ffbd640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f82581462e0 con 0x7f825806e3c0 2024-04-15T15:19:55.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.504+0000 7f825ffbd640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f825806ee90 con 0x7f8258072570 2024-04-15T15:19:55.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.504+0000 7f825e55a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f825806e3c0 0x7f825815d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:55.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.504+0000 7f825dd59640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f8258145f00 0x7f8258071e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:57424/0 (socket says 172.21.15.105:57424) 2024-04-15T15:19:55.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.504+0000 7f825dd59640 1 -- 172.21.15.105:0/669115777 learned_addr learned my addr 172.21.15.105:0/669115777 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:19:55.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.504+0000 7f825e55a640 1 -- 172.21.15.105:0/669115777 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8258072570 msgr2=0x7f825806de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:55.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.504+0000 7f825e55a640 1 --2- 172.21.15.105:0/669115777 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8258072570 0x7f825806de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:55.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.504+0000 7f825e55a640 1 -- 172.21.15.105:0/669115777 >> v1:172.21.15.105:6789/0 conn(0x7f8258145f00 legacy=0x7f8258071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:55.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.504+0000 7f825cd57640 1 -- 172.21.15.105:0/669115777 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 236234940 0 0) 0x7f825806ebb0 con 0x7f8258145f00 2024-04-15T15:19:55.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.504+0000 7f825e55a640 1 -- 172.21.15.105:0/669115777 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f825815cde0 con 0x7f825806e3c0 2024-04-15T15:19:55.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.505+0000 7f825e55a640 1 --2- 172.21.15.105:0/669115777 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f825806e3c0 0x7f825815d5e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8250009d90 tx=0x7f825002bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb24fc2d25bf4cdf server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:55.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.506+0000 7f825cd57640 1 -- 172.21.15.105:0/669115777 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8250039070 con 0x7f825806e3c0 2024-04-15T15:19:55.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.506+0000 7f825cd57640 1 -- 172.21.15.105:0/669115777 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f825002cb30 con 0x7f825806e3c0 2024-04-15T15:19:55.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.506+0000 7f825cd57640 1 -- 172.21.15.105:0/669115777 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8250034440 con 0x7f825806e3c0 2024-04-15T15:19:55.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.506+0000 7f825ffbd640 1 -- 172.21.15.105:0/669115777 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f825806e3c0 msgr2=0x7f825815d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:55.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.506+0000 7f825ffbd640 1 --2- 172.21.15.105:0/669115777 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f825806e3c0 0x7f825815d5e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8250009d90 tx=0x7f825002bd80 comp rx=0 tx=0).stop 2024-04-15T15:19:55.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.506+0000 7f825ffbd640 1 -- 172.21.15.105:0/669115777 shutdown_connections 2024-04-15T15:19:55.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.506+0000 7f825ffbd640 1 --2- 172.21.15.105:0/669115777 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f825806e3c0 0x7f825815d5e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:55.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.506+0000 7f825ffbd640 1 --2- 172.21.15.105:0/669115777 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8258072570 0x7f825806de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:55.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.506+0000 7f825ffbd640 1 -- 172.21.15.105:0/669115777 >> 172.21.15.105:0/669115777 conn(0x7f8258067d50 msgr2=0x7f825806b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:55.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.506+0000 7f825ffbd640 1 -- 172.21.15.105:0/669115777 shutdown_connections 2024-04-15T15:19:55.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.506+0000 7f825ffbd640 1 -- 172.21.15.105:0/669115777 wait complete. 2024-04-15T15:19:55.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.508+0000 7f825ffbd640 1 Processor -- start 2024-04-15T15:19:55.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.508+0000 7f825ffbd640 1 -- start start 2024-04-15T15:19:55.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.508+0000 7f825ffbd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f825806e3c0 0x7f82581f6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:55.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.508+0000 7f825ffbd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8258072570 0x7f82580c84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:55.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.508+0000 7f825ffbd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8258145f00 0x7f8258140e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:55.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.508+0000 7f825ffbd640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f825806e950 con 0x7f825806e3c0 2024-04-15T15:19:55.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.508+0000 7f825ffbd640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f825806e7d0 con 0x7f8258145f00 2024-04-15T15:19:55.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.508+0000 7f825ffbd640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f825815db20 con 0x7f8258072570 2024-04-15T15:19:55.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.508+0000 7f825dd59640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f825806e3c0 0x7f82581f6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:55.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.508+0000 7f825dd59640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f825806e3c0 0x7f82581f6150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46918/0 (socket says 172.21.15.105:46918) 2024-04-15T15:19:55.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.508+0000 7f825dd59640 1 -- 172.21.15.105:0/2914888847 learned_addr learned my addr 172.21.15.105:0/2914888847 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:55.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.509+0000 7f825dd59640 1 -- 172.21.15.105:0/2914888847 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8258072570 msgr2=0x7f82580c84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:55.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.509+0000 7f825dd59640 1 --2- 172.21.15.105:0/2914888847 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8258072570 0x7f82580c84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:55.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.509+0000 7f825dd59640 1 -- 172.21.15.105:0/2914888847 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8258145f00 msgr2=0x7f8258140e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:55.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.509+0000 7f825dd59640 1 --2- 172.21.15.105:0/2914888847 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8258145f00 0x7f8258140e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:55.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.509+0000 7f825dd59640 1 -- 172.21.15.105:0/2914888847 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82580c8cf0 con 0x7f825806e3c0 2024-04-15T15:19:55.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.509+0000 7f825dd59640 1 --2- 172.21.15.105:0/2914888847 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f825806e3c0 0x7f82581f6150 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f824800c430 tx=0x7f824800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:55.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.509+0000 7f824e7fc640 1 -- 172.21.15.105:0/2914888847 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82480074e0 con 0x7f825806e3c0 2024-04-15T15:19:55.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.509+0000 7f825ffbd640 1 -- 172.21.15.105:0/2914888847 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82580c6600 con 0x7f825806e3c0 2024-04-15T15:19:55.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.509+0000 7f824e7fc640 1 -- 172.21.15.105:0/2914888847 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82480047e0 con 0x7f825806e3c0 2024-04-15T15:19:55.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.509+0000 7f824e7fc640 1 -- 172.21.15.105:0/2914888847 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8248004bd0 con 0x7f825806e3c0 2024-04-15T15:19:55.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.509+0000 7f825ffbd640 1 -- 172.21.15.105:0/2914888847 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82580c89e0 con 0x7f825806e3c0 2024-04-15T15:19:55.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.510+0000 7f825ffbd640 1 -- 172.21.15.105:0/2914888847 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82580c69f0 con 0x7f825806e3c0 2024-04-15T15:19:55.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.510+0000 7f824e7fc640 1 -- 172.21.15.105:0/2914888847 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8248011070 con 0x7f825806e3c0 2024-04-15T15:19:55.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.511+0000 7f824e7fc640 1 --2- 172.21.15.105:0/2914888847 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f823c06c8f0 0x7f823c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:55.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.511+0000 7f824e7fc640 1 -- 172.21.15.105:0/2914888847 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f824808b7c0 con 0x7f825806e3c0 2024-04-15T15:19:55.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.511+0000 7f825d558640 1 --2- 172.21.15.105:0/2914888847 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f823c06c8f0 0x7f823c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:55.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.511+0000 7f825d558640 1 --2- 172.21.15.105:0/2914888847 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f823c06c8f0 0x7f823c06edb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f82581f6690 tx=0x7f8254006c90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:55.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.515+0000 7f824e7fc640 1 -- 172.21.15.105:0/2914888847 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8248059c00 con 0x7f825806e3c0 2024-04-15T15:19:55.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.721+0000 7f825ffbd640 1 -- 172.21.15.105:0/2914888847 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8258068ab0 con 0x7f825806e3c0 2024-04-15T15:19:55.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.722+0000 7f824e7fc640 1 -- 172.21.15.105:0/2914888847 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f8248059500 con 0x7f825806e3c0 2024-04-15T15:19:55.724 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-04-15T15:19:55.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.725+0000 7f825ffbd640 1 -- 172.21.15.105:0/2914888847 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f823c06c8f0 msgr2=0x7f823c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:55.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.725+0000 7f825ffbd640 1 --2- 172.21.15.105:0/2914888847 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f823c06c8f0 0x7f823c06edb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f82581f6690 tx=0x7f8254006c90 comp rx=0 tx=0).stop 2024-04-15T15:19:55.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.725+0000 7f825ffbd640 1 -- 172.21.15.105:0/2914888847 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f825806e3c0 msgr2=0x7f82581f6150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:55.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.725+0000 7f825ffbd640 1 --2- 172.21.15.105:0/2914888847 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f825806e3c0 0x7f82581f6150 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f824800c430 tx=0x7f824800c8f0 comp rx=0 tx=0).stop 2024-04-15T15:19:55.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.725+0000 7f825ffbd640 1 -- 172.21.15.105:0/2914888847 shutdown_connections 2024-04-15T15:19:55.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.725+0000 7f825ffbd640 1 --2- 172.21.15.105:0/2914888847 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8258072570 0x7f82580c84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:55.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.725+0000 7f825ffbd640 1 --2- 172.21.15.105:0/2914888847 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f823c06c8f0 0x7f823c06edb0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:55.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.725+0000 7f825ffbd640 1 --2- 172.21.15.105:0/2914888847 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8258145f00 0x7f8258140e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:55.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.725+0000 7f825ffbd640 1 --2- 172.21.15.105:0/2914888847 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f825806e3c0 0x7f82581f6150 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:55.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.725+0000 7f825ffbd640 1 -- 172.21.15.105:0/2914888847 >> 172.21.15.105:0/2914888847 conn(0x7f8258067d50 msgr2=0x7f825806b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:55.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.725+0000 7f825ffbd640 1 -- 172.21.15.105:0/2914888847 shutdown_connections 2024-04-15T15:19:55.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.725+0000 7f825ffbd640 1 -- 172.21.15.105:0/2914888847 wait complete. 2024-04-15T15:19:55.741 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-15T15:19:55.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.865+0000 7f8899315640 1 Processor -- start 2024-04-15T15:19:55.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.865+0000 7f8899315640 1 -- start start 2024-04-15T15:19:55.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.866+0000 7f8899315640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8894160410 0x7f88940cdd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:55.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.866+0000 7f8899315640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f88940ce240 0x7f88941607f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:55.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.866+0000 7f8899315640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88940b3110 con 0x7f88940d0bf0 2024-04-15T15:19:55.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.866+0000 7f8899315640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f88940d10f0 con 0x7f8894160410 2024-04-15T15:19:55.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.866+0000 7f8899315640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f88940d13d0 con 0x7f88940ce240 2024-04-15T15:19:55.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.866+0000 7f8892ffd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8894160410 0x7f88940cdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:55.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.866+0000 7f8892ffd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8894160410 0x7f88940cdd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46924/0 (socket says 172.21.15.105:46924) 2024-04-15T15:19:55.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.866+0000 7f8892ffd640 1 -- 172.21.15.105:0/2655340972 learned_addr learned my addr 172.21.15.105:0/2655340972 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:55.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.866+0000 7f88937fe640 1 --2- 172.21.15.105:0/2655340972 >> v2:172.21.15.154:3300/0 conn(0x7f88940ce240 0x7f88941607f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:55.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.866+0000 7f8892ffd640 1 -- 172.21.15.105:0/2655340972 >> v2:172.21.15.154:3300/0 conn(0x7f88940ce240 msgr2=0x7f88941607f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:55.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.866+0000 7f8892ffd640 1 --2- 172.21.15.105:0/2655340972 >> v2:172.21.15.154:3300/0 conn(0x7f88940ce240 0x7f88941607f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:55.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.866+0000 7f8892ffd640 1 -- 172.21.15.105:0/2655340972 >> v1:172.21.15.154:6789/0 conn(0x7f88940d0bf0 legacy=0x7f88940b5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:55.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.866+0000 7f8892ffd640 1 -- 172.21.15.105:0/2655340972 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88940cd500 con 0x7f8894160410 2024-04-15T15:19:55.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.866+0000 7f88937fe640 1 --2- 172.21.15.105:0/2655340972 >> v2:172.21.15.154:3300/0 conn(0x7f88940ce240 0x7f88941607f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:55.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.867+0000 7f8892ffd640 1 --2- 172.21.15.105:0/2655340972 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8894160410 0x7f88940cdd00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f889406c5e0 tx=0x7f888402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc3a08d2bb4da652 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:55.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.867+0000 7f88927fc640 1 -- 172.21.15.105:0/2655340972 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8884035020 con 0x7f8894160410 2024-04-15T15:19:55.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.867+0000 7f88927fc640 1 -- 172.21.15.105:0/2655340972 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8884031870 con 0x7f8894160410 2024-04-15T15:19:55.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.867+0000 7f88927fc640 1 -- 172.21.15.105:0/2655340972 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8884031c40 con 0x7f8894160410 2024-04-15T15:19:55.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.868+0000 7f8899315640 1 -- 172.21.15.105:0/2655340972 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8894160410 msgr2=0x7f88940cdd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:55.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.868+0000 7f8899315640 1 --2- 172.21.15.105:0/2655340972 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8894160410 0x7f88940cdd00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f889406c5e0 tx=0x7f888402b910 comp rx=0 tx=0).stop 2024-04-15T15:19:55.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.868+0000 7f8899315640 1 -- 172.21.15.105:0/2655340972 shutdown_connections 2024-04-15T15:19:55.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.868+0000 7f8899315640 1 --2- 172.21.15.105:0/2655340972 >> v2:172.21.15.154:3300/0 conn(0x7f88940ce240 0x7f88941607f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:55.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.868+0000 7f8899315640 1 --2- 172.21.15.105:0/2655340972 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8894160410 0x7f88940cdd00 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:55.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.868+0000 7f8899315640 1 -- 172.21.15.105:0/2655340972 >> 172.21.15.105:0/2655340972 conn(0x7f88940b4100 msgr2=0x7f88940b4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:55.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.868+0000 7f8899315640 1 -- 172.21.15.105:0/2655340972 shutdown_connections 2024-04-15T15:19:55.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.868+0000 7f8899315640 1 -- 172.21.15.105:0/2655340972 wait complete. 2024-04-15T15:19:55.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.869+0000 7f8899315640 1 Processor -- start 2024-04-15T15:19:55.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.869+0000 7f8899315640 1 -- start start 2024-04-15T15:19:55.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.870+0000 7f8899315640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88940ce240 0x7f889414c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:55.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.870+0000 7f8899315640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88940d0bf0 0x7f889415cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:55.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.870+0000 7f8899315640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8894160410 0x7f88940b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:55.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.870+0000 7f8899315640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f88940cd120 con 0x7f8894160410 2024-04-15T15:19:55.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.870+0000 7f8899315640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f88940b3540 con 0x7f88940ce240 2024-04-15T15:19:55.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.870+0000 7f8899315640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f88940cd2a0 con 0x7f88940d0bf0 2024-04-15T15:19:55.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.870+0000 7f88937fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8894160410 0x7f88940b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:55.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.870+0000 7f88937fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8894160410 0x7f88940b1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46928/0 (socket says 172.21.15.105:46928) 2024-04-15T15:19:55.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.870+0000 7f88937fe640 1 -- 172.21.15.105:0/385139543 learned_addr learned my addr 172.21.15.105:0/385139543 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:55.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.870+0000 7f8892ffd640 1 --2- 172.21.15.105:0/385139543 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88940ce240 0x7f889414c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:55.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.870+0000 7f888bfff640 1 --2- 172.21.15.105:0/385139543 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88940d0bf0 0x7f889415cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:55.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.870+0000 7f88937fe640 1 -- 172.21.15.105:0/385139543 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88940d0bf0 msgr2=0x7f889415cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:55.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.870+0000 7f88937fe640 1 --2- 172.21.15.105:0/385139543 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88940d0bf0 0x7f889415cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:55.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.870+0000 7f88937fe640 1 -- 172.21.15.105:0/385139543 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88940ce240 msgr2=0x7f889414c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:55.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.870+0000 7f88937fe640 1 --2- 172.21.15.105:0/385139543 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88940ce240 0x7f889414c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:55.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.870+0000 7f88937fe640 1 -- 172.21.15.105:0/385139543 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88940b0a50 con 0x7f8894160410 2024-04-15T15:19:55.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.870+0000 7f8892ffd640 1 --2- 172.21.15.105:0/385139543 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88940ce240 0x7f889414c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:19:55.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.870+0000 7f88937fe640 1 --2- 172.21.15.105:0/385139543 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8894160410 0x7f88940b1af0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f888c00cd80 tx=0x7f888c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:55.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.871+0000 7f888b7fe640 1 -- 172.21.15.105:0/385139543 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888c00f680 con 0x7f8894160410 2024-04-15T15:19:55.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.871+0000 7f888b7fe640 1 -- 172.21.15.105:0/385139543 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f888c00f840 con 0x7f8894160410 2024-04-15T15:19:55.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.871+0000 7f888b7fe640 1 -- 172.21.15.105:0/385139543 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888c00fc30 con 0x7f8894160410 2024-04-15T15:19:55.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.871+0000 7f8899315640 1 -- 172.21.15.105:0/385139543 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88940b0d30 con 0x7f8894160410 2024-04-15T15:19:55.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.871+0000 7f8899315640 1 -- 172.21.15.105:0/385139543 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88940b12f0 con 0x7f8894160410 2024-04-15T15:19:55.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.872+0000 7f8899315640 1 -- 172.21.15.105:0/385139543 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8894069150 con 0x7f8894160410 2024-04-15T15:19:55.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.872+0000 7f888b7fe640 1 -- 172.21.15.105:0/385139543 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f888c0213f0 con 0x7f8894160410 2024-04-15T15:19:55.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.873+0000 7f888b7fe640 1 --2- 172.21.15.105:0/385139543 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f887406c8f0 0x7f887406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:55.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.873+0000 7f8892ffd640 1 --2- 172.21.15.105:0/385139543 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f887406c8f0 0x7f887406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:55.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.873+0000 7f888b7fe640 1 -- 172.21.15.105:0/385139543 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f888c08c910 con 0x7f8894160410 2024-04-15T15:19:55.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.873+0000 7f8892ffd640 1 --2- 172.21.15.105:0/385139543 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f887406c8f0 0x7f887406edb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8884036000 tx=0x7f888403c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:55.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:55.877+0000 7f888b7fe640 1 -- 172.21.15.105:0/385139543 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f888c014030 con 0x7f8894160410 2024-04-15T15:19:56.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.060+0000 7f8899315640 1 -- 172.21.15.105:0/385139543 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f88940b4d20 con 0x7f8894160410 2024-04-15T15:19:56.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.061+0000 7f888b7fe640 1 -- 172.21.15.105:0/385139543 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f888c05a6e0 con 0x7f8894160410 2024-04-15T15:19:56.063 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-04-15T15:19:56.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.063+0000 7f8899315640 1 -- 172.21.15.105:0/385139543 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f887406c8f0 msgr2=0x7f887406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:56.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.063+0000 7f8899315640 1 --2- 172.21.15.105:0/385139543 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f887406c8f0 0x7f887406edb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8884036000 tx=0x7f888403c040 comp rx=0 tx=0).stop 2024-04-15T15:19:56.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.063+0000 7f8899315640 1 -- 172.21.15.105:0/385139543 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8894160410 msgr2=0x7f88940b1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:56.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.063+0000 7f8899315640 1 --2- 172.21.15.105:0/385139543 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8894160410 0x7f88940b1af0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f888c00cd80 tx=0x7f888c00a650 comp rx=0 tx=0).stop 2024-04-15T15:19:56.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.063+0000 7f8899315640 1 -- 172.21.15.105:0/385139543 shutdown_connections 2024-04-15T15:19:56.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.063+0000 7f8899315640 1 --2- 172.21.15.105:0/385139543 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88940d0bf0 0x7f889415cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:56.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.063+0000 7f8899315640 1 --2- 172.21.15.105:0/385139543 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f887406c8f0 0x7f887406edb0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:56.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.063+0000 7f8899315640 1 --2- 172.21.15.105:0/385139543 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8894160410 0x7f88940b1af0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:56.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.063+0000 7f8899315640 1 --2- 172.21.15.105:0/385139543 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88940ce240 0x7f889414c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:56.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.063+0000 7f8899315640 1 -- 172.21.15.105:0/385139543 >> 172.21.15.105:0/385139543 conn(0x7f88940b4100 msgr2=0x7f88940cef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:56.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.063+0000 7f8899315640 1 -- 172.21.15.105:0/385139543 shutdown_connections 2024-04-15T15:19:56.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.063+0000 7f8899315640 1 -- 172.21.15.105:0/385139543 wait complete. 2024-04-15T15:19:56.076 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-15T15:19:56.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.182+0000 7f827f09a640 1 Processor -- start 2024-04-15T15:19:56.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.182+0000 7f827f09a640 1 -- start start 2024-04-15T15:19:56.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.183+0000 7f827f09a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f827815cc50 0x7f8278070050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:56.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.183+0000 7f827f09a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82780cd4e0 con 0x7f827809baa0 2024-04-15T15:19:56.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.183+0000 7f827f09a640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82780cd7c0 con 0x7f827809aa40 2024-04-15T15:19:56.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.183+0000 7f827f09a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f82780cdaa0 con 0x7f827815cc50 2024-04-15T15:19:56.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.183+0000 7f827d637640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f827809baa0 0x7f82780d0480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:57430/0 (socket says 172.21.15.105:57430) 2024-04-15T15:19:56.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.183+0000 7f827d637640 1 -- 172.21.15.105:0/2995111018 learned_addr learned my addr 172.21.15.105:0/2995111018 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:19:56.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.183+0000 7f82777fe640 1 -- 172.21.15.105:0/2995111018 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2162432651 0 0) 0x7f82780cd4e0 con 0x7f827809baa0 2024-04-15T15:19:56.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.183+0000 7f82777fe640 1 -- 172.21.15.105:0/2995111018 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8268003520 con 0x7f827809baa0 2024-04-15T15:19:56.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.183+0000 7f8277fff640 1 --2- 172.21.15.105:0/2995111018 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f827815cc50 0x7f8278070050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:56.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.184+0000 7f82777fe640 1 -- 172.21.15.105:0/2995111018 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1791751988 0 0) 0x7f8268003520 con 0x7f827809baa0 2024-04-15T15:19:56.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.184+0000 7f82777fe640 1 -- 172.21.15.105:0/2995111018 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f827815cc50 msgr2=0x7f8278070050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:56.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.184+0000 7f82777fe640 1 --2- 172.21.15.105:0/2995111018 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f827815cc50 0x7f8278070050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:56.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.184+0000 7f82777fe640 1 -- 172.21.15.105:0/2995111018 >> v1:172.21.15.154:6789/0 conn(0x7f827809aa40 legacy=0x7f827809ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:56.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.184+0000 7f82777fe640 1 -- 172.21.15.105:0/2995111018 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82780cdd80 con 0x7f827809baa0 2024-04-15T15:19:56.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.184+0000 7f82777fe640 1 -- 172.21.15.105:0/2995111018 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7f82700019f0 con 0x7f827809baa0 2024-04-15T15:19:56.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.184+0000 7f8277fff640 1 --2- 172.21.15.105:0/2995111018 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f827815cc50 0x7f8278070050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:19:56.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.184+0000 7f82777fe640 1 -- 172.21.15.105:0/2995111018 >> v1:172.21.15.105:6789/0 conn(0x7f827809baa0 legacy=0x7f82780d0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:56.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.184+0000 7f82777fe640 1 --2- 172.21.15.105:0/2995111018 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82680039e0 0x7f8268003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:56.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.184+0000 7f82777fe640 1 --2- 172.21.15.105:0/2995111018 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8268004800 0x7f8268004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:56.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.184+0000 7f82777fe640 1 --2- 172.21.15.105:0/2995111018 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82680051c0 0x7f826800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:56.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.184+0000 7f82777fe640 1 -- 172.21.15.105:0/2995111018 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f82780cdaa0 con 0x7f8268004800 2024-04-15T15:19:56.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.184+0000 7f82777fe640 1 -- 172.21.15.105:0/2995111018 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f826800bf00 con 0x7f82680039e0 2024-04-15T15:19:56.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.184+0000 7f82777fe640 1 -- 172.21.15.105:0/2995111018 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f826800c0b0 con 0x7f82680051c0 2024-04-15T15:19:56.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.184+0000 7f8277fff640 1 --2- 172.21.15.105:0/2995111018 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8268004800 0x7f8268004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:56.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.184+0000 7f827ce36640 1 --2- 172.21.15.105:0/2995111018 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82680039e0 0x7f8268003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:56.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.184+0000 7f827d637640 1 --2- 172.21.15.105:0/2995111018 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82680051c0 0x7f826800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:56.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.184+0000 7f8277fff640 1 -- 172.21.15.105:0/2995111018 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82680051c0 msgr2=0x7f826800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:56.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.184+0000 7f8277fff640 1 --2- 172.21.15.105:0/2995111018 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82680051c0 0x7f826800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:56.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.184+0000 7f8277fff640 1 -- 172.21.15.105:0/2995111018 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82680039e0 msgr2=0x7f8268003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:56.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.184+0000 7f8277fff640 1 --2- 172.21.15.105:0/2995111018 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82680039e0 0x7f8268003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:56.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.184+0000 7f8277fff640 1 -- 172.21.15.105:0/2995111018 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8268003520 con 0x7f8268004800 2024-04-15T15:19:56.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.184+0000 7f827d637640 1 --2- 172.21.15.105:0/2995111018 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82680051c0 0x7f826800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:19:56.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.185+0000 7f8277fff640 1 --2- 172.21.15.105:0/2995111018 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8268004800 0x7f8268004c80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f82640098a0 tx=0x7f8268005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61afb980a8ba0d17 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:56.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.185+0000 7f82777fe640 1 -- 172.21.15.105:0/2995111018 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826400f040 con 0x7f8268004800 2024-04-15T15:19:56.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.185+0000 7f82777fe640 1 -- 172.21.15.105:0/2995111018 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8264004230 con 0x7f8268004800 2024-04-15T15:19:56.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.185+0000 7f82777fe640 1 -- 172.21.15.105:0/2995111018 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8264003610 con 0x7f8268004800 2024-04-15T15:19:56.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.186+0000 7f827f09a640 1 -- 172.21.15.105:0/2995111018 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8268004800 msgr2=0x7f8268004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:56.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.186+0000 7f827f09a640 1 --2- 172.21.15.105:0/2995111018 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8268004800 0x7f8268004c80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f82640098a0 tx=0x7f8268005830 comp rx=0 tx=0).stop 2024-04-15T15:19:56.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.186+0000 7f827d637640 1 -- 172.21.15.105:0/2995111018 reap_dead start 2024-04-15T15:19:56.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.186+0000 7f827f09a640 1 -- 172.21.15.105:0/2995111018 shutdown_connections 2024-04-15T15:19:56.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.186+0000 7f827f09a640 1 -- 172.21.15.105:0/2995111018 >> 172.21.15.105:0/2995111018 conn(0x7f82780b4070 msgr2=0x7f82780b4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:56.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.186+0000 7f827f09a640 1 -- 172.21.15.105:0/2995111018 shutdown_connections 2024-04-15T15:19:56.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.186+0000 7f827f09a640 1 -- 172.21.15.105:0/2995111018 wait complete. 2024-04-15T15:19:56.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.187+0000 7f827f09a640 1 Processor -- start 2024-04-15T15:19:56.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.187+0000 7f827f09a640 1 -- start start 2024-04-15T15:19:56.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.188+0000 7f827f09a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8268004800 0x7f827806dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:56.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.188+0000 7f827f09a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82780ca1d0 0x7f8278156830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:56.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.188+0000 7f827f09a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82780cab90 0x7f82780b3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:56.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.188+0000 7f827f09a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f82780ce420 con 0x7f82780cab90 2024-04-15T15:19:56.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.188+0000 7f827f09a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f82780ce2a0 con 0x7f8268004800 2024-04-15T15:19:56.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.188+0000 7f827f09a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f8278156d70 con 0x7f82780ca1d0 2024-04-15T15:19:56.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.188+0000 7f827d637640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82780cab90 0x7f82780b3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:56.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.188+0000 7f827d637640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82780cab90 0x7f82780b3310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46956/0 (socket says 172.21.15.105:46956) 2024-04-15T15:19:56.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.188+0000 7f827d637640 1 -- 172.21.15.105:0/3026669877 learned_addr learned my addr 172.21.15.105:0/3026669877 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:56.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.188+0000 7f8277fff640 1 --2- 172.21.15.105:0/3026669877 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82780ca1d0 0x7f8278156830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:56.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.188+0000 7f827ce36640 1 --2- 172.21.15.105:0/3026669877 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8268004800 0x7f827806dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:56.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.188+0000 7f827d637640 1 -- 172.21.15.105:0/3026669877 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82780ca1d0 msgr2=0x7f8278156830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:56.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.188+0000 7f827d637640 1 --2- 172.21.15.105:0/3026669877 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82780ca1d0 0x7f8278156830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:56.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.188+0000 7f827d637640 1 -- 172.21.15.105:0/3026669877 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8268004800 msgr2=0x7f827806dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:56.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.188+0000 7f827d637640 1 --2- 172.21.15.105:0/3026669877 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8268004800 0x7f827806dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:56.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.188+0000 7f827d637640 1 -- 172.21.15.105:0/3026669877 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82780cb3a0 con 0x7f82780cab90 2024-04-15T15:19:56.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.189+0000 7f827d637640 1 --2- 172.21.15.105:0/3026669877 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82780cab90 0x7f82780b3310 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f82780a0680 tx=0x7f827000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:56.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.189+0000 7f82757fa640 1 -- 172.21.15.105:0/3026669877 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827000edd0 con 0x7f82780cab90 2024-04-15T15:19:56.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.189+0000 7f82757fa640 1 -- 172.21.15.105:0/3026669877 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8270014430 con 0x7f82780cab90 2024-04-15T15:19:56.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.189+0000 7f82757fa640 1 -- 172.21.15.105:0/3026669877 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8270014820 con 0x7f82780cab90 2024-04-15T15:19:56.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.189+0000 7f827f09a640 1 -- 172.21.15.105:0/3026669877 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82780c96f0 con 0x7f82780cab90 2024-04-15T15:19:56.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.189+0000 7f827f09a640 1 -- 172.21.15.105:0/3026669877 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82780ca650 con 0x7f82780cab90 2024-04-15T15:19:56.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.190+0000 7f827f09a640 1 -- 172.21.15.105:0/3026669877 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8278069150 con 0x7f82780cab90 2024-04-15T15:19:56.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.191+0000 7f82757fa640 1 -- 172.21.15.105:0/3026669877 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f82700149c0 con 0x7f82780cab90 2024-04-15T15:19:56.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.191+0000 7f82757fa640 1 --2- 172.21.15.105:0/3026669877 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f825c06c920 0x7f825c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:56.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.191+0000 7f82757fa640 1 -- 172.21.15.105:0/3026669877 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f827008b9d0 con 0x7f82780cab90 2024-04-15T15:19:56.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.191+0000 7f827ce36640 1 --2- 172.21.15.105:0/3026669877 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f825c06c920 0x7f825c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:56.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.191+0000 7f827ce36640 1 --2- 172.21.15.105:0/3026669877 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f825c06c920 0x7f825c06ede0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f827815a6d0 tx=0x7f826c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:56.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.195+0000 7f82757fa640 1 -- 172.21.15.105:0/3026669877 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8270059e00 con 0x7f82780cab90 2024-04-15T15:19:56.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.372+0000 7f827f09a640 1 -- 172.21.15.105:0/3026669877 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8278070f60 con 0x7f82780cab90 2024-04-15T15:19:56.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.372+0000 7f82757fa640 1 -- 172.21.15.105:0/3026669877 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f8270059700 con 0x7f82780cab90 2024-04-15T15:19:56.374 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-04-15T15:19:56.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.374+0000 7f827f09a640 1 -- 172.21.15.105:0/3026669877 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f825c06c920 msgr2=0x7f825c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:56.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.374+0000 7f827f09a640 1 --2- 172.21.15.105:0/3026669877 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f825c06c920 0x7f825c06ede0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f827815a6d0 tx=0x7f826c009040 comp rx=0 tx=0).stop 2024-04-15T15:19:56.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.374+0000 7f827f09a640 1 -- 172.21.15.105:0/3026669877 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82780cab90 msgr2=0x7f82780b3310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:56.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.374+0000 7f827f09a640 1 --2- 172.21.15.105:0/3026669877 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82780cab90 0x7f82780b3310 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f82780a0680 tx=0x7f827000e530 comp rx=0 tx=0).stop 2024-04-15T15:19:56.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.374+0000 7f827f09a640 1 -- 172.21.15.105:0/3026669877 shutdown_connections 2024-04-15T15:19:56.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.374+0000 7f827f09a640 1 --2- 172.21.15.105:0/3026669877 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82780ca1d0 0x7f8278156830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:56.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.374+0000 7f827f09a640 1 --2- 172.21.15.105:0/3026669877 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f825c06c920 0x7f825c06ede0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:56.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.374+0000 7f827f09a640 1 --2- 172.21.15.105:0/3026669877 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82780cab90 0x7f82780b3310 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:56.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.374+0000 7f827f09a640 1 --2- 172.21.15.105:0/3026669877 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8268004800 0x7f827806dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:56.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.374+0000 7f827f09a640 1 -- 172.21.15.105:0/3026669877 >> 172.21.15.105:0/3026669877 conn(0x7f82780b4070 msgr2=0x7f827815a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:56.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.374+0000 7f827f09a640 1 -- 172.21.15.105:0/3026669877 shutdown_connections 2024-04-15T15:19:56.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.374+0000 7f827f09a640 1 -- 172.21.15.105:0/3026669877 wait complete. 2024-04-15T15:19:56.387 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-15T15:19:56.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.490+0000 7f5b4aa88640 1 Processor -- start 2024-04-15T15:19:56.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.490+0000 7f5b4aa88640 1 -- start start 2024-04-15T15:19:56.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.491+0000 7f5b4aa88640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5b4415e940 0x7f5b440b1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:56.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.491+0000 7f5b4aa88640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5b440b2430 0x7f5b440cef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:56.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.491+0000 7f5b4aa88640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5b4409aa40 0x7f5b440cf480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:56.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.491+0000 7f5b4aa88640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f5b440b2120 con 0x7f5b440b2430 2024-04-15T15:19:56.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.491+0000 7f5b4aa88640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f5b4409afe0 con 0x7f5b4415e940 2024-04-15T15:19:56.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.491+0000 7f5b4aa88640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f5b4409b2c0 con 0x7f5b4409aa40 2024-04-15T15:19:56.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.491+0000 7f5b43fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5b440b2430 0x7f5b440cef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:56.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.491+0000 7f5b43fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5b440b2430 0x7f5b440cef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46960/0 (socket says 172.21.15.105:46960) 2024-04-15T15:19:56.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.491+0000 7f5b43fff640 1 -- 172.21.15.105:0/1788104977 learned_addr learned my addr 172.21.15.105:0/1788104977 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:56.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.491+0000 7f5b48824640 1 --2- 172.21.15.105:0/1788104977 >> v2:172.21.15.154:3300/0 conn(0x7f5b4415e940 0x7f5b440b1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:56.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.491+0000 7f5b49025640 1 --2- 172.21.15.105:0/1788104977 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5b4409aa40 0x7f5b440cf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:56.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.491+0000 7f5b43fff640 1 -- 172.21.15.105:0/1788104977 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5b4409aa40 msgr2=0x7f5b440cf480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:56.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.491+0000 7f5b43fff640 1 --2- 172.21.15.105:0/1788104977 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5b4409aa40 0x7f5b440cf480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:56.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.491+0000 7f5b43fff640 1 -- 172.21.15.105:0/1788104977 >> v2:172.21.15.154:3300/0 conn(0x7f5b4415e940 msgr2=0x7f5b440b1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:56.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.491+0000 7f5b43fff640 1 --2- 172.21.15.105:0/1788104977 >> v2:172.21.15.154:3300/0 conn(0x7f5b4415e940 0x7f5b440b1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:56.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.491+0000 7f5b43fff640 1 -- 172.21.15.105:0/1788104977 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b4409b5a0 con 0x7f5b440b2430 2024-04-15T15:19:56.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.491+0000 7f5b48824640 1 --2- 172.21.15.105:0/1788104977 >> v2:172.21.15.154:3300/0 conn(0x7f5b4415e940 0x7f5b440b1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:19:56.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.492+0000 7f5b43fff640 1 --2- 172.21.15.105:0/1788104977 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b440b2430 0x7f5b440cef40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5b4409bbe0 tx=0x7f5b3002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a78e9c74696700a server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:56.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.492+0000 7f5b437fe640 1 -- 172.21.15.105:0/1788104977 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b30035020 con 0x7f5b440b2430 2024-04-15T15:19:56.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.492+0000 7f5b437fe640 1 -- 172.21.15.105:0/1788104977 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b30031870 con 0x7f5b440b2430 2024-04-15T15:19:56.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.492+0000 7f5b437fe640 1 -- 172.21.15.105:0/1788104977 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b30031c40 con 0x7f5b440b2430 2024-04-15T15:19:56.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.493+0000 7f5b4aa88640 1 -- 172.21.15.105:0/1788104977 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b440b2430 msgr2=0x7f5b440cef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:56.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.493+0000 7f5b4aa88640 1 --2- 172.21.15.105:0/1788104977 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b440b2430 0x7f5b440cef40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5b4409bbe0 tx=0x7f5b3002b910 comp rx=0 tx=0).stop 2024-04-15T15:19:56.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.493+0000 7f5b4aa88640 1 -- 172.21.15.105:0/1788104977 shutdown_connections 2024-04-15T15:19:56.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.493+0000 7f5b4aa88640 1 --2- 172.21.15.105:0/1788104977 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5b4409aa40 0x7f5b440cf480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:56.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.493+0000 7f5b4aa88640 1 --2- 172.21.15.105:0/1788104977 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b440b2430 0x7f5b440cef40 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:56.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.493+0000 7f5b4aa88640 1 --2- 172.21.15.105:0/1788104977 >> v2:172.21.15.154:3300/0 conn(0x7f5b4415e940 0x7f5b440b1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:56.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.493+0000 7f5b4aa88640 1 -- 172.21.15.105:0/1788104977 >> 172.21.15.105:0/1788104977 conn(0x7f5b440d25a0 msgr2=0x7f5b440d29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:56.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.493+0000 7f5b4aa88640 1 -- 172.21.15.105:0/1788104977 shutdown_connections 2024-04-15T15:19:56.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.493+0000 7f5b4aa88640 1 -- 172.21.15.105:0/1788104977 wait complete. 2024-04-15T15:19:56.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.494+0000 7f5b4aa88640 1 Processor -- start 2024-04-15T15:19:56.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.494+0000 7f5b4aa88640 1 -- start start 2024-04-15T15:19:56.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.495+0000 7f5b4aa88640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b4409aa40 0x7f5b440b53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:56.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.495+0000 7f5b4aa88640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5b440b2430 0x7f5b441f8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:56.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.495+0000 7f5b4aa88640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b4415e940 0x7f5b44158f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:56.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.495+0000 7f5b4aa88640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5b440d0730 con 0x7f5b4415e940 2024-04-15T15:19:56.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.495+0000 7f5b4aa88640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4415ed20 con 0x7f5b4409aa40 2024-04-15T15:19:56.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.495+0000 7f5b4aa88640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f5b440d0cc0 con 0x7f5b440b2430 2024-04-15T15:19:56.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.495+0000 7f5b49025640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b4415e940 0x7f5b44158f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:56.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.495+0000 7f5b49025640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b4415e940 0x7f5b44158f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46972/0 (socket says 172.21.15.105:46972) 2024-04-15T15:19:56.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.495+0000 7f5b49025640 1 -- 172.21.15.105:0/3620985931 learned_addr learned my addr 172.21.15.105:0/3620985931 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:56.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.495+0000 7f5b48824640 1 --2- 172.21.15.105:0/3620985931 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b4409aa40 0x7f5b440b53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:56.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.495+0000 7f5b43fff640 1 --2- 172.21.15.105:0/3620985931 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5b440b2430 0x7f5b441f8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:56.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.496+0000 7f5b49025640 1 -- 172.21.15.105:0/3620985931 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5b440b2430 msgr2=0x7f5b441f8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:56.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.496+0000 7f5b49025640 1 --2- 172.21.15.105:0/3620985931 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5b440b2430 0x7f5b441f8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:56.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.496+0000 7f5b49025640 1 -- 172.21.15.105:0/3620985931 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b4409aa40 msgr2=0x7f5b440b53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:56.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.496+0000 7f5b49025640 1 --2- 172.21.15.105:0/3620985931 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b4409aa40 0x7f5b440b53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:56.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.496+0000 7f5b49025640 1 -- 172.21.15.105:0/3620985931 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b440b7a20 con 0x7f5b4415e940 2024-04-15T15:19:56.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.496+0000 7f5b49025640 1 --2- 172.21.15.105:0/3620985931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b4415e940 0x7f5b44158f70 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c007e90 tx=0x7f5b3c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:56.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.496+0000 7f5b417fa640 1 -- 172.21.15.105:0/3620985931 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c014420 con 0x7f5b4415e940 2024-04-15T15:19:56.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.496+0000 7f5b417fa640 1 -- 172.21.15.105:0/3620985931 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b3c019070 con 0x7f5b4415e940 2024-04-15T15:19:56.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.496+0000 7f5b417fa640 1 -- 172.21.15.105:0/3620985931 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c014820 con 0x7f5b4415e940 2024-04-15T15:19:56.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.496+0000 7f5b4aa88640 1 -- 172.21.15.105:0/3620985931 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b440b7cc0 con 0x7f5b4415e940 2024-04-15T15:19:56.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.496+0000 7f5b4aa88640 1 -- 172.21.15.105:0/3620985931 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b440b8280 con 0x7f5b4415e940 2024-04-15T15:19:56.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.497+0000 7f5b4aa88640 1 -- 172.21.15.105:0/3620985931 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b44069150 con 0x7f5b4415e940 2024-04-15T15:19:56.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.498+0000 7f5b417fa640 1 -- 172.21.15.105:0/3620985931 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5b3c0149c0 con 0x7f5b4415e940 2024-04-15T15:19:56.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.498+0000 7f5b417fa640 1 --2- 172.21.15.105:0/3620985931 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f5b2806c8f0 0x7f5b2806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:56.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.498+0000 7f5b48824640 1 --2- 172.21.15.105:0/3620985931 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f5b2806c8f0 0x7f5b2806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:56.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.498+0000 7f5b417fa640 1 -- 172.21.15.105:0/3620985931 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5b3c08c270 con 0x7f5b4415e940 2024-04-15T15:19:56.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.499+0000 7f5b48824640 1 --2- 172.21.15.105:0/3620985931 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f5b2806c8f0 0x7f5b2806edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5b38006e80 tx=0x7f5b38008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:56.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.503+0000 7f5b417fa640 1 -- 172.21.15.105:0/3620985931 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5b3c05a050 con 0x7f5b4415e940 2024-04-15T15:19:56.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.695+0000 7f5b4aa88640 1 -- 172.21.15.105:0/3620985931 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5b4406d100 con 0x7f5b4415e940 2024-04-15T15:19:56.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.695+0000 7f5b417fa640 1 -- 172.21.15.105:0/3620985931 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7f5b3c05e350 con 0x7f5b4415e940 2024-04-15T15:19:56.697 INFO:teuthology.orchestra.run.smithi105.stderr:listed 0 entries 2024-04-15T15:19:56.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.697+0000 7f5b4aa88640 1 -- 172.21.15.105:0/3620985931 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f5b2806c8f0 msgr2=0x7f5b2806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:56.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.697+0000 7f5b4aa88640 1 --2- 172.21.15.105:0/3620985931 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f5b2806c8f0 0x7f5b2806edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5b38006e80 tx=0x7f5b38008040 comp rx=0 tx=0).stop 2024-04-15T15:19:56.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.697+0000 7f5b4aa88640 1 -- 172.21.15.105:0/3620985931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b4415e940 msgr2=0x7f5b44158f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:56.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.697+0000 7f5b4aa88640 1 --2- 172.21.15.105:0/3620985931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b4415e940 0x7f5b44158f70 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c007e90 tx=0x7f5b3c00e4b0 comp rx=0 tx=0).stop 2024-04-15T15:19:56.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.697+0000 7f5b4aa88640 1 -- 172.21.15.105:0/3620985931 shutdown_connections 2024-04-15T15:19:56.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.697+0000 7f5b4aa88640 1 --2- 172.21.15.105:0/3620985931 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5b440b2430 0x7f5b441f8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:56.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.697+0000 7f5b4aa88640 1 --2- 172.21.15.105:0/3620985931 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f5b2806c8f0 0x7f5b2806edb0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:56.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.697+0000 7f5b4aa88640 1 --2- 172.21.15.105:0/3620985931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b4415e940 0x7f5b44158f70 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:56.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.697+0000 7f5b4aa88640 1 --2- 172.21.15.105:0/3620985931 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b4409aa40 0x7f5b440b53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:56.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.697+0000 7f5b4aa88640 1 -- 172.21.15.105:0/3620985931 >> 172.21.15.105:0/3620985931 conn(0x7f5b440d25a0 msgr2=0x7f5b4415ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:56.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.697+0000 7f5b4aa88640 1 -- 172.21.15.105:0/3620985931 shutdown_connections 2024-04-15T15:19:56.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.697+0000 7f5b4aa88640 1 -- 172.21.15.105:0/3620985931 wait complete. 2024-04-15T15:19:56.710 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-04-15T15:19:56.710 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:19:56.728 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:19:56.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.869+0000 7fc818260640 1 Processor -- start 2024-04-15T15:19:56.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.869+0000 7fc818260640 1 -- start start 2024-04-15T15:19:56.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.870+0000 7fc818260640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc810147730 0x7fc8100b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:56.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.870+0000 7fc818260640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc8100af6b0 0x7fc8100725f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:56.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.870+0000 7fc818260640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8100afcd0 con 0x7fc810072160 2024-04-15T15:19:56.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.870+0000 7fc818260640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fc8100affb0 con 0x7fc8100af6b0 2024-04-15T15:19:56.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.870+0000 7fc818260640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fc81006e130 con 0x7fc810147730 2024-04-15T15:19:56.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.870+0000 7fc8167fd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc8100af6b0 0x7fc8100725f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:56.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.870+0000 7fc8167fd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc8100af6b0 0x7fc8100725f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46980/0 (socket says 172.21.15.105:46980) 2024-04-15T15:19:56.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.870+0000 7fc8167fd640 1 -- 172.21.15.105:0/280910270 learned_addr learned my addr 172.21.15.105:0/280910270 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:56.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.870+0000 7fc815ffc640 1 --2- 172.21.15.105:0/280910270 >> v2:172.21.15.154:3300/0 conn(0x7fc810147730 0x7fc8100b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:56.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.870+0000 7fc8167fd640 1 -- 172.21.15.105:0/280910270 >> v2:172.21.15.154:3300/0 conn(0x7fc810147730 msgr2=0x7fc8100b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:56.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.870+0000 7fc8167fd640 1 --2- 172.21.15.105:0/280910270 >> v2:172.21.15.154:3300/0 conn(0x7fc810147730 0x7fc8100b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:56.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.870+0000 7fc8167fd640 1 -- 172.21.15.105:0/280910270 >> v1:172.21.15.154:6789/0 conn(0x7fc810072160 legacy=0x7fc8100aefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:56.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.870+0000 7fc8167fd640 1 -- 172.21.15.105:0/280910270 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc81006e410 con 0x7fc8100af6b0 2024-04-15T15:19:56.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.871+0000 7fc8167fd640 1 --2- 172.21.15.105:0/280910270 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8100af6b0 0x7fc8100725f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc800009e50 tx=0x7fc80002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=354cd0be37068c8a server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:56.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.871+0000 7fc814ffa640 1 -- 172.21.15.105:0/280910270 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc80002ca80 con 0x7fc8100af6b0 2024-04-15T15:19:56.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.871+0000 7fc814ffa640 1 -- 172.21.15.105:0/280910270 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc80002cc20 con 0x7fc8100af6b0 2024-04-15T15:19:56.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.871+0000 7fc814ffa640 1 -- 172.21.15.105:0/280910270 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8000343f0 con 0x7fc8100af6b0 2024-04-15T15:19:56.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.872+0000 7fc818260640 1 -- 172.21.15.105:0/280910270 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8100af6b0 msgr2=0x7fc8100725f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:56.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.872+0000 7fc818260640 1 --2- 172.21.15.105:0/280910270 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8100af6b0 0x7fc8100725f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc800009e50 tx=0x7fc80002bee0 comp rx=0 tx=0).stop 2024-04-15T15:19:56.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.872+0000 7fc818260640 1 -- 172.21.15.105:0/280910270 shutdown_connections 2024-04-15T15:19:56.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.872+0000 7fc818260640 1 --2- 172.21.15.105:0/280910270 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8100af6b0 0x7fc8100725f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:56.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.872+0000 7fc818260640 1 --2- 172.21.15.105:0/280910270 >> v2:172.21.15.154:3300/0 conn(0x7fc810147730 0x7fc8100b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:56.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.872+0000 7fc818260640 1 -- 172.21.15.105:0/280910270 >> 172.21.15.105:0/280910270 conn(0x7fc81006b9c0 msgr2=0x7fc81006bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:56.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.872+0000 7fc818260640 1 -- 172.21.15.105:0/280910270 shutdown_connections 2024-04-15T15:19:56.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.872+0000 7fc818260640 1 -- 172.21.15.105:0/280910270 wait complete. 2024-04-15T15:19:56.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.873+0000 7fc818260640 1 Processor -- start 2024-04-15T15:19:56.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.873+0000 7fc818260640 1 -- start start 2024-04-15T15:19:56.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.874+0000 7fc818260640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc810072160 0x7fc81015e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:56.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.874+0000 7fc818260640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc8100af6b0 0x7fc8101385f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:56.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.874+0000 7fc818260640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc810147730 0x7fc8100c8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:56.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.874+0000 7fc818260640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc8100b3e80 con 0x7fc810072160 2024-04-15T15:19:56.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.874+0000 7fc818260640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc810147b10 con 0x7fc810147730 2024-04-15T15:19:56.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.874+0000 7fc818260640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fc81006ec10 con 0x7fc8100af6b0 2024-04-15T15:19:56.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.874+0000 7fc815ffc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc810072160 0x7fc81015e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:56.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.874+0000 7fc815ffc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc810072160 0x7fc81015e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46984/0 (socket says 172.21.15.105:46984) 2024-04-15T15:19:56.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.874+0000 7fc815ffc640 1 -- 172.21.15.105:0/2729013367 learned_addr learned my addr 172.21.15.105:0/2729013367 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:56.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.874+0000 7fc815ffc640 1 -- 172.21.15.105:0/2729013367 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc8100af6b0 msgr2=0x7fc8101385f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:56.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.874+0000 7fc815ffc640 1 --2- 172.21.15.105:0/2729013367 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc8100af6b0 0x7fc8101385f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:56.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.874+0000 7fc815ffc640 1 -- 172.21.15.105:0/2729013367 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc810147730 msgr2=0x7fc8100c8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:56.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.874+0000 7fc815ffc640 1 --2- 172.21.15.105:0/2729013367 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc810147730 0x7fc8100c8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:56.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.874+0000 7fc815ffc640 1 -- 172.21.15.105:0/2729013367 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc810144c50 con 0x7fc810072160 2024-04-15T15:19:56.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.874+0000 7fc815ffc640 1 --2- 172.21.15.105:0/2729013367 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc810072160 0x7fc81015e490 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc80800d5f0 tx=0x7fc80800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:56.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.874+0000 7fc80e7fc640 1 -- 172.21.15.105:0/2729013367 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8080125f0 con 0x7fc810072160 2024-04-15T15:19:56.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.874+0000 7fc80e7fc640 1 -- 172.21.15.105:0/2729013367 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc808019070 con 0x7fc810072160 2024-04-15T15:19:56.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.874+0000 7fc80e7fc640 1 -- 172.21.15.105:0/2729013367 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8080129d0 con 0x7fc810072160 2024-04-15T15:19:56.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.874+0000 7fc818260640 1 -- 172.21.15.105:0/2729013367 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc810138b30 con 0x7fc810072160 2024-04-15T15:19:56.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.874+0000 7fc818260640 1 -- 172.21.15.105:0/2729013367 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc810144ef0 con 0x7fc810072160 2024-04-15T15:19:56.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.875+0000 7fc818260640 1 -- 172.21.15.105:0/2729013367 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc810068a40 con 0x7fc810072160 2024-04-15T15:19:56.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.876+0000 7fc80e7fc640 1 -- 172.21.15.105:0/2729013367 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc808012b70 con 0x7fc810072160 2024-04-15T15:19:56.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.876+0000 7fc80e7fc640 1 --2- 172.21.15.105:0/2729013367 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fc7fc06c8f0 0x7fc7fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:56.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.876+0000 7fc80e7fc640 1 -- 172.21.15.105:0/2729013367 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc80808be60 con 0x7fc810072160 2024-04-15T15:19:56.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.876+0000 7fc8157fb640 1 --2- 172.21.15.105:0/2729013367 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fc7fc06c8f0 0x7fc7fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:56.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.877+0000 7fc8157fb640 1 --2- 172.21.15.105:0/2729013367 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fc7fc06c8f0 0x7fc7fc06edb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc804000f60 tx=0x7fc8040079d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:56.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:56.881+0000 7fc80e7fc640 1 -- 172.21.15.105:0/2729013367 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc80805a320 con 0x7fc810072160 2024-04-15T15:19:57.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.056+0000 7fc818260640 1 -- 172.21.15.105:0/2729013367 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc8100cd100 con 0x7fc810072160 2024-04-15T15:19:57.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.058+0000 7fc80e7fc640 1 -- 172.21.15.105:0/2729013367 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16653 (secure 0 0 0) 0x7fc808059c20 con 0x7fc810072160 2024-04-15T15:19:57.060 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:19:57.060 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":20,"fsid":"09e4d2a4-a324-4793-af77-644b25e520e1","created":"2024-04-15T15:19:01.524543+0000","modified":"2024-04-15T15:19:27.414890+0000","last_up_change":"2024-04-15T15:19:12.825815+0000","last_in_change":"2024-04-15T15:19:03.788126+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:19:13.748585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:19:23.789287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:19:24.722250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6f6292b-2d7e-4aa0-9d1c-6ddd0f97287c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6801","nonce":2873040054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6803","nonce":2873040054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6807","nonce":2873040054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6805","nonce":2873040054}]},"public_addr":"172.21.15.105:6801/2873040054","cluster_addr":"172.21.15.105:6803/2873040054","heartbeat_back_addr":"172.21.15.105:6807/2873040054","heartbeat_front_addr":"172.21.15.105:6805/2873040054","state":["exists","up"]},{"osd":1,"uuid":"8900966a-b393-41e3-8f93-e5a43bd23622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6818","nonce":2494994479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6821","nonce":2494994479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6828","nonce":2494994479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6823","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6825","nonce":2494994479}]},"public_addr":"172.21.15.105:6818/2494994479","cluster_addr":"172.21.15.105:6821/2494994479","heartbeat_back_addr":"172.21.15.105:6828/2494994479","heartbeat_front_addr":"172.21.15.105:6825/2494994479","state":["exists","up"]},{"osd":2,"uuid":"81a5dc7a-9325-4ff5-941d-a2d5fba21f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6809","nonce":3798408699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6811","nonce":3798408699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6815","nonce":3798408699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6813","nonce":3798408699}]},"public_addr":"172.21.15.105:6809/3798408699","cluster_addr":"172.21.15.105:6811/3798408699","heartbeat_back_addr":"172.21.15.105:6815/3798408699","heartbeat_front_addr":"172.21.15.105:6813/3798408699","state":["exists","up"]},{"osd":3,"uuid":"df766ed2-5c71-4805-96f4-3d4970edc960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6819","nonce":1920720186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6824","nonce":1920720186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6831","nonce":1920720186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6827","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6829","nonce":1920720186}]},"public_addr":"172.21.15.105:6819/1920720186","cluster_addr":"172.21.15.105:6824/1920720186","heartbeat_back_addr":"172.21.15.105:6831/1920720186","heartbeat_front_addr":"172.21.15.105:6829/1920720186","state":["exists","up"]},{"osd":4,"uuid":"e1039e03-28b8-49df-b951-c91930b93c9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6810","nonce":3516494456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6813","nonce":3516494456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6820","nonce":3516494456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6817","nonce":3516494456}]},"public_addr":"172.21.15.154:6810/3516494456","cluster_addr":"172.21.15.154:6813/3516494456","heartbeat_back_addr":"172.21.15.154:6820/3516494456","heartbeat_front_addr":"172.21.15.154:6817/3516494456","state":["exists","up"]},{"osd":5,"uuid":"a9f5eaf1-66ff-405a-a9e4-ec3340812a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6801","nonce":540367751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6803","nonce":540367751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6807","nonce":540367751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6805","nonce":540367751}]},"public_addr":"172.21.15.154:6801/540367751","cluster_addr":"172.21.15.154:6803/540367751","heartbeat_back_addr":"172.21.15.154:6807/540367751","heartbeat_front_addr":"172.21.15.154:6805/540367751","state":["exists","up"]},{"osd":6,"uuid":"267d02ea-675f-428f-aae8-e0375e4d4feb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6825","nonce":1550829005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6827","nonce":1550829005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6831","nonce":1550829005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6829","nonce":1550829005}]},"public_addr":"172.21.15.154:6825/1550829005","cluster_addr":"172.21.15.154:6827/1550829005","heartbeat_back_addr":"172.21.15.154:6831/1550829005","heartbeat_front_addr":"172.21.15.154:6829/1550829005","state":["exists","up"]},{"osd":7,"uuid":"759ac6cc-896a-423c-885f-9fe90d8dcb14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6809","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6811","nonce":3920089903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6816","nonce":3920089903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6823","nonce":3920089903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6819","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6821","nonce":3920089903}]},"public_addr":"172.21.15.154:6811/3920089903","cluster_addr":"172.21.15.154:6816/3920089903","heartbeat_back_addr":"172.21.15.154:6823/3920089903","heartbeat_front_addr":"172.21.15.154:6821/3920089903","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.949509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.984072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.907278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.947391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.975447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.998896+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:09.028739+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.894119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:19:57.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.060+0000 7fc818260640 1 -- 172.21.15.105:0/2729013367 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fc7fc06c8f0 msgr2=0x7fc7fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:57.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.060+0000 7fc818260640 1 --2- 172.21.15.105:0/2729013367 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fc7fc06c8f0 0x7fc7fc06edb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc804000f60 tx=0x7fc8040079d0 comp rx=0 tx=0).stop 2024-04-15T15:19:57.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.060+0000 7fc818260640 1 -- 172.21.15.105:0/2729013367 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc810072160 msgr2=0x7fc81015e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:57.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.060+0000 7fc818260640 1 --2- 172.21.15.105:0/2729013367 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc810072160 0x7fc81015e490 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc80800d5f0 tx=0x7fc80800dab0 comp rx=0 tx=0).stop 2024-04-15T15:19:57.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.060+0000 7fc818260640 1 -- 172.21.15.105:0/2729013367 shutdown_connections 2024-04-15T15:19:57.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.060+0000 7fc818260640 1 --2- 172.21.15.105:0/2729013367 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc8100af6b0 0x7fc8101385f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:57.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.060+0000 7fc818260640 1 --2- 172.21.15.105:0/2729013367 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fc7fc06c8f0 0x7fc7fc06edb0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:57.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.060+0000 7fc818260640 1 --2- 172.21.15.105:0/2729013367 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc810147730 0x7fc8100c8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:57.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.060+0000 7fc818260640 1 --2- 172.21.15.105:0/2729013367 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc810072160 0x7fc81015e490 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:57.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.060+0000 7fc818260640 1 -- 172.21.15.105:0/2729013367 >> 172.21.15.105:0/2729013367 conn(0x7fc81006b9c0 msgr2=0x7fc8100725f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:57.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.060+0000 7fc818260640 1 -- 172.21.15.105:0/2729013367 shutdown_connections 2024-04-15T15:19:57.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.060+0000 7fc818260640 1 -- 172.21.15.105:0/2729013367 wait complete. 2024-04-15T15:19:57.073 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:19:13.748585+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-15T15:19:23.789287+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-15T15:19:24.722250+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:19:57.073 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:19:57.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.177+0000 7efed3270640 1 Processor -- start 2024-04-15T15:19:57.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.177+0000 7efed3270640 1 -- start start 2024-04-15T15:19:57.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.178+0000 7efed3270640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efecc0af930 0x7efecc0b3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:57.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.178+0000 7efed3270640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7efecc06f8b0 0x7efecc164890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:57.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.178+0000 7efed3270640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efecc0b4780 con 0x7efecc0aded0 2024-04-15T15:19:57.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.178+0000 7efed3270640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7efecc0b4a60 con 0x7efecc06f8b0 2024-04-15T15:19:57.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.178+0000 7efed3270640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7efecc0b4d40 con 0x7efecc0af930 2024-04-15T15:19:57.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.178+0000 7efed180d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7efecc06f8b0 0x7efecc164890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:57.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.178+0000 7efed080b640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7efecc0aded0 0x7efecc0afd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.105:50800/0 (socket says 172.21.15.105:50800) 2024-04-15T15:19:57.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.178+0000 7efed080b640 1 -- 172.21.15.105:0/2865103920 learned_addr learned my addr 172.21.15.105:0/2865103920 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:19:57.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.178+0000 7efed100c640 1 --2- 172.21.15.105:0/2865103920 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efecc0af930 0x7efecc0b3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:57.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.179+0000 7efec3fff640 1 -- 172.21.15.105:0/2865103920 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2414120868 0 0) 0x7efecc0b4780 con 0x7efecc0aded0 2024-04-15T15:19:57.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.179+0000 7efec3fff640 1 -- 172.21.15.105:0/2865103920 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efeb8003520 con 0x7efecc0aded0 2024-04-15T15:19:57.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.179+0000 7efed100c640 1 -- 172.21.15.105:0/2865103920 >> v2:172.21.15.154:3300/0 conn(0x7efecc06f8b0 msgr2=0x7efecc164890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:57.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.179+0000 7efed100c640 1 --2- 172.21.15.105:0/2865103920 >> v2:172.21.15.154:3300/0 conn(0x7efecc06f8b0 0x7efecc164890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:57.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.179+0000 7efed100c640 1 -- 172.21.15.105:0/2865103920 >> v1:172.21.15.154:6789/0 conn(0x7efecc0aded0 legacy=0x7efecc0afd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:57.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.179+0000 7efed100c640 1 -- 172.21.15.105:0/2865103920 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efecc0b5020 con 0x7efecc0af930 2024-04-15T15:19:57.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.179+0000 7efed180d640 1 --2- 172.21.15.105:0/2865103920 >> v2:172.21.15.154:3300/0 conn(0x7efecc06f8b0 0x7efecc164890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:19:57.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.180+0000 7efed100c640 1 --2- 172.21.15.105:0/2865103920 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efecc0af930 0x7efecc0b3e80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7efec800a940 tx=0x7efec802cd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=871bf5df5ce88597 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:57.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.180+0000 7efec3fff640 1 -- 172.21.15.105:0/2865103920 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec803a070 con 0x7efecc0af930 2024-04-15T15:19:57.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.180+0000 7efec3fff640 1 -- 172.21.15.105:0/2865103920 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efec802db30 con 0x7efecc0af930 2024-04-15T15:19:57.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.180+0000 7efec3fff640 1 -- 172.21.15.105:0/2865103920 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec800b040 con 0x7efecc0af930 2024-04-15T15:19:57.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.180+0000 7efed3270640 1 -- 172.21.15.105:0/2865103920 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efecc0af930 msgr2=0x7efecc0b3e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:57.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.180+0000 7efed3270640 1 --2- 172.21.15.105:0/2865103920 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efecc0af930 0x7efecc0b3e80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7efec800a940 tx=0x7efec802cd80 comp rx=0 tx=0).stop 2024-04-15T15:19:57.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.180+0000 7efed3270640 1 -- 172.21.15.105:0/2865103920 shutdown_connections 2024-04-15T15:19:57.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.180+0000 7efed3270640 1 --2- 172.21.15.105:0/2865103920 >> v2:172.21.15.154:3300/0 conn(0x7efecc06f8b0 0x7efecc164890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:57.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.180+0000 7efed3270640 1 --2- 172.21.15.105:0/2865103920 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efecc0af930 0x7efecc0b3e80 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:57.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.180+0000 7efed3270640 1 -- 172.21.15.105:0/2865103920 >> 172.21.15.105:0/2865103920 conn(0x7efecc073570 msgr2=0x7efecc06b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:57.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.180+0000 7efed3270640 1 -- 172.21.15.105:0/2865103920 shutdown_connections 2024-04-15T15:19:57.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.180+0000 7efed3270640 1 -- 172.21.15.105:0/2865103920 wait complete. 2024-04-15T15:19:57.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.182+0000 7efed3270640 1 Processor -- start 2024-04-15T15:19:57.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.182+0000 7efed3270640 1 -- start start 2024-04-15T15:19:57.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.182+0000 7efed3270640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efecc06f8b0 0x7efecc0c6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:57.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.182+0000 7efed3270640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efecc0aded0 0x7efecc0c5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:57.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.182+0000 7efed3270640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efecc0af930 0x7efecc09dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:57.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.182+0000 7efed3270640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7efecc0b4540 con 0x7efecc06f8b0 2024-04-15T15:19:57.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.182+0000 7efed3270640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7efecc0b43c0 con 0x7efecc0aded0 2024-04-15T15:19:57.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.182+0000 7efed3270640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7efecc0b0420 con 0x7efecc0af930 2024-04-15T15:19:57.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.182+0000 7efed100c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efecc06f8b0 0x7efecc0c6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:57.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.182+0000 7efed100c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efecc06f8b0 0x7efecc0c6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46996/0 (socket says 172.21.15.105:46996) 2024-04-15T15:19:57.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.182+0000 7efed100c640 1 -- 172.21.15.105:0/606007349 learned_addr learned my addr 172.21.15.105:0/606007349 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:57.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.183+0000 7efed080b640 1 --2- 172.21.15.105:0/606007349 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efecc0aded0 0x7efecc0c5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:57.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.183+0000 7efed180d640 1 --2- 172.21.15.105:0/606007349 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efecc0af930 0x7efecc09dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:57.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.183+0000 7efed100c640 1 -- 172.21.15.105:0/606007349 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efecc0af930 msgr2=0x7efecc09dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:57.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.183+0000 7efed100c640 1 --2- 172.21.15.105:0/606007349 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efecc0af930 0x7efecc09dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:57.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.183+0000 7efed100c640 1 -- 172.21.15.105:0/606007349 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efecc0aded0 msgr2=0x7efecc0c5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:57.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.183+0000 7efed100c640 1 --2- 172.21.15.105:0/606007349 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efecc0aded0 0x7efecc0c5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:57.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.183+0000 7efed100c640 1 -- 172.21.15.105:0/606007349 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efecc09d020 con 0x7efecc06f8b0 2024-04-15T15:19:57.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.183+0000 7efed100c640 1 --2- 172.21.15.105:0/606007349 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efecc06f8b0 0x7efecc0c6760 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7efec802cdc0 tx=0x7efec80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:57.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.183+0000 7efec1ffb640 1 -- 172.21.15.105:0/606007349 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec803a040 con 0x7efecc06f8b0 2024-04-15T15:19:57.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.183+0000 7efec1ffb640 1 -- 172.21.15.105:0/606007349 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efec8043070 con 0x7efecc06f8b0 2024-04-15T15:19:57.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.183+0000 7efed3270640 1 -- 172.21.15.105:0/606007349 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efecc09d300 con 0x7efecc06f8b0 2024-04-15T15:19:57.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.183+0000 7efec1ffb640 1 -- 172.21.15.105:0/606007349 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec8003be0 con 0x7efecc06f8b0 2024-04-15T15:19:57.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.183+0000 7efed3270640 1 -- 172.21.15.105:0/606007349 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efecc15e400 con 0x7efecc06f8b0 2024-04-15T15:19:57.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.184+0000 7efed3270640 1 -- 172.21.15.105:0/606007349 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efecc068ab0 con 0x7efecc06f8b0 2024-04-15T15:19:57.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.185+0000 7efec1ffb640 1 -- 172.21.15.105:0/606007349 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efec8048460 con 0x7efecc06f8b0 2024-04-15T15:19:57.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.185+0000 7efec1ffb640 1 --2- 172.21.15.105:0/606007349 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7efeb006c920 0x7efeb006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:57.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.186+0000 7efec1ffb640 1 -- 172.21.15.105:0/606007349 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7efec80ad580 con 0x7efecc06f8b0 2024-04-15T15:19:57.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.186+0000 7efed080b640 1 --2- 172.21.15.105:0/606007349 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7efeb006c920 0x7efeb006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:57.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.186+0000 7efed080b640 1 --2- 172.21.15.105:0/606007349 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7efeb006c920 0x7efeb006ede0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7efecc15dbd0 tx=0x7efebc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:57.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.190+0000 7efec1ffb640 1 -- 172.21.15.105:0/606007349 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efec807b9b0 con 0x7efecc06f8b0 2024-04-15T15:19:57.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.363+0000 7efed3270640 1 -- 172.21.15.105:0/606007349 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efecc0b11e0 con 0x7efecc06f8b0 2024-04-15T15:19:57.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.363+0000 7efec1ffb640 1 -- 172.21.15.105:0/606007349 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7efec807b2b0 con 0x7efecc06f8b0 2024-04-15T15:19:57.365 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-04-15T15:19:57.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.365+0000 7efed3270640 1 -- 172.21.15.105:0/606007349 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7efeb006c920 msgr2=0x7efeb006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:57.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.365+0000 7efed3270640 1 --2- 172.21.15.105:0/606007349 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7efeb006c920 0x7efeb006ede0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7efecc15dbd0 tx=0x7efebc009040 comp rx=0 tx=0).stop 2024-04-15T15:19:57.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.365+0000 7efed3270640 1 -- 172.21.15.105:0/606007349 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efecc06f8b0 msgr2=0x7efecc0c6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:57.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.365+0000 7efed3270640 1 --2- 172.21.15.105:0/606007349 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efecc06f8b0 0x7efecc0c6760 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7efec802cdc0 tx=0x7efec80046c0 comp rx=0 tx=0).stop 2024-04-15T15:19:57.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.365+0000 7efed3270640 1 -- 172.21.15.105:0/606007349 shutdown_connections 2024-04-15T15:19:57.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.365+0000 7efed3270640 1 --2- 172.21.15.105:0/606007349 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efecc0af930 0x7efecc09dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:57.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.365+0000 7efed3270640 1 --2- 172.21.15.105:0/606007349 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7efeb006c920 0x7efeb006ede0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:57.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.365+0000 7efed3270640 1 --2- 172.21.15.105:0/606007349 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efecc0aded0 0x7efecc0c5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:57.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.365+0000 7efed3270640 1 --2- 172.21.15.105:0/606007349 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efecc06f8b0 0x7efecc0c6760 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:57.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.365+0000 7efed3270640 1 -- 172.21.15.105:0/606007349 >> 172.21.15.105:0/606007349 conn(0x7efecc073570 msgr2=0x7efecc0b0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:57.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.365+0000 7efed3270640 1 -- 172.21.15.105:0/606007349 shutdown_connections 2024-04-15T15:19:57.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.366+0000 7efed3270640 1 -- 172.21.15.105:0/606007349 wait complete. 2024-04-15T15:19:57.378 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-15T15:19:57.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.483+0000 7f36dfc4a640 1 Processor -- start 2024-04-15T15:19:57.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.484+0000 7f36dfc4a640 1 -- start start 2024-04-15T15:19:57.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.484+0000 7f36dfc4a640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f36d80cfe90 0x7f36d80cf110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:57.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.484+0000 7f36dfc4a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36d80d23f0 con 0x7f36d80cfab0 2024-04-15T15:19:57.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.485+0000 7f36dd9e6640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f36d80cfab0 0x7f36d815c670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:57442/0 (socket says 172.21.15.105:57442) 2024-04-15T15:19:57.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.485+0000 7f36dd9e6640 1 -- 172.21.15.105:0/833330974 learned_addr learned my addr 172.21.15.105:0/833330974 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:19:57.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.485+0000 7f36dfc4a640 1 -- 172.21.15.105:0/833330974 --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36d80d1e30 con 0x7f36d80cbfc0 2024-04-15T15:19:57.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.485+0000 7f36dfc4a640 1 -- 172.21.15.105:0/833330974 --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f36d80d1be0 con 0x7f36d80cfe90 2024-04-15T15:19:57.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.485+0000 7f36dd1e5640 1 --2- 172.21.15.105:0/833330974 >> v2:172.21.15.154:3300/0 conn(0x7f36d80cfe90 0x7f36d80cf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:57.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.485+0000 7f36dc9e4640 1 -- 172.21.15.105:0/833330974 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2100888829 0 0) 0x7f36d80d23f0 con 0x7f36d80cfab0 2024-04-15T15:19:57.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.485+0000 7f36dc9e4640 1 -- 172.21.15.105:0/833330974 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f36c0003520 con 0x7f36d80cfab0 2024-04-15T15:19:57.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.485+0000 7f36dc9e4640 1 -- 172.21.15.105:0/833330974 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1471462534 0 0) 0x7f36d80d1e30 con 0x7f36d80cbfc0 2024-04-15T15:19:57.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.485+0000 7f36dc9e4640 1 -- 172.21.15.105:0/833330974 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f36d80d23f0 con 0x7f36d80cbfc0 2024-04-15T15:19:57.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.485+0000 7f36dc9e4640 1 -- 172.21.15.105:0/833330974 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2646036032 0 0) 0x7f36c0003520 con 0x7f36d80cfab0 2024-04-15T15:19:57.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.485+0000 7f36dc9e4640 1 -- 172.21.15.105:0/833330974 >> v2:172.21.15.154:3300/0 conn(0x7f36d80cfe90 msgr2=0x7f36d80cf110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:57.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.485+0000 7f36dc9e4640 1 --2- 172.21.15.105:0/833330974 >> v2:172.21.15.154:3300/0 conn(0x7f36d80cfe90 0x7f36d80cf110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:57.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.485+0000 7f36dc9e4640 1 -- 172.21.15.105:0/833330974 >> v1:172.21.15.154:6789/0 conn(0x7f36d80cbfc0 legacy=0x7f36d80cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:57.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.485+0000 7f36dc9e4640 1 -- 172.21.15.105:0/833330974 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36d80d2110 con 0x7f36d80cfab0 2024-04-15T15:19:57.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.485+0000 7f36dc9e4640 1 -- 172.21.15.105:0/833330974 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7f36c80019f0 con 0x7f36d80cfab0 2024-04-15T15:19:57.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.485+0000 7f36dc9e4640 1 -- 172.21.15.105:0/833330974 >> v1:172.21.15.105:6789/0 conn(0x7f36d80cfab0 legacy=0x7f36d815c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:57.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.485+0000 7f36dc9e4640 1 --2- 172.21.15.105:0/833330974 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36c0003bc0 0x7f36c0003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:57.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.485+0000 7f36dc9e4640 1 --2- 172.21.15.105:0/833330974 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36c0004800 0x7f36c0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:57.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.485+0000 7f36dc9e4640 1 --2- 172.21.15.105:0/833330974 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f36c00051c0 0x7f36c000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:57.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.485+0000 7f36dc9e4640 1 -- 172.21.15.105:0/833330974 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f36d80d1be0 con 0x7f36c0004800 2024-04-15T15:19:57.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.485+0000 7f36dc9e4640 1 -- 172.21.15.105:0/833330974 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f36c000c0c0 con 0x7f36c0003bc0 2024-04-15T15:19:57.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.485+0000 7f36dc9e4640 1 -- 172.21.15.105:0/833330974 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f36c000c2c0 con 0x7f36c00051c0 2024-04-15T15:19:57.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.485+0000 7f36dd1e5640 1 --2- 172.21.15.105:0/833330974 >> v2:172.21.15.154:3300/0 conn(0x7f36d80cfe90 0x7f36d80cf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:19:57.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.485+0000 7f36dd1e5640 1 --2- 172.21.15.105:0/833330974 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36c0004800 0x7f36c0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:57.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.485+0000 7f36dd9e6640 1 --2- 172.21.15.105:0/833330974 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36c0003bc0 0x7f36c0003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:57.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.485+0000 7f36de1e7640 1 --2- 172.21.15.105:0/833330974 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f36c00051c0 0x7f36c000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:57.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.486+0000 7f36dd1e5640 1 -- 172.21.15.105:0/833330974 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f36c00051c0 msgr2=0x7f36c000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:57.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.486+0000 7f36dd1e5640 1 --2- 172.21.15.105:0/833330974 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f36c00051c0 0x7f36c000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:57.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.486+0000 7f36dd1e5640 1 -- 172.21.15.105:0/833330974 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36c0003bc0 msgr2=0x7f36c0003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:57.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.486+0000 7f36dd1e5640 1 --2- 172.21.15.105:0/833330974 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36c0003bc0 0x7f36c0003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:57.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.486+0000 7f36dd1e5640 1 -- 172.21.15.105:0/833330974 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36c0003520 con 0x7f36c0004800 2024-04-15T15:19:57.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.486+0000 7f36de1e7640 1 --2- 172.21.15.105:0/833330974 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f36c00051c0 0x7f36c000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:19:57.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.486+0000 7f36de1e7640 1 -- 172.21.15.105:0/833330974 reap_dead start 2024-04-15T15:19:57.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.486+0000 7f36dd1e5640 1 --2- 172.21.15.105:0/833330974 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36c0004800 0x7f36c0004c80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f36cc009c70 tx=0x7f36c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=547835da6e4fbde7 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:57.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.486+0000 7f36dc9e4640 1 -- 172.21.15.105:0/833330974 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36cc009840 con 0x7f36c0004800 2024-04-15T15:19:57.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.486+0000 7f36dc9e4640 1 -- 172.21.15.105:0/833330974 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36cc004020 con 0x7f36c0004800 2024-04-15T15:19:57.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.486+0000 7f36dc9e4640 1 -- 172.21.15.105:0/833330974 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36cc004450 con 0x7f36c0004800 2024-04-15T15:19:57.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.487+0000 7f36dfc4a640 1 -- 172.21.15.105:0/833330974 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36c0004800 msgr2=0x7f36c0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:57.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.487+0000 7f36dfc4a640 1 --2- 172.21.15.105:0/833330974 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36c0004800 0x7f36c0004c80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f36cc009c70 tx=0x7f36c0005830 comp rx=0 tx=0).stop 2024-04-15T15:19:57.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.487+0000 7f36dfc4a640 1 -- 172.21.15.105:0/833330974 shutdown_connections 2024-04-15T15:19:57.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.487+0000 7f36dfc4a640 1 --2- 172.21.15.105:0/833330974 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36c0004800 0x7f36c0004c80 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:57.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.487+0000 7f36dfc4a640 1 -- 172.21.15.105:0/833330974 >> 172.21.15.105:0/833330974 conn(0x7f36d809aa40 msgr2=0x7f36d80d14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:57.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.487+0000 7f36dfc4a640 1 -- 172.21.15.105:0/833330974 shutdown_connections 2024-04-15T15:19:57.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.487+0000 7f36dfc4a640 1 -- 172.21.15.105:0/833330974 wait complete. 2024-04-15T15:19:57.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.488+0000 7f36dfc4a640 1 Processor -- start 2024-04-15T15:19:57.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.488+0000 7f36dfc4a640 1 -- start start 2024-04-15T15:19:57.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.489+0000 7f36dfc4a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36c0004800 0x7f36d80c8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:57.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.489+0000 7f36dfc4a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36d80b6390 0x7f36d80b6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:57.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.489+0000 7f36dfc4a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f36d80cb460 0x7f36d81f5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:57.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.489+0000 7f36dfc4a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f36d815bb80 con 0x7f36c0004800 2024-04-15T15:19:57.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.489+0000 7f36dfc4a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f36d815c150 con 0x7f36d80b6390 2024-04-15T15:19:57.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.489+0000 7f36dfc4a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f36d80d0340 con 0x7f36d80cb460 2024-04-15T15:19:57.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.489+0000 7f36dd9e6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36c0004800 0x7f36d80c8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:57.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.489+0000 7f36dd9e6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36c0004800 0x7f36d80c8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47012/0 (socket says 172.21.15.105:47012) 2024-04-15T15:19:57.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.489+0000 7f36dd9e6640 1 -- 172.21.15.105:0/1798635434 learned_addr learned my addr 172.21.15.105:0/1798635434 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:57.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.489+0000 7f36dd9e6640 1 -- 172.21.15.105:0/1798635434 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f36d80cb460 msgr2=0x7f36d81f5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:57.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.489+0000 7f36dd9e6640 1 --2- 172.21.15.105:0/1798635434 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f36d80cb460 0x7f36d81f5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:57.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.489+0000 7f36dd9e6640 1 -- 172.21.15.105:0/1798635434 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36d80b6390 msgr2=0x7f36d80b6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:57.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.489+0000 7f36dd9e6640 1 --2- 172.21.15.105:0/1798635434 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36d80b6390 0x7f36d80b6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:57.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.489+0000 7f36dd9e6640 1 -- 172.21.15.105:0/1798635434 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36d80b43b0 con 0x7f36c0004800 2024-04-15T15:19:57.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.490+0000 7f36dd9e6640 1 --2- 172.21.15.105:0/1798635434 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36c0004800 0x7f36d80c8630 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f36c8001560 tx=0x7f36c800a600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:57.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.490+0000 7f36d67fc640 1 -- 172.21.15.105:0/1798635434 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c8017020 con 0x7f36c0004800 2024-04-15T15:19:57.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.490+0000 7f36d67fc640 1 -- 172.21.15.105:0/1798635434 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36c800ae40 con 0x7f36c0004800 2024-04-15T15:19:57.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.490+0000 7f36dfc4a640 1 -- 172.21.15.105:0/1798635434 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36d80b4690 con 0x7f36c0004800 2024-04-15T15:19:57.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.490+0000 7f36d67fc640 1 -- 172.21.15.105:0/1798635434 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c8013610 con 0x7f36c0004800 2024-04-15T15:19:57.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.490+0000 7f36dfc4a640 1 -- 172.21.15.105:0/1798635434 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36d80caf40 con 0x7f36c0004800 2024-04-15T15:19:57.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.491+0000 7f36dfc4a640 1 -- 172.21.15.105:0/1798635434 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36d8069150 con 0x7f36c0004800 2024-04-15T15:19:57.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.491+0000 7f36d67fc640 1 -- 172.21.15.105:0/1798635434 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f36c801a020 con 0x7f36c0004800 2024-04-15T15:19:57.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.492+0000 7f36d67fc640 1 --2- 172.21.15.105:0/1798635434 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f36c406c920 0x7f36c406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:57.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.492+0000 7f36dd1e5640 1 --2- 172.21.15.105:0/1798635434 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f36c406c920 0x7f36c406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:57.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.492+0000 7f36d67fc640 1 -- 172.21.15.105:0/1798635434 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f36c808c4c0 con 0x7f36c0004800 2024-04-15T15:19:57.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.492+0000 7f36dd1e5640 1 --2- 172.21.15.105:0/1798635434 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f36c406c920 0x7f36c406ede0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f36cc035000 tx=0x7f36cc02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:57.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.496+0000 7f36d67fc640 1 -- 172.21.15.105:0/1798635434 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f36c8021020 con 0x7f36c0004800 2024-04-15T15:19:57.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.672+0000 7f36dfc4a640 1 -- 172.21.15.105:0/1798635434 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f36d80cec30 con 0x7f36c0004800 2024-04-15T15:19:57.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.673+0000 7f36d67fc640 1 -- 172.21.15.105:0/1798635434 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f36c805a980 con 0x7f36c0004800 2024-04-15T15:19:57.675 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-04-15T15:19:57.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.674+0000 7f36dfc4a640 1 -- 172.21.15.105:0/1798635434 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f36c406c920 msgr2=0x7f36c406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:57.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.674+0000 7f36dfc4a640 1 --2- 172.21.15.105:0/1798635434 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f36c406c920 0x7f36c406ede0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f36cc035000 tx=0x7f36cc02e000 comp rx=0 tx=0).stop 2024-04-15T15:19:57.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.674+0000 7f36dfc4a640 1 -- 172.21.15.105:0/1798635434 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36c0004800 msgr2=0x7f36d80c8630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:57.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.674+0000 7f36dfc4a640 1 --2- 172.21.15.105:0/1798635434 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36c0004800 0x7f36d80c8630 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f36c8001560 tx=0x7f36c800a600 comp rx=0 tx=0).stop 2024-04-15T15:19:57.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.674+0000 7f36dfc4a640 1 -- 172.21.15.105:0/1798635434 shutdown_connections 2024-04-15T15:19:57.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.674+0000 7f36dfc4a640 1 --2- 172.21.15.105:0/1798635434 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f36d80cb460 0x7f36d81f5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:57.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.675+0000 7f36dfc4a640 1 --2- 172.21.15.105:0/1798635434 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f36c406c920 0x7f36c406ede0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:57.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.675+0000 7f36dfc4a640 1 --2- 172.21.15.105:0/1798635434 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36d80b6390 0x7f36d80b6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:57.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.675+0000 7f36dfc4a640 1 --2- 172.21.15.105:0/1798635434 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36c0004800 0x7f36d80c8630 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:57.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.675+0000 7f36dfc4a640 1 -- 172.21.15.105:0/1798635434 >> 172.21.15.105:0/1798635434 conn(0x7f36d809aa40 msgr2=0x7f36d8074a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:57.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.675+0000 7f36dfc4a640 1 -- 172.21.15.105:0/1798635434 shutdown_connections 2024-04-15T15:19:57.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.675+0000 7f36dfc4a640 1 -- 172.21.15.105:0/1798635434 wait complete. 2024-04-15T15:19:57.687 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-15T15:19:57.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.792+0000 7f28c164a640 1 Processor -- start 2024-04-15T15:19:57.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.792+0000 7f28c164a640 1 -- start start 2024-04-15T15:19:57.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.793+0000 7f28c164a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f28bc1461e0 0x7f28bc0b0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:57.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.793+0000 7f28c164a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f28bc06b9c0 0x7f28bc06be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:57.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.793+0000 7f28c164a640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28bc0afd30 con 0x7f28bc15d9c0 2024-04-15T15:19:57.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.793+0000 7f28c164a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f28bc0b65b0 con 0x7f28bc1461e0 2024-04-15T15:19:57.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.793+0000 7f28c164a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f28bc06c350 con 0x7f28bc06b9c0 2024-04-15T15:19:57.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.793+0000 7f28baffd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f28bc1461e0 0x7f28bc0b0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:57.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.793+0000 7f28baffd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f28bc1461e0 0x7f28bc0b0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47028/0 (socket says 172.21.15.105:47028) 2024-04-15T15:19:57.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.793+0000 7f28baffd640 1 -- 172.21.15.105:0/4054954988 learned_addr learned my addr 172.21.15.105:0/4054954988 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:57.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.793+0000 7f28bb7fe640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f28bc06b9c0 0x7f28bc06be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:57.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.793+0000 7f28b3fff640 1 --1- 172.21.15.105:0/4054954988 >> v1:172.21.15.154:6789/0 conn(0x7f28bc15d9c0 0x7f28bc0b5ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.105:50806/0 (socket says 172.21.15.105:50806) 2024-04-15T15:19:57.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.793+0000 7f28baffd640 1 -- 172.21.15.105:0/4054954988 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f28bc06b9c0 msgr2=0x7f28bc06be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:57.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.793+0000 7f28baffd640 1 --2- 172.21.15.105:0/4054954988 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f28bc06b9c0 0x7f28bc06be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:57.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.793+0000 7f28baffd640 1 -- 172.21.15.105:0/4054954988 >> v1:172.21.15.154:6789/0 conn(0x7f28bc15d9c0 legacy=0x7f28bc0b5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:57.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.793+0000 7f28baffd640 1 -- 172.21.15.105:0/4054954988 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28bc0b0010 con 0x7f28bc1461e0 2024-04-15T15:19:57.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.794+0000 7f28baffd640 1 --2- 172.21.15.105:0/4054954988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28bc1461e0 0x7f28bc0b0af0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f28ac009870 tx=0x7f28ac02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5de28e1b12368d5a server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:57.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.794+0000 7f28ba7fc640 1 -- 172.21.15.105:0/4054954988 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ac035020 con 0x7f28bc1461e0 2024-04-15T15:19:57.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.794+0000 7f28ba7fc640 1 -- 172.21.15.105:0/4054954988 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28ac031640 con 0x7f28bc1461e0 2024-04-15T15:19:57.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.794+0000 7f28ba7fc640 1 -- 172.21.15.105:0/4054954988 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ac02d040 con 0x7f28bc1461e0 2024-04-15T15:19:57.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.795+0000 7f28c164a640 1 -- 172.21.15.105:0/4054954988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28bc1461e0 msgr2=0x7f28bc0b0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:57.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.795+0000 7f28c164a640 1 --2- 172.21.15.105:0/4054954988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28bc1461e0 0x7f28bc0b0af0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f28ac009870 tx=0x7f28ac02b8f0 comp rx=0 tx=0).stop 2024-04-15T15:19:57.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.795+0000 7f28c164a640 1 -- 172.21.15.105:0/4054954988 shutdown_connections 2024-04-15T15:19:57.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.795+0000 7f28c164a640 1 --2- 172.21.15.105:0/4054954988 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f28bc06b9c0 0x7f28bc06be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:57.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.795+0000 7f28c164a640 1 --2- 172.21.15.105:0/4054954988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28bc1461e0 0x7f28bc0b0af0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:57.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.795+0000 7f28c164a640 1 -- 172.21.15.105:0/4054954988 >> 172.21.15.105:0/4054954988 conn(0x7f28bc0b6b10 msgr2=0x7f28bc09aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:57.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.795+0000 7f28c164a640 1 -- 172.21.15.105:0/4054954988 shutdown_connections 2024-04-15T15:19:57.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.795+0000 7f28c164a640 1 -- 172.21.15.105:0/4054954988 wait complete. 2024-04-15T15:19:57.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.796+0000 7f28c164a640 1 Processor -- start 2024-04-15T15:19:57.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.796+0000 7f28c164a640 1 -- start start 2024-04-15T15:19:57.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.797+0000 7f28c164a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28bc06b9c0 0x7f28bc13a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:57.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.797+0000 7f28c164a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28bc1461e0 0x7f28bc09d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:57.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.797+0000 7f28c164a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f28bc15d9c0 0x7f28bc09d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:57.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.797+0000 7f28c164a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f28bc0b67d0 con 0x7f28bc1461e0 2024-04-15T15:19:57.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.797+0000 7f28c164a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f28bc0b05d0 con 0x7f28bc06b9c0 2024-04-15T15:19:57.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.797+0000 7f28c164a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f28bc0b43d0 con 0x7f28bc15d9c0 2024-04-15T15:19:57.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.797+0000 7f28b3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28bc1461e0 0x7f28bc09d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:57.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.797+0000 7f28b3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28bc1461e0 0x7f28bc09d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47044/0 (socket says 172.21.15.105:47044) 2024-04-15T15:19:57.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.797+0000 7f28b3fff640 1 -- 172.21.15.105:0/368670997 learned_addr learned my addr 172.21.15.105:0/368670997 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:57.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.797+0000 7f28bb7fe640 1 --2- 172.21.15.105:0/368670997 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f28bc15d9c0 0x7f28bc09d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:57.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.797+0000 7f28baffd640 1 --2- 172.21.15.105:0/368670997 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28bc06b9c0 0x7f28bc13a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:57.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.797+0000 7f28b3fff640 1 -- 172.21.15.105:0/368670997 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f28bc15d9c0 msgr2=0x7f28bc09d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:57.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.797+0000 7f28b3fff640 1 --2- 172.21.15.105:0/368670997 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f28bc15d9c0 0x7f28bc09d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:57.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.797+0000 7f28b3fff640 1 -- 172.21.15.105:0/368670997 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28bc06b9c0 msgr2=0x7f28bc13a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:57.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.797+0000 7f28b3fff640 1 --2- 172.21.15.105:0/368670997 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28bc06b9c0 0x7f28bc13a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:57.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.797+0000 7f28b3fff640 1 -- 172.21.15.105:0/368670997 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28bc13aab0 con 0x7f28bc1461e0 2024-04-15T15:19:57.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.798+0000 7f28b3fff640 1 --2- 172.21.15.105:0/368670997 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28bc1461e0 0x7f28bc09d3c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f28a400c3c0 tx=0x7f28a400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:57.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.798+0000 7f28b37fe640 1 -- 172.21.15.105:0/368670997 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a40074e0 con 0x7f28bc1461e0 2024-04-15T15:19:57.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.798+0000 7f28b37fe640 1 -- 172.21.15.105:0/368670997 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28a4007680 con 0x7f28bc1461e0 2024-04-15T15:19:57.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.798+0000 7f28b37fe640 1 -- 172.21.15.105:0/368670997 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a40049e0 con 0x7f28bc1461e0 2024-04-15T15:19:57.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.798+0000 7f28c164a640 1 -- 172.21.15.105:0/368670997 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28bc09c4d0 con 0x7f28bc1461e0 2024-04-15T15:19:57.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.798+0000 7f28c164a640 1 -- 172.21.15.105:0/368670997 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28bc09ca90 con 0x7f28bc1461e0 2024-04-15T15:19:57.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.799+0000 7f28c164a640 1 -- 172.21.15.105:0/368670997 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28bc069150 con 0x7f28bc1461e0 2024-04-15T15:19:57.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.799+0000 7f28b37fe640 1 -- 172.21.15.105:0/368670997 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f28a401a020 con 0x7f28bc1461e0 2024-04-15T15:19:57.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.800+0000 7f28b37fe640 1 --2- 172.21.15.105:0/368670997 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f289c06c8f0 0x7f289c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:57.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.800+0000 7f28baffd640 1 --2- 172.21.15.105:0/368670997 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f289c06c8f0 0x7f289c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:57.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.800+0000 7f28b37fe640 1 -- 172.21.15.105:0/368670997 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f28a408b250 con 0x7f28bc1461e0 2024-04-15T15:19:57.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.800+0000 7f28baffd640 1 --2- 172.21.15.105:0/368670997 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f289c06c8f0 0x7f289c06edb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f28ac036000 tx=0x7f28ac03c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:57.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.804+0000 7f28b37fe640 1 -- 172.21.15.105:0/368670997 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f28a4059710 con 0x7f28bc1461e0 2024-04-15T15:19:57.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.978+0000 7f28c164a640 1 -- 172.21.15.105:0/368670997 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f28bc06c4f0 con 0x7f28bc1461e0 2024-04-15T15:19:57.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.978+0000 7f28b37fe640 1 -- 172.21.15.105:0/368670997 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f28a4059010 con 0x7f28bc1461e0 2024-04-15T15:19:57.980 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-04-15T15:19:57.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.980+0000 7f28c164a640 1 -- 172.21.15.105:0/368670997 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f289c06c8f0 msgr2=0x7f289c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:57.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.980+0000 7f28c164a640 1 --2- 172.21.15.105:0/368670997 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f289c06c8f0 0x7f289c06edb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f28ac036000 tx=0x7f28ac03c040 comp rx=0 tx=0).stop 2024-04-15T15:19:57.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.980+0000 7f28c164a640 1 -- 172.21.15.105:0/368670997 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28bc1461e0 msgr2=0x7f28bc09d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:57.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.980+0000 7f28c164a640 1 --2- 172.21.15.105:0/368670997 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28bc1461e0 0x7f28bc09d3c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f28a400c3c0 tx=0x7f28a400c880 comp rx=0 tx=0).stop 2024-04-15T15:19:57.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.980+0000 7f28c164a640 1 -- 172.21.15.105:0/368670997 shutdown_connections 2024-04-15T15:19:57.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.980+0000 7f28c164a640 1 --2- 172.21.15.105:0/368670997 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f28bc15d9c0 0x7f28bc09d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:57.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.980+0000 7f28c164a640 1 --2- 172.21.15.105:0/368670997 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f289c06c8f0 0x7f289c06edb0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:57.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.980+0000 7f28c164a640 1 --2- 172.21.15.105:0/368670997 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28bc1461e0 0x7f28bc09d3c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:57.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.980+0000 7f28c164a640 1 --2- 172.21.15.105:0/368670997 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28bc06b9c0 0x7f28bc13a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:57.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.980+0000 7f28c164a640 1 -- 172.21.15.105:0/368670997 >> 172.21.15.105:0/368670997 conn(0x7f28bc0b6b10 msgr2=0x7f28bc0b2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:57.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.980+0000 7f28c164a640 1 -- 172.21.15.105:0/368670997 shutdown_connections 2024-04-15T15:19:57.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:57.980+0000 7f28c164a640 1 -- 172.21.15.105:0/368670997 wait complete. 2024-04-15T15:19:57.993 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-15T15:19:58.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.166+0000 7faf188c8640 1 Processor -- start 2024-04-15T15:19:58.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.166+0000 7faf188c8640 1 -- start start 2024-04-15T15:19:58.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.167+0000 7faf188c8640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7faf140b3c80 0x7faf14169ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:58.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.167+0000 7faf188c8640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf140c8270 con 0x7faf140d0540 2024-04-15T15:19:58.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.167+0000 7faf188c8640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf140d0920 con 0x7faf1406c0d0 2024-04-15T15:19:58.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.167+0000 7faf188c8640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7faf140c7cb0 con 0x7faf140b3c80 2024-04-15T15:19:58.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.167+0000 7faf1259c640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7faf140d0540 0x7faf1406b9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:57448/0 (socket says 172.21.15.105:57448) 2024-04-15T15:19:58.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.167+0000 7faf1259c640 1 -- 172.21.15.105:0/1611353830 learned_addr learned my addr 172.21.15.105:0/1611353830 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:19:58.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.167+0000 7faf11d9b640 1 -- 172.21.15.105:0/1611353830 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3214389110 0 0) 0x7faf140c8270 con 0x7faf140d0540 2024-04-15T15:19:58.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.167+0000 7faf11d9b640 1 -- 172.21.15.105:0/1611353830 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faf00003520 con 0x7faf140d0540 2024-04-15T15:19:58.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.167+0000 7faf12d9d640 1 --2- 172.21.15.105:0/1611353830 >> v2:172.21.15.154:3300/0 conn(0x7faf140b3c80 0x7faf14169ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:58.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.167+0000 7faf11d9b640 1 -- 172.21.15.105:0/1611353830 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1951041716 0 0) 0x7faf00003520 con 0x7faf140d0540 2024-04-15T15:19:58.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.167+0000 7faf11d9b640 1 -- 172.21.15.105:0/1611353830 >> v2:172.21.15.154:3300/0 conn(0x7faf140b3c80 msgr2=0x7faf14169ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:58.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.167+0000 7faf11d9b640 1 --2- 172.21.15.105:0/1611353830 >> v2:172.21.15.154:3300/0 conn(0x7faf140b3c80 0x7faf14169ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:58.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.167+0000 7faf11d9b640 1 -- 172.21.15.105:0/1611353830 >> v1:172.21.15.154:6789/0 conn(0x7faf1406c0d0 legacy=0x7faf14166570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:58.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.167+0000 7faf11d9b640 1 -- 172.21.15.105:0/1611353830 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf140c7f90 con 0x7faf140d0540 2024-04-15T15:19:58.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.167+0000 7faf11d9b640 1 -- 172.21.15.105:0/1611353830 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7faf04001670 con 0x7faf140d0540 2024-04-15T15:19:58.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.167+0000 7faf12d9d640 1 --2- 172.21.15.105:0/1611353830 >> v2:172.21.15.154:3300/0 conn(0x7faf140b3c80 0x7faf14169ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:19:58.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.167+0000 7faf11d9b640 1 -- 172.21.15.105:0/1611353830 >> v1:172.21.15.105:6789/0 conn(0x7faf140d0540 legacy=0x7faf1406b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:58.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.167+0000 7faf11d9b640 1 --2- 172.21.15.105:0/1611353830 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faf000039e0 0x7faf00003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:58.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.167+0000 7faf11d9b640 1 --2- 172.21.15.105:0/1611353830 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf00004800 0x7faf00004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:58.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.167+0000 7faf11d9b640 1 --2- 172.21.15.105:0/1611353830 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf00005210 0x7faf0000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:58.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.167+0000 7faf11d9b640 1 -- 172.21.15.105:0/1611353830 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7faf140c7cb0 con 0x7faf00005210 2024-04-15T15:19:58.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.167+0000 7faf11d9b640 1 -- 172.21.15.105:0/1611353830 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7faf0000c140 con 0x7faf00004800 2024-04-15T15:19:58.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.167+0000 7faf11d9b640 1 -- 172.21.15.105:0/1611353830 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7faf0000c340 con 0x7faf000039e0 2024-04-15T15:19:58.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.167+0000 7faf12d9d640 1 --2- 172.21.15.105:0/1611353830 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf00005210 0x7faf0000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:58.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.168+0000 7faf1259c640 1 --2- 172.21.15.105:0/1611353830 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faf000039e0 0x7faf00003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:58.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.168+0000 7faf09d9b640 1 --2- 172.21.15.105:0/1611353830 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf00004800 0x7faf00004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:58.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.168+0000 7faf12d9d640 1 -- 172.21.15.105:0/1611353830 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faf000039e0 msgr2=0x7faf00003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:58.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.168+0000 7faf12d9d640 1 --2- 172.21.15.105:0/1611353830 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faf000039e0 0x7faf00003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:58.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.168+0000 7faf12d9d640 1 -- 172.21.15.105:0/1611353830 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf00004800 msgr2=0x7faf00004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:58.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.168+0000 7faf12d9d640 1 --2- 172.21.15.105:0/1611353830 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf00004800 0x7faf00004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:58.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.168+0000 7faf12d9d640 1 -- 172.21.15.105:0/1611353830 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf00003520 con 0x7faf00005210 2024-04-15T15:19:58.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.168+0000 7faf09d9b640 1 --2- 172.21.15.105:0/1611353830 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf00004800 0x7faf00004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:19:58.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.168+0000 7faf1259c640 1 --2- 172.21.15.105:0/1611353830 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faf000039e0 0x7faf00003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:19:58.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.168+0000 7faf12d9d640 1 -- 172.21.15.105:0/1611353830 reap_dead start 2024-04-15T15:19:58.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.168+0000 7faf12d9d640 1 --2- 172.21.15.105:0/1611353830 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf00005210 0x7faf0000b9c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7faf0c0073b0 tx=0x7faf00005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ddb3acfd077d067 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:58.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.169+0000 7faf11d9b640 1 -- 172.21.15.105:0/1611353830 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c039070 con 0x7faf00005210 2024-04-15T15:19:58.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.169+0000 7faf11d9b640 1 -- 172.21.15.105:0/1611353830 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf0c02cd90 con 0x7faf00005210 2024-04-15T15:19:58.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.169+0000 7faf11d9b640 1 -- 172.21.15.105:0/1611353830 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c009c80 con 0x7faf00005210 2024-04-15T15:19:58.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.169+0000 7faf188c8640 1 -- 172.21.15.105:0/1611353830 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf00005210 msgr2=0x7faf0000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:58.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.169+0000 7faf188c8640 1 --2- 172.21.15.105:0/1611353830 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf00005210 0x7faf0000b9c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7faf0c0073b0 tx=0x7faf00005830 comp rx=0 tx=0).stop 2024-04-15T15:19:58.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.169+0000 7faf188c8640 1 -- 172.21.15.105:0/1611353830 shutdown_connections 2024-04-15T15:19:58.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.169+0000 7faf188c8640 1 --2- 172.21.15.105:0/1611353830 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf00005210 0x7faf0000b9c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:58.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.169+0000 7faf188c8640 1 -- 172.21.15.105:0/1611353830 >> 172.21.15.105:0/1611353830 conn(0x7faf1409aa40 msgr2=0x7faf140b30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:58.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.169+0000 7faf188c8640 1 -- 172.21.15.105:0/1611353830 shutdown_connections 2024-04-15T15:19:58.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.169+0000 7faf188c8640 1 -- 172.21.15.105:0/1611353830 wait complete. 2024-04-15T15:19:58.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.171+0000 7faf188c8640 1 Processor -- start 2024-04-15T15:19:58.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.171+0000 7faf188c8640 1 -- start start 2024-04-15T15:19:58.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.171+0000 7faf188c8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf00005210 0x7faf14161a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:58.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.171+0000 7faf188c8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faf1409ba60 0x7faf141f79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:58.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.171+0000 7faf188c8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf1415f5e0 0x7faf140b5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:58.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.171+0000 7faf188c8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7faf14070e40 con 0x7faf00005210 2024-04-15T15:19:58.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.171+0000 7faf188c8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7faf1406f180 con 0x7faf1415f5e0 2024-04-15T15:19:58.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.171+0000 7faf188c8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7faf14070fc0 con 0x7faf1409ba60 2024-04-15T15:19:58.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.171+0000 7faf1259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf00005210 0x7faf14161a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:58.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.171+0000 7faf1259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf00005210 0x7faf14161a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47066/0 (socket says 172.21.15.105:47066) 2024-04-15T15:19:58.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.172+0000 7faf1259c640 1 -- 172.21.15.105:0/2804645294 learned_addr learned my addr 172.21.15.105:0/2804645294 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:58.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.172+0000 7faf12d9d640 1 --2- 172.21.15.105:0/2804645294 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf1415f5e0 0x7faf140b5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:58.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.172+0000 7faf09d9b640 1 --2- 172.21.15.105:0/2804645294 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faf1409ba60 0x7faf141f79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:58.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.172+0000 7faf1259c640 1 -- 172.21.15.105:0/2804645294 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faf1409ba60 msgr2=0x7faf141f79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:58.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.172+0000 7faf1259c640 1 --2- 172.21.15.105:0/2804645294 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faf1409ba60 0x7faf141f79b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:58.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.172+0000 7faf1259c640 1 -- 172.21.15.105:0/2804645294 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf1415f5e0 msgr2=0x7faf140b5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:58.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.172+0000 7faf1259c640 1 --2- 172.21.15.105:0/2804645294 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf1415f5e0 0x7faf140b5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:58.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.172+0000 7faf1259c640 1 -- 172.21.15.105:0/2804645294 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf141f8320 con 0x7faf00005210 2024-04-15T15:19:58.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.172+0000 7faf1259c640 1 --2- 172.21.15.105:0/2804645294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf00005210 0x7faf14161a90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7faf0400e820 tx=0x7faf0400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:58.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.172+0000 7faf0b7fe640 1 -- 172.21.15.105:0/2804645294 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0400c870 con 0x7faf00005210 2024-04-15T15:19:58.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.172+0000 7faf0b7fe640 1 -- 172.21.15.105:0/2804645294 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf04018070 con 0x7faf00005210 2024-04-15T15:19:58.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.172+0000 7faf0b7fe640 1 -- 172.21.15.105:0/2804645294 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0400cc10 con 0x7faf00005210 2024-04-15T15:19:58.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.172+0000 7faf188c8640 1 -- 172.21.15.105:0/2804645294 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf141f8600 con 0x7faf00005210 2024-04-15T15:19:58.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.172+0000 7faf188c8640 1 -- 172.21.15.105:0/2804645294 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf141f7ef0 con 0x7faf00005210 2024-04-15T15:19:58.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.173+0000 7faf188c8640 1 -- 172.21.15.105:0/2804645294 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf14068a40 con 0x7faf00005210 2024-04-15T15:19:58.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.174+0000 7faf0b7fe640 1 -- 172.21.15.105:0/2804645294 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faf04013420 con 0x7faf00005210 2024-04-15T15:19:58.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.174+0000 7faf0b7fe640 1 --2- 172.21.15.105:0/2804645294 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7faef406c920 0x7faef406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:58.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.174+0000 7faf0b7fe640 1 -- 172.21.15.105:0/2804645294 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7faf0408b600 con 0x7faf00005210 2024-04-15T15:19:58.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.174+0000 7faf09d9b640 1 --2- 172.21.15.105:0/2804645294 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7faef406c920 0x7faef406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:58.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.175+0000 7faf09d9b640 1 --2- 172.21.15.105:0/2804645294 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7faef406c920 0x7faef406ede0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7faf1409be60 tx=0x7faefc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:58.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.178+0000 7faf0b7fe640 1 -- 172.21.15.105:0/2804645294 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faf04059ac0 con 0x7faf00005210 2024-04-15T15:19:58.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.364+0000 7faf188c8640 1 -- 172.21.15.105:0/2804645294 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7faf1415d9f0 con 0x7faf00005210 2024-04-15T15:19:58.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.364+0000 7faf0b7fe640 1 -- 172.21.15.105:0/2804645294 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7faf040593c0 con 0x7faf00005210 2024-04-15T15:19:58.366 INFO:teuthology.orchestra.run.smithi105.stderr:listed 0 entries 2024-04-15T15:19:58.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.366+0000 7faf188c8640 1 -- 172.21.15.105:0/2804645294 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7faef406c920 msgr2=0x7faef406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:58.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.366+0000 7faf188c8640 1 --2- 172.21.15.105:0/2804645294 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7faef406c920 0x7faef406ede0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7faf1409be60 tx=0x7faefc009040 comp rx=0 tx=0).stop 2024-04-15T15:19:58.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.366+0000 7faf188c8640 1 -- 172.21.15.105:0/2804645294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf00005210 msgr2=0x7faf14161a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:58.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.366+0000 7faf188c8640 1 --2- 172.21.15.105:0/2804645294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf00005210 0x7faf14161a90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7faf0400e820 tx=0x7faf0400ece0 comp rx=0 tx=0).stop 2024-04-15T15:19:58.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.366+0000 7faf188c8640 1 -- 172.21.15.105:0/2804645294 shutdown_connections 2024-04-15T15:19:58.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.366+0000 7faf188c8640 1 --2- 172.21.15.105:0/2804645294 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faf1409ba60 0x7faf141f79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:58.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.366+0000 7faf188c8640 1 --2- 172.21.15.105:0/2804645294 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7faef406c920 0x7faef406ede0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:58.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.366+0000 7faf188c8640 1 --2- 172.21.15.105:0/2804645294 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf1415f5e0 0x7faf140b5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:58.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.366+0000 7faf188c8640 1 --2- 172.21.15.105:0/2804645294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf00005210 0x7faf14161a90 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:58.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.366+0000 7faf188c8640 1 -- 172.21.15.105:0/2804645294 >> 172.21.15.105:0/2804645294 conn(0x7faf1409aa40 msgr2=0x7faf140afcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:58.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.366+0000 7faf188c8640 1 -- 172.21.15.105:0/2804645294 shutdown_connections 2024-04-15T15:19:58.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.366+0000 7faf188c8640 1 -- 172.21.15.105:0/2804645294 wait complete. 2024-04-15T15:19:58.379 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi105.front.sepia.ceph.com... 2024-04-15T15:19:58.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:58.379 DEBUG:teuthology.orchestra.run.smithi105:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-15T15:19:58.407 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:19:58.408 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi105.front.sepia.ceph.com 2024-04-15T15:19:58.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:58.408 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-15T15:19:58.468 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-15T15:19:58.468 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:19:58.488 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-15T15:19:58.488 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:19:58.546 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:19:58.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.648+0000 7f2e0ce4a640 1 Processor -- start 2024-04-15T15:19:58.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.648+0000 7f2e0ce4a640 1 -- start start 2024-04-15T15:19:58.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.649+0000 7f2e0ce4a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e081554d0 0x7f2e080afaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:58.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.649+0000 7f2e0ce4a640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2e080b5580 0x7f2e080cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:58.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.649+0000 7f2e0ce4a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2e080cfc50 0x7f2e080726f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:58.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.649+0000 7f2e0ce4a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f2e080cf280 con 0x7f2e080cfc50 2024-04-15T15:19:58.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.649+0000 7f2e0ce4a640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f2e080b5a00 con 0x7f2e080b5580 2024-04-15T15:19:58.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.649+0000 7f2e0ce4a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f2e0815bf70 con 0x7f2e081554d0 2024-04-15T15:19:58.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.649+0000 7f2e06d9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2e080cfc50 0x7f2e080726f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:58.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.649+0000 7f2e06d9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2e080cfc50 0x7f2e080726f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47078/0 (socket says 172.21.15.105:47078) 2024-04-15T15:19:58.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.649+0000 7f2e06d9d640 1 -- 172.21.15.105:0/1697984833 learned_addr learned my addr 172.21.15.105:0/1697984833 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:58.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.650+0000 7f2e0659c640 1 --2- 172.21.15.105:0/1697984833 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e081554d0 0x7f2e080afaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:58.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.650+0000 7f2e05d9b640 1 --2- 172.21.15.105:0/1697984833 >> v2:172.21.15.154:3300/0 conn(0x7f2e080b5580 0x7f2e080cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:58.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.650+0000 7f2e06d9d640 1 -- 172.21.15.105:0/1697984833 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e081554d0 msgr2=0x7f2e080afaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:58.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.650+0000 7f2e06d9d640 1 --2- 172.21.15.105:0/1697984833 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e081554d0 0x7f2e080afaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:58.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.650+0000 7f2e06d9d640 1 -- 172.21.15.105:0/1697984833 >> v2:172.21.15.154:3300/0 conn(0x7f2e080b5580 msgr2=0x7f2e080cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:58.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.650+0000 7f2e06d9d640 1 --2- 172.21.15.105:0/1697984833 >> v2:172.21.15.154:3300/0 conn(0x7f2e080b5580 0x7f2e080cf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:58.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.650+0000 7f2e06d9d640 1 -- 172.21.15.105:0/1697984833 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e080cec30 con 0x7f2e080cfc50 2024-04-15T15:19:58.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.650+0000 7f2e06d9d640 1 --2- 172.21.15.105:0/1697984833 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e080cfc50 0x7f2e080726f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2e0806c5d0 tx=0x7f2e0002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf3a370410fe0625 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:58.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.651+0000 7f2e0559a640 1 -- 172.21.15.105:0/1697984833 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e00035020 con 0x7f2e080cfc50 2024-04-15T15:19:58.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.651+0000 7f2e0559a640 1 -- 172.21.15.105:0/1697984833 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e00031870 con 0x7f2e080cfc50 2024-04-15T15:19:58.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.651+0000 7f2e0559a640 1 -- 172.21.15.105:0/1697984833 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e00031c40 con 0x7f2e080cfc50 2024-04-15T15:19:58.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.651+0000 7f2e0ce4a640 1 -- 172.21.15.105:0/1697984833 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e080cfc50 msgr2=0x7f2e080726f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:58.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.651+0000 7f2e0ce4a640 1 --2- 172.21.15.105:0/1697984833 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e080cfc50 0x7f2e080726f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2e0806c5d0 tx=0x7f2e0002b910 comp rx=0 tx=0).stop 2024-04-15T15:19:58.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.651+0000 7f2e0ce4a640 1 -- 172.21.15.105:0/1697984833 shutdown_connections 2024-04-15T15:19:58.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.651+0000 7f2e0ce4a640 1 --2- 172.21.15.105:0/1697984833 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e080cfc50 0x7f2e080726f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:58.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.651+0000 7f2e0ce4a640 1 --2- 172.21.15.105:0/1697984833 >> v2:172.21.15.154:3300/0 conn(0x7f2e080b5580 0x7f2e080cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:58.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.651+0000 7f2e0ce4a640 1 --2- 172.21.15.105:0/1697984833 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e081554d0 0x7f2e080afaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:58.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.651+0000 7f2e0ce4a640 1 -- 172.21.15.105:0/1697984833 >> 172.21.15.105:0/1697984833 conn(0x7f2e08067d50 msgr2=0x7f2e0806b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:58.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.651+0000 7f2e0ce4a640 1 -- 172.21.15.105:0/1697984833 shutdown_connections 2024-04-15T15:19:58.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.651+0000 7f2e0ce4a640 1 -- 172.21.15.105:0/1697984833 wait complete. 2024-04-15T15:19:58.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.653+0000 7f2e0ce4a640 1 Processor -- start 2024-04-15T15:19:58.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.653+0000 7f2e0ce4a640 1 -- start start 2024-04-15T15:19:58.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.653+0000 7f2e0ce4a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e080b5580 0x7f2e081581d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:58.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.653+0000 7f2e0ce4a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e080cfc50 0x7f2e08143f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:58.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.653+0000 7f2e0ce4a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e081554d0 0x7f2e08140fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:58.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.653+0000 7f2e0ce4a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2e08072c30 con 0x7f2e080cfc50 2024-04-15T15:19:58.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.653+0000 7f2e0ce4a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0815c250 con 0x7f2e081554d0 2024-04-15T15:19:58.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.653+0000 7f2e0ce4a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f2e08072db0 con 0x7f2e080b5580 2024-04-15T15:19:58.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.653+0000 7f2e05d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e080cfc50 0x7f2e08143f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:58.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.653+0000 7f2e05d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e080cfc50 0x7f2e08143f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47094/0 (socket says 172.21.15.105:47094) 2024-04-15T15:19:58.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.653+0000 7f2e05d9b640 1 -- 172.21.15.105:0/2716977993 learned_addr learned my addr 172.21.15.105:0/2716977993 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:58.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.654+0000 7f2e06d9d640 1 --2- 172.21.15.105:0/2716977993 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e081554d0 0x7f2e08140fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:58.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.654+0000 7f2e0659c640 1 --2- 172.21.15.105:0/2716977993 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e080b5580 0x7f2e081581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:58.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.654+0000 7f2e05d9b640 1 -- 172.21.15.105:0/2716977993 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e080b5580 msgr2=0x7f2e081581d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:58.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.654+0000 7f2e05d9b640 1 --2- 172.21.15.105:0/2716977993 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e080b5580 0x7f2e081581d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:58.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.654+0000 7f2e05d9b640 1 -- 172.21.15.105:0/2716977993 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e081554d0 msgr2=0x7f2e08140fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:58.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.654+0000 7f2e05d9b640 1 --2- 172.21.15.105:0/2716977993 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e081554d0 0x7f2e08140fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:58.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.654+0000 7f2e05d9b640 1 -- 172.21.15.105:0/2716977993 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e08144ca0 con 0x7f2e080cfc50 2024-04-15T15:19:58.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.654+0000 7f2e06d9d640 1 --2- 172.21.15.105:0/2716977993 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e081554d0 0x7f2e08140fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:19:58.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.654+0000 7f2e05d9b640 1 --2- 172.21.15.105:0/2716977993 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e080cfc50 0x7f2e08143f20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc00d800 tx=0x7f2dfc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:58.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.654+0000 7f2df6ffd640 1 -- 172.21.15.105:0/2716977993 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc018020 con 0x7f2e080cfc50 2024-04-15T15:19:58.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.654+0000 7f2df6ffd640 1 -- 172.21.15.105:0/2716977993 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dfc0096a0 con 0x7f2e080cfc50 2024-04-15T15:19:58.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.654+0000 7f2e0ce4a640 1 -- 172.21.15.105:0/2716977993 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e08144f80 con 0x7f2e080cfc50 2024-04-15T15:19:58.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.654+0000 7f2df6ffd640 1 -- 172.21.15.105:0/2716977993 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc009a70 con 0x7f2e080cfc50 2024-04-15T15:19:58.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.654+0000 7f2e0ce4a640 1 -- 172.21.15.105:0/2716977993 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e08145540 con 0x7f2e080cfc50 2024-04-15T15:19:58.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.655+0000 7f2e0ce4a640 1 -- 172.21.15.105:0/2716977993 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e08068a40 con 0x7f2e080cfc50 2024-04-15T15:19:58.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.656+0000 7f2df6ffd640 1 -- 172.21.15.105:0/2716977993 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2dfc01b020 con 0x7f2e080cfc50 2024-04-15T15:19:58.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.656+0000 7f2df6ffd640 1 --2- 172.21.15.105:0/2716977993 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f2de806c8f0 0x7f2de806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:58.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.656+0000 7f2df6ffd640 1 -- 172.21.15.105:0/2716977993 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2dfc08bd40 con 0x7f2e080cfc50 2024-04-15T15:19:58.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.656+0000 7f2e0659c640 1 --2- 172.21.15.105:0/2716977993 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f2de806c8f0 0x7f2de806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:58.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.656+0000 7f2e0659c640 1 --2- 172.21.15.105:0/2716977993 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f2de806c8f0 0x7f2de806edb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2df80099c0 tx=0x7f2df8003990 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:58.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.660+0000 7f2df6ffd640 1 -- 172.21.15.105:0/2716977993 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2dfc022030 con 0x7f2e080cfc50 2024-04-15T15:19:58.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.838+0000 7f2e0ce4a640 1 -- 172.21.15.105:0/2716977993 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2e0814e5e0 con 0x7f2e080cfc50 2024-04-15T15:19:58.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.839+0000 7f2df6ffd640 1 -- 172.21.15.105:0/2716977993 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16653 (secure 0 0 0) 0x7f2dfc05a170 con 0x7f2e080cfc50 2024-04-15T15:19:58.842 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:19:58.842 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":20,"fsid":"09e4d2a4-a324-4793-af77-644b25e520e1","created":"2024-04-15T15:19:01.524543+0000","modified":"2024-04-15T15:19:27.414890+0000","last_up_change":"2024-04-15T15:19:12.825815+0000","last_in_change":"2024-04-15T15:19:03.788126+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:19:13.748585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:19:23.789287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:19:24.722250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6f6292b-2d7e-4aa0-9d1c-6ddd0f97287c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6801","nonce":2873040054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6803","nonce":2873040054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6807","nonce":2873040054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6805","nonce":2873040054}]},"public_addr":"172.21.15.105:6801/2873040054","cluster_addr":"172.21.15.105:6803/2873040054","heartbeat_back_addr":"172.21.15.105:6807/2873040054","heartbeat_front_addr":"172.21.15.105:6805/2873040054","state":["exists","up"]},{"osd":1,"uuid":"8900966a-b393-41e3-8f93-e5a43bd23622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6818","nonce":2494994479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6821","nonce":2494994479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6828","nonce":2494994479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6823","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6825","nonce":2494994479}]},"public_addr":"172.21.15.105:6818/2494994479","cluster_addr":"172.21.15.105:6821/2494994479","heartbeat_back_addr":"172.21.15.105:6828/2494994479","heartbeat_front_addr":"172.21.15.105:6825/2494994479","state":["exists","up"]},{"osd":2,"uuid":"81a5dc7a-9325-4ff5-941d-a2d5fba21f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6809","nonce":3798408699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6811","nonce":3798408699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6815","nonce":3798408699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6813","nonce":3798408699}]},"public_addr":"172.21.15.105:6809/3798408699","cluster_addr":"172.21.15.105:6811/3798408699","heartbeat_back_addr":"172.21.15.105:6815/3798408699","heartbeat_front_addr":"172.21.15.105:6813/3798408699","state":["exists","up"]},{"osd":3,"uuid":"df766ed2-5c71-4805-96f4-3d4970edc960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6819","nonce":1920720186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6824","nonce":1920720186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6831","nonce":1920720186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6827","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6829","nonce":1920720186}]},"public_addr":"172.21.15.105:6819/1920720186","cluster_addr":"172.21.15.105:6824/1920720186","heartbeat_back_addr":"172.21.15.105:6831/1920720186","heartbeat_front_addr":"172.21.15.105:6829/1920720186","state":["exists","up"]},{"osd":4,"uuid":"e1039e03-28b8-49df-b951-c91930b93c9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6810","nonce":3516494456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6813","nonce":3516494456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6820","nonce":3516494456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6817","nonce":3516494456}]},"public_addr":"172.21.15.154:6810/3516494456","cluster_addr":"172.21.15.154:6813/3516494456","heartbeat_back_addr":"172.21.15.154:6820/3516494456","heartbeat_front_addr":"172.21.15.154:6817/3516494456","state":["exists","up"]},{"osd":5,"uuid":"a9f5eaf1-66ff-405a-a9e4-ec3340812a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6801","nonce":540367751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6803","nonce":540367751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6807","nonce":540367751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6805","nonce":540367751}]},"public_addr":"172.21.15.154:6801/540367751","cluster_addr":"172.21.15.154:6803/540367751","heartbeat_back_addr":"172.21.15.154:6807/540367751","heartbeat_front_addr":"172.21.15.154:6805/540367751","state":["exists","up"]},{"osd":6,"uuid":"267d02ea-675f-428f-aae8-e0375e4d4feb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6825","nonce":1550829005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6827","nonce":1550829005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6831","nonce":1550829005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6829","nonce":1550829005}]},"public_addr":"172.21.15.154:6825/1550829005","cluster_addr":"172.21.15.154:6827/1550829005","heartbeat_back_addr":"172.21.15.154:6831/1550829005","heartbeat_front_addr":"172.21.15.154:6829/1550829005","state":["exists","up"]},{"osd":7,"uuid":"759ac6cc-896a-423c-885f-9fe90d8dcb14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6809","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6811","nonce":3920089903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6816","nonce":3920089903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6823","nonce":3920089903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6819","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6821","nonce":3920089903}]},"public_addr":"172.21.15.154:6811/3920089903","cluster_addr":"172.21.15.154:6816/3920089903","heartbeat_back_addr":"172.21.15.154:6823/3920089903","heartbeat_front_addr":"172.21.15.154:6821/3920089903","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.949509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.984072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.907278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.947391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.975447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.998896+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:09.028739+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.894119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:19:58.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.841+0000 7f2e0ce4a640 1 -- 172.21.15.105:0/2716977993 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f2de806c8f0 msgr2=0x7f2de806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:58.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.841+0000 7f2e0ce4a640 1 --2- 172.21.15.105:0/2716977993 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f2de806c8f0 0x7f2de806edb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2df80099c0 tx=0x7f2df8003990 comp rx=0 tx=0).stop 2024-04-15T15:19:58.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.842+0000 7f2e0ce4a640 1 -- 172.21.15.105:0/2716977993 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e080cfc50 msgr2=0x7f2e08143f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:58.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.842+0000 7f2e0ce4a640 1 --2- 172.21.15.105:0/2716977993 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e080cfc50 0x7f2e08143f20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc00d800 tx=0x7f2dfc00dcc0 comp rx=0 tx=0).stop 2024-04-15T15:19:58.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.842+0000 7f2e0ce4a640 1 -- 172.21.15.105:0/2716977993 shutdown_connections 2024-04-15T15:19:58.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.842+0000 7f2e0ce4a640 1 --2- 172.21.15.105:0/2716977993 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f2de806c8f0 0x7f2de806edb0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:58.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.842+0000 7f2e0ce4a640 1 --2- 172.21.15.105:0/2716977993 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e081554d0 0x7f2e08140fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:58.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.842+0000 7f2e0ce4a640 1 --2- 172.21.15.105:0/2716977993 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e080cfc50 0x7f2e08143f20 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:58.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.842+0000 7f2e0ce4a640 1 --2- 172.21.15.105:0/2716977993 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e080b5580 0x7f2e081581d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:58.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.842+0000 7f2e0ce4a640 1 -- 172.21.15.105:0/2716977993 >> 172.21.15.105:0/2716977993 conn(0x7f2e08067d50 msgr2=0x7f2e0815aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:58.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.842+0000 7f2e0ce4a640 1 -- 172.21.15.105:0/2716977993 shutdown_connections 2024-04-15T15:19:58.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.842+0000 7f2e0ce4a640 1 -- 172.21.15.105:0/2716977993 wait complete. 2024-04-15T15:19:58.855 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:19:13.748585+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-15T15:19:23.789287+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-15T15:19:24.722250+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:19:58.855 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:19:58.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.960+0000 7f7c24f0e640 1 Processor -- start 2024-04-15T15:19:58.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.960+0000 7f7c24f0e640 1 -- start start 2024-04-15T15:19:58.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.960+0000 7f7c24f0e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7c2015e8d0 0x7f7c200d2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:58.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.960+0000 7f7c24f0e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7c200ba480 0x7f7c200d1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:58.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.961+0000 7f7c24f0e640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c200d3400 con 0x7f7c200d1650 2024-04-15T15:19:58.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.961+0000 7f7c24f0e640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f7c200d1ac0 con 0x7f7c2015e8d0 2024-04-15T15:19:58.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.961+0000 7f7c24f0e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f7c200d36e0 con 0x7f7c200ba480 2024-04-15T15:19:58.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.961+0000 7f7c1e59c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7c2015e8d0 0x7f7c200d2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:58.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.961+0000 7f7c1ed9d640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f7c200d1650 0x7f7c200b61d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:57458/0 (socket says 172.21.15.105:57458) 2024-04-15T15:19:58.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.961+0000 7f7c1ed9d640 1 -- 172.21.15.105:0/1773885718 learned_addr learned my addr 172.21.15.105:0/1773885718 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:19:58.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.961+0000 7f7c17fff640 1 --2- 172.21.15.105:0/1773885718 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7c200ba480 0x7f7c200d1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:58.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.961+0000 7f7c1e59c640 1 -- 172.21.15.105:0/1773885718 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7c200ba480 msgr2=0x7f7c200d1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:58.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.961+0000 7f7c1e59c640 1 --2- 172.21.15.105:0/1773885718 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7c200ba480 0x7f7c200d1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:58.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.961+0000 7f7c1e59c640 1 -- 172.21.15.105:0/1773885718 >> v1:172.21.15.105:6789/0 conn(0x7f7c200d1650 legacy=0x7f7c200b61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:58.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.961+0000 7f7c1e59c640 1 -- 172.21.15.105:0/1773885718 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c200d39c0 con 0x7f7c2015e8d0 2024-04-15T15:19:58.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.961+0000 7f7c1dd9b640 1 -- 172.21.15.105:0/1773885718 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2606085641 0 0) 0x7f7c200d3400 con 0x7f7c200d1650 2024-04-15T15:19:58.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.962+0000 7f7c1e59c640 1 --2- 172.21.15.105:0/1773885718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c2015e8d0 0x7f7c200d2ab0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7c100097a0 tx=0x7f7c1002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4686848f38b8cd46 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:58.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.962+0000 7f7c1dd9b640 1 -- 172.21.15.105:0/1773885718 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c10034070 con 0x7f7c2015e8d0 2024-04-15T15:19:58.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.962+0000 7f7c1dd9b640 1 -- 172.21.15.105:0/1773885718 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c10030610 con 0x7f7c2015e8d0 2024-04-15T15:19:58.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.962+0000 7f7c1dd9b640 1 -- 172.21.15.105:0/1773885718 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c100309e0 con 0x7f7c2015e8d0 2024-04-15T15:19:58.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.962+0000 7f7c24f0e640 1 -- 172.21.15.105:0/1773885718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c2015e8d0 msgr2=0x7f7c200d2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:58.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.962+0000 7f7c24f0e640 1 --2- 172.21.15.105:0/1773885718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c2015e8d0 0x7f7c200d2ab0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7c100097a0 tx=0x7f7c1002b960 comp rx=0 tx=0).stop 2024-04-15T15:19:58.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.962+0000 7f7c24f0e640 1 -- 172.21.15.105:0/1773885718 shutdown_connections 2024-04-15T15:19:58.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.962+0000 7f7c24f0e640 1 --2- 172.21.15.105:0/1773885718 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7c200ba480 0x7f7c200d1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:58.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.962+0000 7f7c24f0e640 1 --2- 172.21.15.105:0/1773885718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c2015e8d0 0x7f7c200d2ab0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:58.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.962+0000 7f7c24f0e640 1 -- 172.21.15.105:0/1773885718 >> 172.21.15.105:0/1773885718 conn(0x7f7c2006d460 msgr2=0x7f7c20067720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:58.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.963+0000 7f7c24f0e640 1 -- 172.21.15.105:0/1773885718 shutdown_connections 2024-04-15T15:19:58.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.963+0000 7f7c24f0e640 1 -- 172.21.15.105:0/1773885718 wait complete. 2024-04-15T15:19:58.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.964+0000 7f7c24f0e640 1 Processor -- start 2024-04-15T15:19:58.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.964+0000 7f7c24f0e640 1 -- start start 2024-04-15T15:19:58.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.964+0000 7f7c24f0e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7c200ba480 0x7f7c20151fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:58.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.964+0000 7f7c24f0e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c200d1650 0x7f7c200c9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:58.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.965+0000 7f7c24f0e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c2015e8d0 0x7f7c200c9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:58.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.965+0000 7f7c24f0e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7c200d44a0 con 0x7f7c200d1650 2024-04-15T15:19:58.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.965+0000 7f7c24f0e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7c200d3f80 con 0x7f7c2015e8d0 2024-04-15T15:19:58.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.965+0000 7f7c24f0e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f7c200b8890 con 0x7f7c200ba480 2024-04-15T15:19:58.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.965+0000 7f7c17fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c200d1650 0x7f7c200c9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:58.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.965+0000 7f7c17fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c200d1650 0x7f7c200c9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47112/0 (socket says 172.21.15.105:47112) 2024-04-15T15:19:58.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.965+0000 7f7c17fff640 1 -- 172.21.15.105:0/2046392839 learned_addr learned my addr 172.21.15.105:0/2046392839 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:58.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.965+0000 7f7c1e59c640 1 --2- 172.21.15.105:0/2046392839 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7c200ba480 0x7f7c20151fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:58.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.965+0000 7f7c1ed9d640 1 --2- 172.21.15.105:0/2046392839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c2015e8d0 0x7f7c200c9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:58.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.965+0000 7f7c17fff640 1 -- 172.21.15.105:0/2046392839 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7c200ba480 msgr2=0x7f7c20151fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:58.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.965+0000 7f7c17fff640 1 --2- 172.21.15.105:0/2046392839 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7c200ba480 0x7f7c20151fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:58.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.965+0000 7f7c17fff640 1 -- 172.21.15.105:0/2046392839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c2015e8d0 msgr2=0x7f7c200c9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:58.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.965+0000 7f7c17fff640 1 --2- 172.21.15.105:0/2046392839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c2015e8d0 0x7f7c200c9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:58.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.965+0000 7f7c17fff640 1 -- 172.21.15.105:0/2046392839 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c2015f990 con 0x7f7c200d1650 2024-04-15T15:19:58.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.965+0000 7f7c1e59c640 1 --2- 172.21.15.105:0/2046392839 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7c200ba480 0x7f7c20151fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:19:58.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.965+0000 7f7c17fff640 1 --2- 172.21.15.105:0/2046392839 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c200d1650 0x7f7c200c9960 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7c0800e9b0 tx=0x7f7c0800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:58.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.965+0000 7f7c16ffd640 1 -- 172.21.15.105:0/2046392839 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c08017070 con 0x7f7c200d1650 2024-04-15T15:19:58.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.965+0000 7f7c16ffd640 1 -- 172.21.15.105:0/2046392839 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c0800c9a0 con 0x7f7c200d1650 2024-04-15T15:19:58.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.965+0000 7f7c24f0e640 1 -- 172.21.15.105:0/2046392839 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c201528d0 con 0x7f7c200d1650 2024-04-15T15:19:58.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.965+0000 7f7c16ffd640 1 -- 172.21.15.105:0/2046392839 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0800cd70 con 0x7f7c200d1650 2024-04-15T15:19:58.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.965+0000 7f7c24f0e640 1 -- 172.21.15.105:0/2046392839 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c2015fc70 con 0x7f7c200d1650 2024-04-15T15:19:58.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.966+0000 7f7c24f0e640 1 -- 172.21.15.105:0/2046392839 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c20068ab0 con 0x7f7c200d1650 2024-04-15T15:19:58.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.967+0000 7f7c16ffd640 1 -- 172.21.15.105:0/2046392839 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7c0801a070 con 0x7f7c200d1650 2024-04-15T15:19:58.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.967+0000 7f7c16ffd640 1 --2- 172.21.15.105:0/2046392839 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f7c0006c8f0 0x7f7c0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:58.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.967+0000 7f7c16ffd640 1 -- 172.21.15.105:0/2046392839 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7c0808be10 con 0x7f7c200d1650 2024-04-15T15:19:58.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.967+0000 7f7c1e59c640 1 --2- 172.21.15.105:0/2046392839 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f7c0006c8f0 0x7f7c0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:58.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.968+0000 7f7c1e59c640 1 --2- 172.21.15.105:0/2046392839 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f7c0006c8f0 0x7f7c0006edb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7c10036000 tx=0x7f7c1003b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:58.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:58.972+0000 7f7c16ffd640 1 -- 172.21.15.105:0/2046392839 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7c08059bf0 con 0x7f7c200d1650 2024-04-15T15:19:59.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.148+0000 7f7c24f0e640 1 -- 172.21.15.105:0/2046392839 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7c2015e010 con 0x7f7c200d1650 2024-04-15T15:19:59.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.148+0000 7f7c16ffd640 1 -- 172.21.15.105:0/2046392839 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f7c0805def0 con 0x7f7c200d1650 2024-04-15T15:19:59.150 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-04-15T15:19:59.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.151+0000 7f7c24f0e640 1 -- 172.21.15.105:0/2046392839 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f7c0006c8f0 msgr2=0x7f7c0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:59.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.151+0000 7f7c24f0e640 1 --2- 172.21.15.105:0/2046392839 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f7c0006c8f0 0x7f7c0006edb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7c10036000 tx=0x7f7c1003b040 comp rx=0 tx=0).stop 2024-04-15T15:19:59.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.151+0000 7f7c24f0e640 1 -- 172.21.15.105:0/2046392839 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c200d1650 msgr2=0x7f7c200c9960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:59.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.151+0000 7f7c24f0e640 1 --2- 172.21.15.105:0/2046392839 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c200d1650 0x7f7c200c9960 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7c0800e9b0 tx=0x7f7c0800ee70 comp rx=0 tx=0).stop 2024-04-15T15:19:59.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.151+0000 7f7c24f0e640 1 -- 172.21.15.105:0/2046392839 shutdown_connections 2024-04-15T15:19:59.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.151+0000 7f7c24f0e640 1 --2- 172.21.15.105:0/2046392839 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f7c0006c8f0 0x7f7c0006edb0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:59.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.151+0000 7f7c24f0e640 1 --2- 172.21.15.105:0/2046392839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c2015e8d0 0x7f7c200c9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:59.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.151+0000 7f7c24f0e640 1 --2- 172.21.15.105:0/2046392839 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c200d1650 0x7f7c200c9960 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:59.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.151+0000 7f7c24f0e640 1 --2- 172.21.15.105:0/2046392839 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7c200ba480 0x7f7c20151fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:59.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.151+0000 7f7c24f0e640 1 -- 172.21.15.105:0/2046392839 >> 172.21.15.105:0/2046392839 conn(0x7f7c2006d460 msgr2=0x7f7c200c8d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:59.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.151+0000 7f7c24f0e640 1 -- 172.21.15.105:0/2046392839 shutdown_connections 2024-04-15T15:19:59.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.151+0000 7f7c24f0e640 1 -- 172.21.15.105:0/2046392839 wait complete. 2024-04-15T15:19:59.163 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-15T15:19:59.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.267+0000 7fad55c0d640 1 Processor -- start 2024-04-15T15:19:59.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.268+0000 7fad55c0d640 1 -- start start 2024-04-15T15:19:59.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.268+0000 7fad55c0d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fad500d0bf0 0x7fad500b5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:59.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.268+0000 7fad55c0d640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad500b3110 con 0x7fad500b59a0 2024-04-15T15:19:59.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.268+0000 7fad55c0d640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad500d10f0 con 0x7fad50160410 2024-04-15T15:19:59.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.268+0000 7fad55c0d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fad500d13d0 con 0x7fad500d0bf0 2024-04-15T15:19:59.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.269+0000 7fad4ffff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fad500b59a0 0x7fad501607f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:57470/0 (socket says 172.21.15.105:57470) 2024-04-15T15:19:59.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.269+0000 7fad4ffff640 1 -- 172.21.15.105:0/3209352844 learned_addr learned my addr 172.21.15.105:0/3209352844 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:19:59.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.269+0000 7fad46ffd640 1 --2- 172.21.15.105:0/3209352844 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fad500d0bf0 0x7fad500b5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:59.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.269+0000 7fad4effd640 1 -- 172.21.15.105:0/3209352844 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 651182935 0 0) 0x7fad48001490 con 0x7fad500b59a0 2024-04-15T15:19:59.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.269+0000 7fad4effd640 1 -- 172.21.15.105:0/3209352844 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fad3c003520 con 0x7fad500b59a0 2024-04-15T15:19:59.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.269+0000 7fad4effd640 1 -- 172.21.15.105:0/3209352844 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2239446900 0 0) 0x7fad3c003520 con 0x7fad500b59a0 2024-04-15T15:19:59.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.269+0000 7fad4effd640 1 -- 172.21.15.105:0/3209352844 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fad500d0bf0 msgr2=0x7fad500b5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:59.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.269+0000 7fad4effd640 1 --2- 172.21.15.105:0/3209352844 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fad500d0bf0 0x7fad500b5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:59.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.269+0000 7fad4effd640 1 -- 172.21.15.105:0/3209352844 >> v1:172.21.15.154:6789/0 conn(0x7fad50160410 legacy=0x7fad500cdd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:59.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.269+0000 7fad4effd640 1 -- 172.21.15.105:0/3209352844 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad500cd500 con 0x7fad500b59a0 2024-04-15T15:19:59.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.269+0000 7fad4effd640 1 -- 172.21.15.105:0/3209352844 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7fad48001bc0 con 0x7fad500b59a0 2024-04-15T15:19:59.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.269+0000 7fad4effd640 1 -- 172.21.15.105:0/3209352844 >> v1:172.21.15.105:6789/0 conn(0x7fad500b59a0 legacy=0x7fad501607f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:59.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.269+0000 7fad4effd640 1 --2- 172.21.15.105:0/3209352844 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad3c0039e0 0x7fad3c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:59.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.269+0000 7fad4effd640 1 --2- 172.21.15.105:0/3209352844 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad3c004390 0x7fad3c004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:59.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.269+0000 7fad4effd640 1 --2- 172.21.15.105:0/3209352844 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fad500d0bf0 0x7fad3c0050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:59.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.269+0000 7fad4effd640 1 -- 172.21.15.105:0/3209352844 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fad500d13d0 con 0x7fad3c0039e0 2024-04-15T15:19:59.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.269+0000 7fad4effd640 1 -- 172.21.15.105:0/3209352844 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fad3c00b9f0 con 0x7fad3c004390 2024-04-15T15:19:59.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.269+0000 7fad4effd640 1 -- 172.21.15.105:0/3209352844 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fad3c00bbf0 con 0x7fad500d0bf0 2024-04-15T15:19:59.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.269+0000 7fad4f7fe640 1 --2- 172.21.15.105:0/3209352844 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad3c0039e0 0x7fad3c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:59.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.269+0000 7fad46ffd640 1 --2- 172.21.15.105:0/3209352844 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad3c004390 0x7fad3c004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:59.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.269+0000 7fad4ffff640 1 --2- 172.21.15.105:0/3209352844 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fad500d0bf0 0x7fad3c0050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:59.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.270+0000 7fad4f7fe640 1 -- 172.21.15.105:0/3209352844 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fad500d0bf0 msgr2=0x7fad3c0050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:59.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.270+0000 7fad4f7fe640 1 --2- 172.21.15.105:0/3209352844 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fad500d0bf0 0x7fad3c0050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:59.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.270+0000 7fad4f7fe640 1 -- 172.21.15.105:0/3209352844 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad3c004390 msgr2=0x7fad3c004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:59.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.270+0000 7fad4f7fe640 1 --2- 172.21.15.105:0/3209352844 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad3c004390 0x7fad3c004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:59.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.270+0000 7fad4f7fe640 1 -- 172.21.15.105:0/3209352844 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad3c003520 con 0x7fad3c0039e0 2024-04-15T15:19:59.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.270+0000 7fad4ffff640 1 --2- 172.21.15.105:0/3209352844 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fad500d0bf0 0x7fad3c0050c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:19:59.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.270+0000 7fad4f7fe640 1 --2- 172.21.15.105:0/3209352844 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad3c0039e0 0x7fad3c003dc0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fad40001910 tx=0x7fad3c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=265f67336963da0c server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:59.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.270+0000 7fad4effd640 1 -- 172.21.15.105:0/3209352844 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad400014e0 con 0x7fad3c0039e0 2024-04-15T15:19:59.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.270+0000 7fad4effd640 1 -- 172.21.15.105:0/3209352844 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad40037070 con 0x7fad3c0039e0 2024-04-15T15:19:59.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.270+0000 7fad4effd640 1 -- 172.21.15.105:0/3209352844 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad40032870 con 0x7fad3c0039e0 2024-04-15T15:19:59.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.271+0000 7fad55c0d640 1 -- 172.21.15.105:0/3209352844 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad3c0039e0 msgr2=0x7fad3c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:59.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.271+0000 7fad55c0d640 1 --2- 172.21.15.105:0/3209352844 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad3c0039e0 0x7fad3c003dc0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fad40001910 tx=0x7fad3c005830 comp rx=0 tx=0).stop 2024-04-15T15:19:59.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.271+0000 7fad4ffff640 1 -- 172.21.15.105:0/3209352844 reap_dead start 2024-04-15T15:19:59.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.271+0000 7fad55c0d640 1 -- 172.21.15.105:0/3209352844 shutdown_connections 2024-04-15T15:19:59.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.271+0000 7fad55c0d640 1 -- 172.21.15.105:0/3209352844 >> 172.21.15.105:0/3209352844 conn(0x7fad500b4100 msgr2=0x7fad500b4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:59.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.271+0000 7fad55c0d640 1 -- 172.21.15.105:0/3209352844 shutdown_connections 2024-04-15T15:19:59.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.271+0000 7fad55c0d640 1 -- 172.21.15.105:0/3209352844 wait complete. 2024-04-15T15:19:59.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.272+0000 7fad55c0d640 1 Processor -- start 2024-04-15T15:19:59.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.272+0000 7fad55c0d640 1 -- start start 2024-04-15T15:19:59.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.273+0000 7fad55c0d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad3c0039e0 0x7fad500b1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:59.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.273+0000 7fad55c0d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fad500caeb0 0x7fad501f2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:59.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.273+0000 7fad55c0d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad500cb310 0x7fad500c8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:59.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.273+0000 7fad55c0d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fad500ce410 con 0x7fad3c0039e0 2024-04-15T15:19:59.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.273+0000 7fad55c0d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fad500b3540 con 0x7fad500cb310 2024-04-15T15:19:59.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.273+0000 7fad55c0d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fad500cd120 con 0x7fad500caeb0 2024-04-15T15:19:59.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.273+0000 7fad4f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad3c0039e0 0x7fad500b1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:59.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.273+0000 7fad4f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad3c0039e0 0x7fad500b1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47124/0 (socket says 172.21.15.105:47124) 2024-04-15T15:19:59.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.273+0000 7fad4f7fe640 1 -- 172.21.15.105:0/990742118 learned_addr learned my addr 172.21.15.105:0/990742118 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:59.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.273+0000 7fad4f7fe640 1 -- 172.21.15.105:0/990742118 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fad500caeb0 msgr2=0x7fad501f2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:59.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.273+0000 7fad4f7fe640 1 --2- 172.21.15.105:0/990742118 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fad500caeb0 0x7fad501f2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:59.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.273+0000 7fad4f7fe640 1 -- 172.21.15.105:0/990742118 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad500cb310 msgr2=0x7fad500c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:59.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.273+0000 7fad4f7fe640 1 --2- 172.21.15.105:0/990742118 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad500cb310 0x7fad500c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:59.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.273+0000 7fad4f7fe640 1 -- 172.21.15.105:0/990742118 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad500c75e0 con 0x7fad3c0039e0 2024-04-15T15:19:59.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.274+0000 7fad4f7fe640 1 --2- 172.21.15.105:0/990742118 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad3c0039e0 0x7fad500b1b90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fad40001910 tx=0x7fad400012b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:59.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.274+0000 7fad4cff9640 1 -- 172.21.15.105:0/990742118 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad40007a70 con 0x7fad3c0039e0 2024-04-15T15:19:59.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.274+0000 7fad4cff9640 1 -- 172.21.15.105:0/990742118 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad40040070 con 0x7fad3c0039e0 2024-04-15T15:19:59.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.274+0000 7fad55c0d640 1 -- 172.21.15.105:0/990742118 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad500c78c0 con 0x7fad3c0039e0 2024-04-15T15:19:59.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.274+0000 7fad4cff9640 1 -- 172.21.15.105:0/990742118 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad40005270 con 0x7fad3c0039e0 2024-04-15T15:19:59.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.274+0000 7fad55c0d640 1 -- 172.21.15.105:0/990742118 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad500c7e80 con 0x7fad3c0039e0 2024-04-15T15:19:59.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.275+0000 7fad55c0d640 1 -- 172.21.15.105:0/990742118 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad50069150 con 0x7fad3c0039e0 2024-04-15T15:19:59.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.275+0000 7fad4cff9640 1 -- 172.21.15.105:0/990742118 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fad400035f0 con 0x7fad3c0039e0 2024-04-15T15:19:59.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.276+0000 7fad4cff9640 1 --2- 172.21.15.105:0/990742118 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fad3006c8f0 0x7fad3006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:59.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.276+0000 7fad4cff9640 1 -- 172.21.15.105:0/990742118 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fad400addc0 con 0x7fad3c0039e0 2024-04-15T15:19:59.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.276+0000 7fad46ffd640 1 --2- 172.21.15.105:0/990742118 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fad3006c8f0 0x7fad3006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:59.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.276+0000 7fad46ffd640 1 --2- 172.21.15.105:0/990742118 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fad3006c8f0 0x7fad3006edb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fad500cc4e0 tx=0x7fad38006c30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:59.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.280+0000 7fad4cff9640 1 -- 172.21.15.105:0/990742118 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fad4007c160 con 0x7fad3c0039e0 2024-04-15T15:19:59.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.454+0000 7fad55c0d640 1 -- 172.21.15.105:0/990742118 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fad500b3ee0 con 0x7fad3c0039e0 2024-04-15T15:19:59.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.455+0000 7fad4cff9640 1 -- 172.21.15.105:0/990742118 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fad4007ba60 con 0x7fad3c0039e0 2024-04-15T15:19:59.457 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-04-15T15:19:59.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.457+0000 7fad55c0d640 1 -- 172.21.15.105:0/990742118 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fad3006c8f0 msgr2=0x7fad3006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:59.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.457+0000 7fad55c0d640 1 --2- 172.21.15.105:0/990742118 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fad3006c8f0 0x7fad3006edb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fad500cc4e0 tx=0x7fad38006c30 comp rx=0 tx=0).stop 2024-04-15T15:19:59.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.457+0000 7fad55c0d640 1 -- 172.21.15.105:0/990742118 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad3c0039e0 msgr2=0x7fad500b1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:59.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.457+0000 7fad55c0d640 1 --2- 172.21.15.105:0/990742118 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad3c0039e0 0x7fad500b1b90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fad40001910 tx=0x7fad400012b0 comp rx=0 tx=0).stop 2024-04-15T15:19:59.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.457+0000 7fad55c0d640 1 -- 172.21.15.105:0/990742118 shutdown_connections 2024-04-15T15:19:59.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.457+0000 7fad55c0d640 1 --2- 172.21.15.105:0/990742118 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fad500caeb0 0x7fad501f2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:59.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.457+0000 7fad55c0d640 1 --2- 172.21.15.105:0/990742118 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fad3006c8f0 0x7fad3006edb0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:59.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.457+0000 7fad55c0d640 1 --2- 172.21.15.105:0/990742118 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad500cb310 0x7fad500c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:59.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.457+0000 7fad55c0d640 1 --2- 172.21.15.105:0/990742118 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad3c0039e0 0x7fad500b1b90 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:59.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.457+0000 7fad55c0d640 1 -- 172.21.15.105:0/990742118 >> 172.21.15.105:0/990742118 conn(0x7fad500b4100 msgr2=0x7fad500b6230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:59.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.457+0000 7fad55c0d640 1 -- 172.21.15.105:0/990742118 shutdown_connections 2024-04-15T15:19:59.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.457+0000 7fad55c0d640 1 -- 172.21.15.105:0/990742118 wait complete. 2024-04-15T15:19:59.470 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-15T15:19:59.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.575+0000 7f5c5e715640 1 Processor -- start 2024-04-15T15:19:59.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.575+0000 7f5c5e715640 1 -- start start 2024-04-15T15:19:59.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.576+0000 7f5c5e715640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5c5806e460 0x7f5c580700f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:59.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.576+0000 7f5c5e715640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5c5806e840 0x7f5c5809aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:59.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.576+0000 7f5c5e715640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c5806cad0 0x7f5c5809af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:59.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.576+0000 7f5c5e715640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f5c58070840 con 0x7f5c5806e460 2024-04-15T15:19:59.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.576+0000 7f5c5e715640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f5c58070b20 con 0x7f5c5806e840 2024-04-15T15:19:59.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.576+0000 7f5c5e715640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f5c58070e00 con 0x7f5c5806cad0 2024-04-15T15:19:59.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.576+0000 7f5c57fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5c5806e460 0x7f5c580700f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:59.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.576+0000 7f5c57fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5c5806e460 0x7f5c580700f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47132/0 (socket says 172.21.15.105:47132) 2024-04-15T15:19:59.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.576+0000 7f5c57fff640 1 -- 172.21.15.105:0/4227958558 learned_addr learned my addr 172.21.15.105:0/4227958558 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:59.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.576+0000 7f5c5ccb2640 1 --2- 172.21.15.105:0/4227958558 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c5806cad0 0x7f5c5809af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:59.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.576+0000 7f5c577fe640 1 --2- 172.21.15.105:0/4227958558 >> v2:172.21.15.154:3300/0 conn(0x7f5c5806e840 0x7f5c5809aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:59.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.576+0000 7f5c57fff640 1 -- 172.21.15.105:0/4227958558 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c5806cad0 msgr2=0x7f5c5809af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:59.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.576+0000 7f5c57fff640 1 --2- 172.21.15.105:0/4227958558 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c5806cad0 0x7f5c5809af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:59.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.576+0000 7f5c57fff640 1 -- 172.21.15.105:0/4227958558 >> v2:172.21.15.154:3300/0 conn(0x7f5c5806e840 msgr2=0x7f5c5809aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:59.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.576+0000 7f5c57fff640 1 --2- 172.21.15.105:0/4227958558 >> v2:172.21.15.154:3300/0 conn(0x7f5c5806e840 0x7f5c5809aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:59.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.576+0000 7f5c57fff640 1 -- 172.21.15.105:0/4227958558 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c580710e0 con 0x7f5c5806e460 2024-04-15T15:19:59.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.576+0000 7f5c577fe640 1 --2- 172.21.15.105:0/4227958558 >> v2:172.21.15.154:3300/0 conn(0x7f5c5806e840 0x7f5c5809aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:59.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.577+0000 7f5c57fff640 1 --2- 172.21.15.105:0/4227958558 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c5806e460 0x7f5c580700f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5c58071c50 tx=0x7f5c4802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbb533909bc8830 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:59.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.577+0000 7f5c56ffd640 1 -- 172.21.15.105:0/4227958558 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c48035020 con 0x7f5c5806e460 2024-04-15T15:19:59.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.577+0000 7f5c56ffd640 1 -- 172.21.15.105:0/4227958558 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c48031870 con 0x7f5c5806e460 2024-04-15T15:19:59.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.577+0000 7f5c56ffd640 1 -- 172.21.15.105:0/4227958558 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c48031c40 con 0x7f5c5806e460 2024-04-15T15:19:59.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.578+0000 7f5c5e715640 1 -- 172.21.15.105:0/4227958558 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c5806e460 msgr2=0x7f5c580700f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:59.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.578+0000 7f5c5e715640 1 --2- 172.21.15.105:0/4227958558 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c5806e460 0x7f5c580700f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5c58071c50 tx=0x7f5c4802b910 comp rx=0 tx=0).stop 2024-04-15T15:19:59.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.578+0000 7f5c5e715640 1 -- 172.21.15.105:0/4227958558 shutdown_connections 2024-04-15T15:19:59.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.578+0000 7f5c5e715640 1 --2- 172.21.15.105:0/4227958558 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c5806cad0 0x7f5c5809af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:59.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.578+0000 7f5c5e715640 1 --2- 172.21.15.105:0/4227958558 >> v2:172.21.15.154:3300/0 conn(0x7f5c5806e840 0x7f5c5809aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:59.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.578+0000 7f5c5e715640 1 --2- 172.21.15.105:0/4227958558 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c5806e460 0x7f5c580700f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:59.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.578+0000 7f5c5e715640 1 -- 172.21.15.105:0/4227958558 >> 172.21.15.105:0/4227958558 conn(0x7f5c580b4030 msgr2=0x7f5c580b4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:59.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.578+0000 7f5c5e715640 1 -- 172.21.15.105:0/4227958558 shutdown_connections 2024-04-15T15:19:59.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.578+0000 7f5c5e715640 1 -- 172.21.15.105:0/4227958558 wait complete. 2024-04-15T15:19:59.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.579+0000 7f5c5e715640 1 Processor -- start 2024-04-15T15:19:59.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.579+0000 7f5c5e715640 1 -- start start 2024-04-15T15:19:59.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.580+0000 7f5c5e715640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c5806cad0 0x7f5c581f5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:59.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.580+0000 7f5c5e715640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c5806e460 0x7f5c580b79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:59.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.580+0000 7f5c5e715640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c5806e840 0x7f5c581504b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:59.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.580+0000 7f5c5e715640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5c58162670 con 0x7f5c5806e840 2024-04-15T15:19:59.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.580+0000 7f5c5e715640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5c581624f0 con 0x7f5c5806e460 2024-04-15T15:19:59.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.580+0000 7f5c5e715640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f5c581627f0 con 0x7f5c5806cad0 2024-04-15T15:19:59.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.580+0000 7f5c5ccb2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c5806e840 0x7f5c581504b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:59.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.580+0000 7f5c5ccb2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c5806e840 0x7f5c581504b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47148/0 (socket says 172.21.15.105:47148) 2024-04-15T15:19:59.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.580+0000 7f5c5ccb2640 1 -- 172.21.15.105:0/2910168751 learned_addr learned my addr 172.21.15.105:0/2910168751 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:59.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.580+0000 7f5c57fff640 1 --2- 172.21.15.105:0/2910168751 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c5806cad0 0x7f5c581f5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:59.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.580+0000 7f5c577fe640 1 --2- 172.21.15.105:0/2910168751 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c5806e460 0x7f5c580b79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:59.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.580+0000 7f5c5ccb2640 1 -- 172.21.15.105:0/2910168751 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c5806cad0 msgr2=0x7f5c581f5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:59.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.580+0000 7f5c5ccb2640 1 --2- 172.21.15.105:0/2910168751 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c5806cad0 0x7f5c581f5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:59.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.580+0000 7f5c5ccb2640 1 -- 172.21.15.105:0/2910168751 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c5806e460 msgr2=0x7f5c580b79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:59.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.580+0000 7f5c5ccb2640 1 --2- 172.21.15.105:0/2910168751 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c5806e460 0x7f5c580b79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:59.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.580+0000 7f5c5ccb2640 1 -- 172.21.15.105:0/2910168751 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c581f5990 con 0x7f5c5806e840 2024-04-15T15:19:59.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.580+0000 7f5c5ccb2640 1 --2- 172.21.15.105:0/2910168751 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c5806e840 0x7f5c581504b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5c50007d50 tx=0x7f5c5000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:59.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.580+0000 7f5c54ff9640 1 -- 172.21.15.105:0/2910168751 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c50014420 con 0x7f5c5806e840 2024-04-15T15:19:59.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.580+0000 7f5c54ff9640 1 -- 172.21.15.105:0/2910168751 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c50019070 con 0x7f5c5806e840 2024-04-15T15:19:59.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.580+0000 7f5c5e715640 1 -- 172.21.15.105:0/2910168751 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c580c6320 con 0x7f5c5806e840 2024-04-15T15:19:59.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.580+0000 7f5c54ff9640 1 -- 172.21.15.105:0/2910168751 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c50014820 con 0x7f5c5806e840 2024-04-15T15:19:59.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.581+0000 7f5c5e715640 1 -- 172.21.15.105:0/2910168751 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c580c68e0 con 0x7f5c5806e840 2024-04-15T15:19:59.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.582+0000 7f5c5e715640 1 -- 172.21.15.105:0/2910168751 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c58068ab0 con 0x7f5c5806e840 2024-04-15T15:19:59.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.582+0000 7f5c54ff9640 1 -- 172.21.15.105:0/2910168751 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5c500149c0 con 0x7f5c5806e840 2024-04-15T15:19:59.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.582+0000 7f5c54ff9640 1 --2- 172.21.15.105:0/2910168751 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f5c4406c8f0 0x7f5c4406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:59.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.582+0000 7f5c54ff9640 1 -- 172.21.15.105:0/2910168751 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5c5008c350 con 0x7f5c5806e840 2024-04-15T15:19:59.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.582+0000 7f5c57fff640 1 --2- 172.21.15.105:0/2910168751 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f5c4406c8f0 0x7f5c4406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:59.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.583+0000 7f5c57fff640 1 --2- 172.21.15.105:0/2910168751 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f5c4406c8f0 0x7f5c4406edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5c48036000 tx=0x7f5c4803c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:59.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.587+0000 7f5c54ff9640 1 -- 172.21.15.105:0/2910168751 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5c5005a780 con 0x7f5c5806e840 2024-04-15T15:19:59.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.762+0000 7f5c5e715640 1 -- 172.21.15.105:0/2910168751 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5c5806f070 con 0x7f5c5806e840 2024-04-15T15:19:59.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.763+0000 7f5c54ff9640 1 -- 172.21.15.105:0/2910168751 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f5c5005a080 con 0x7f5c5806e840 2024-04-15T15:19:59.766 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-04-15T15:19:59.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.765+0000 7f5c5e715640 1 -- 172.21.15.105:0/2910168751 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f5c4406c8f0 msgr2=0x7f5c4406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:59.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.765+0000 7f5c5e715640 1 --2- 172.21.15.105:0/2910168751 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f5c4406c8f0 0x7f5c4406edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5c48036000 tx=0x7f5c4803c040 comp rx=0 tx=0).stop 2024-04-15T15:19:59.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.765+0000 7f5c5e715640 1 -- 172.21.15.105:0/2910168751 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c5806e840 msgr2=0x7f5c581504b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:59.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.765+0000 7f5c5e715640 1 --2- 172.21.15.105:0/2910168751 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c5806e840 0x7f5c581504b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5c50007d50 tx=0x7f5c5000e4b0 comp rx=0 tx=0).stop 2024-04-15T15:19:59.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.765+0000 7f5c5e715640 1 -- 172.21.15.105:0/2910168751 shutdown_connections 2024-04-15T15:19:59.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.765+0000 7f5c5e715640 1 --2- 172.21.15.105:0/2910168751 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f5c4406c8f0 0x7f5c4406edb0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:59.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.765+0000 7f5c5e715640 1 --2- 172.21.15.105:0/2910168751 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c5806e840 0x7f5c581504b0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:59.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.765+0000 7f5c5e715640 1 --2- 172.21.15.105:0/2910168751 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c5806e460 0x7f5c580b79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:59.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.765+0000 7f5c5e715640 1 --2- 172.21.15.105:0/2910168751 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c5806cad0 0x7f5c581f5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:59.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.765+0000 7f5c5e715640 1 -- 172.21.15.105:0/2910168751 >> 172.21.15.105:0/2910168751 conn(0x7f5c580b4030 msgr2=0x7f5c580af3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:59.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.765+0000 7f5c5e715640 1 -- 172.21.15.105:0/2910168751 shutdown_connections 2024-04-15T15:19:59.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.765+0000 7f5c5e715640 1 -- 172.21.15.105:0/2910168751 wait complete. 2024-04-15T15:19:59.778 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-15T15:19:59.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.880+0000 7f247b531640 1 Processor -- start 2024-04-15T15:19:59.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.880+0000 7f247b531640 1 -- start start 2024-04-15T15:19:59.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.881+0000 7f247b531640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f247415eb60 0x7f24740d11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:59.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.881+0000 7f247b531640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f24740d1710 0x7f2474162630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:59.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.881+0000 7f247b531640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f247415ef40 con 0x7f2474162b70 2024-04-15T15:19:59.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.881+0000 7f247b531640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f24740d1b60 con 0x7f247415eb60 2024-04-15T15:19:59.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.881+0000 7f247b531640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f2474163540 con 0x7f24740d1710 2024-04-15T15:19:59.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.881+0000 7f24792cd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f247415eb60 0x7f24740d11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:59.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.881+0000 7f24792cd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f247415eb60 0x7f24740d11d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47160/0 (socket says 172.21.15.105:47160) 2024-04-15T15:19:59.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.881+0000 7f24792cd640 1 -- 172.21.15.105:0/52714605 learned_addr learned my addr 172.21.15.105:0/52714605 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:59.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.881+0000 7f2479ace640 1 --1- 172.21.15.105:0/52714605 >> v1:172.21.15.105:6789/0 conn(0x7f2474162b70 0x7f24740b6240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:57474/0 (socket says 172.21.15.105:57474) 2024-04-15T15:19:59.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.882+0000 7f2478acc640 1 --2- 172.21.15.105:0/52714605 >> v2:172.21.15.154:3300/0 conn(0x7f24740d1710 0x7f2474162630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:59.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.882+0000 7f24792cd640 1 -- 172.21.15.105:0/52714605 >> v2:172.21.15.154:3300/0 conn(0x7f24740d1710 msgr2=0x7f2474162630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:59.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.882+0000 7f24792cd640 1 --2- 172.21.15.105:0/52714605 >> v2:172.21.15.154:3300/0 conn(0x7f24740d1710 0x7f2474162630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:59.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.882+0000 7f2473fff640 1 -- 172.21.15.105:0/52714605 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3415894795 0 0) 0x7f247415ef40 con 0x7f2474162b70 2024-04-15T15:19:59.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.882+0000 7f24792cd640 1 -- 172.21.15.105:0/52714605 >> v1:172.21.15.105:6789/0 conn(0x7f2474162b70 legacy=0x7f24740b6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:59.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.882+0000 7f24792cd640 1 -- 172.21.15.105:0/52714605 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2474163260 con 0x7f247415eb60 2024-04-15T15:19:59.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.882+0000 7f24792cd640 1 --2- 172.21.15.105:0/52714605 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f247415eb60 0x7f24740d11d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f24740b7680 tx=0x7f246402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de692e891d3b44e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:59.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.883+0000 7f2473fff640 1 -- 172.21.15.105:0/52714605 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246402cb20 con 0x7f247415eb60 2024-04-15T15:19:59.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.883+0000 7f2473fff640 1 -- 172.21.15.105:0/52714605 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f246402ccc0 con 0x7f247415eb60 2024-04-15T15:19:59.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.883+0000 7f2473fff640 1 -- 172.21.15.105:0/52714605 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2464034610 con 0x7f247415eb60 2024-04-15T15:19:59.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.883+0000 7f247b531640 1 -- 172.21.15.105:0/52714605 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f247415eb60 msgr2=0x7f24740d11d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:59.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.883+0000 7f247b531640 1 --2- 172.21.15.105:0/52714605 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f247415eb60 0x7f24740d11d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f24740b7680 tx=0x7f246402bf80 comp rx=0 tx=0).stop 2024-04-15T15:19:59.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.883+0000 7f247b531640 1 -- 172.21.15.105:0/52714605 shutdown_connections 2024-04-15T15:19:59.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.883+0000 7f247b531640 1 --2- 172.21.15.105:0/52714605 >> v2:172.21.15.154:3300/0 conn(0x7f24740d1710 0x7f2474162630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:59.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.883+0000 7f247b531640 1 --2- 172.21.15.105:0/52714605 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f247415eb60 0x7f24740d11d0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:59.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.883+0000 7f247b531640 1 -- 172.21.15.105:0/52714605 >> 172.21.15.105:0/52714605 conn(0x7f2474067720 msgr2=0x7f2474067b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:59.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.883+0000 7f247b531640 1 -- 172.21.15.105:0/52714605 shutdown_connections 2024-04-15T15:19:59.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.883+0000 7f247b531640 1 -- 172.21.15.105:0/52714605 wait complete. 2024-04-15T15:19:59.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.884+0000 7f247b531640 1 Processor -- start 2024-04-15T15:19:59.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.884+0000 7f247b531640 1 -- start start 2024-04-15T15:19:59.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.885+0000 7f247b531640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f24740d1710 0x7f247415dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:59.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.885+0000 7f247b531640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f247415eb60 0x7f24740b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:59.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.885+0000 7f247b531640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2474162b70 0x7f24741f4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:59.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.885+0000 7f247b531640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f24740d2c50 con 0x7f247415eb60 2024-04-15T15:19:59.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.885+0000 7f247b531640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f24740d2fc0 con 0x7f2474162b70 2024-04-15T15:19:59.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.885+0000 7f247b531640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f2474162f80 con 0x7f24740d1710 2024-04-15T15:19:59.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.885+0000 7f2478acc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f247415eb60 0x7f24740b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:59.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.885+0000 7f2478acc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f247415eb60 0x7f24740b4d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47172/0 (socket says 172.21.15.105:47172) 2024-04-15T15:19:59.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.885+0000 7f2478acc640 1 -- 172.21.15.105:0/4175473767 learned_addr learned my addr 172.21.15.105:0/4175473767 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:19:59.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.886+0000 7f2478acc640 1 -- 172.21.15.105:0/4175473767 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f24740d1710 msgr2=0x7f247415dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:59.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.886+0000 7f2478acc640 1 --2- 172.21.15.105:0/4175473767 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f24740d1710 0x7f247415dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:59.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.886+0000 7f2478acc640 1 -- 172.21.15.105:0/4175473767 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2474162b70 msgr2=0x7f24741f4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:59.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.886+0000 7f2479ace640 1 --2- 172.21.15.105:0/4175473767 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2474162b70 0x7f24741f4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:59.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.886+0000 7f2478acc640 1 --2- 172.21.15.105:0/4175473767 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2474162b70 0x7f24741f4520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:59.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.886+0000 7f2478acc640 1 -- 172.21.15.105:0/4175473767 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24740ccb30 con 0x7f247415eb60 2024-04-15T15:19:59.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.886+0000 7f2478acc640 1 --2- 172.21.15.105:0/4175473767 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f247415eb60 0x7f24740b4d70 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f246800d5f0 tx=0x7f246800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:59.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.886+0000 7f2471ffb640 1 -- 172.21.15.105:0/4175473767 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24680125f0 con 0x7f247415eb60 2024-04-15T15:19:59.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.886+0000 7f2471ffb640 1 -- 172.21.15.105:0/4175473767 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2468019070 con 0x7f247415eb60 2024-04-15T15:19:59.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.886+0000 7f2471ffb640 1 -- 172.21.15.105:0/4175473767 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24680129d0 con 0x7f247415eb60 2024-04-15T15:19:59.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.886+0000 7f247b531640 1 -- 172.21.15.105:0/4175473767 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24740cce10 con 0x7f247415eb60 2024-04-15T15:19:59.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.886+0000 7f247b531640 1 -- 172.21.15.105:0/4175473767 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24740cd3d0 con 0x7f247415eb60 2024-04-15T15:19:59.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.887+0000 7f247b531640 1 -- 172.21.15.105:0/4175473767 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24740b3500 con 0x7f247415eb60 2024-04-15T15:19:59.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.887+0000 7f2471ffb640 1 -- 172.21.15.105:0/4175473767 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2468012b70 con 0x7f247415eb60 2024-04-15T15:19:59.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.888+0000 7f2471ffb640 1 --2- 172.21.15.105:0/4175473767 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f246006c8f0 0x7f246006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:59.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.888+0000 7f2471ffb640 1 -- 172.21.15.105:0/4175473767 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f246808bec0 con 0x7f247415eb60 2024-04-15T15:19:59.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.888+0000 7f24792cd640 1 --2- 172.21.15.105:0/4175473767 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f246006c8f0 0x7f246006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:59.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.888+0000 7f24792cd640 1 --2- 172.21.15.105:0/4175473767 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f246006c8f0 0x7f246006edb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2464035000 tx=0x7f246402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:59.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:19:59.892+0000 7f2471ffb640 1 -- 172.21.15.105:0/4175473767 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f24740b3500 con 0x7f247415eb60 2024-04-15T15:20:00.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.078+0000 7f247b531640 1 -- 172.21.15.105:0/4175473767 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2474068a40 con 0x7f247415eb60 2024-04-15T15:20:00.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.079+0000 7f2471ffb640 1 -- 172.21.15.105:0/4175473767 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7f246805a380 con 0x7f247415eb60 2024-04-15T15:20:00.081 INFO:teuthology.orchestra.run.smithi105.stderr:listed 0 entries 2024-04-15T15:20:00.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.081+0000 7f247b531640 1 -- 172.21.15.105:0/4175473767 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f246006c8f0 msgr2=0x7f246006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:00.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.081+0000 7f247b531640 1 --2- 172.21.15.105:0/4175473767 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f246006c8f0 0x7f246006edb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2464035000 tx=0x7f246402e000 comp rx=0 tx=0).stop 2024-04-15T15:20:00.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.081+0000 7f247b531640 1 -- 172.21.15.105:0/4175473767 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f247415eb60 msgr2=0x7f24740b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:00.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.081+0000 7f247b531640 1 --2- 172.21.15.105:0/4175473767 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f247415eb60 0x7f24740b4d70 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f246800d5f0 tx=0x7f246800dab0 comp rx=0 tx=0).stop 2024-04-15T15:20:00.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.081+0000 7f247b531640 1 -- 172.21.15.105:0/4175473767 shutdown_connections 2024-04-15T15:20:00.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.081+0000 7f247b531640 1 --2- 172.21.15.105:0/4175473767 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f246006c8f0 0x7f246006edb0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:00.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.081+0000 7f247b531640 1 --2- 172.21.15.105:0/4175473767 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2474162b70 0x7f24741f4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:00.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.081+0000 7f247b531640 1 --2- 172.21.15.105:0/4175473767 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f247415eb60 0x7f24740b4d70 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:00.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.081+0000 7f247b531640 1 --2- 172.21.15.105:0/4175473767 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f24740d1710 0x7f247415dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:00.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.081+0000 7f247b531640 1 -- 172.21.15.105:0/4175473767 >> 172.21.15.105:0/4175473767 conn(0x7f2474067720 msgr2=0x7f24740c88e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:00.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.081+0000 7f247b531640 1 -- 172.21.15.105:0/4175473767 shutdown_connections 2024-04-15T15:20:00.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.081+0000 7f247b531640 1 -- 172.21.15.105:0/4175473767 wait complete. 2024-04-15T15:20:00.093 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-15T15:20:00.093 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:20:00.114 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:20:00.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.217+0000 7f1557e55640 1 Processor -- start 2024-04-15T15:20:00.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.217+0000 7f1557e55640 1 -- start start 2024-04-15T15:20:00.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.218+0000 7f1557e55640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f155013e680 0x7f15500cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:00.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.218+0000 7f1557e55640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f155006f6a0 0x7f155015f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:00.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.218+0000 7f1557e55640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f155015bc90 con 0x7f155013ec60 2024-04-15T15:20:00.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.218+0000 7f1557e55640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f155015bf70 con 0x7f155013e680 2024-04-15T15:20:00.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.218+0000 7f1557e55640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f155015c250 con 0x7f155006f6a0 2024-04-15T15:20:00.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.218+0000 7f1555bf1640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f155013e680 0x7f15500cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:00.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.218+0000 7f1555bf1640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f155013e680 0x7f15500cfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47188/0 (socket says 172.21.15.105:47188) 2024-04-15T15:20:00.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.218+0000 7f1555bf1640 1 -- 172.21.15.105:0/2050090007 learned_addr learned my addr 172.21.15.105:0/2050090007 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:00.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.218+0000 7f1555bf1640 1 -- 172.21.15.105:0/2050090007 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f155006f6a0 msgr2=0x7f155015f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:00.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.218+0000 7f1555bf1640 1 --2- 172.21.15.105:0/2050090007 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f155006f6a0 0x7f155015f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:00.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.218+0000 7f1555bf1640 1 -- 172.21.15.105:0/2050090007 >> v1:172.21.15.154:6789/0 conn(0x7f155013ec60 legacy=0x7f155006ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:00.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.218+0000 7f1555bf1640 1 -- 172.21.15.105:0/2050090007 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f155015c530 con 0x7f155013e680 2024-04-15T15:20:00.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.219+0000 7f1555bf1640 1 --2- 172.21.15.105:0/2050090007 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f155013e680 0x7f15500cfb70 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1540009870 tx=0x7f154002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d5ed4a1043a4681 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:00.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.219+0000 7f1554bef640 1 -- 172.21.15.105:0/2050090007 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1540035020 con 0x7f155013e680 2024-04-15T15:20:00.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.219+0000 7f1554bef640 1 -- 172.21.15.105:0/2050090007 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1540031870 con 0x7f155013e680 2024-04-15T15:20:00.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.219+0000 7f1554bef640 1 -- 172.21.15.105:0/2050090007 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1540031c40 con 0x7f155013e680 2024-04-15T15:20:00.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.219+0000 7f1557e55640 1 -- 172.21.15.105:0/2050090007 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f155013e680 msgr2=0x7f15500cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:00.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.219+0000 7f1557e55640 1 --2- 172.21.15.105:0/2050090007 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f155013e680 0x7f15500cfb70 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1540009870 tx=0x7f154002b910 comp rx=0 tx=0).stop 2024-04-15T15:20:00.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.220+0000 7f1557e55640 1 -- 172.21.15.105:0/2050090007 shutdown_connections 2024-04-15T15:20:00.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.220+0000 7f1557e55640 1 --2- 172.21.15.105:0/2050090007 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f155006f6a0 0x7f155015f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:00.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.220+0000 7f1557e55640 1 --2- 172.21.15.105:0/2050090007 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f155013e680 0x7f15500cfb70 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:00.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.220+0000 7f1557e55640 1 -- 172.21.15.105:0/2050090007 >> 172.21.15.105:0/2050090007 conn(0x7f1550067d50 msgr2=0x7f155006b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:00.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.220+0000 7f1557e55640 1 -- 172.21.15.105:0/2050090007 shutdown_connections 2024-04-15T15:20:00.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.220+0000 7f1557e55640 1 -- 172.21.15.105:0/2050090007 wait complete. 2024-04-15T15:20:00.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.221+0000 7f1557e55640 1 Processor -- start 2024-04-15T15:20:00.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.221+0000 7f1557e55640 1 -- start start 2024-04-15T15:20:00.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.222+0000 7f1557e55640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f155006f6a0 0x7f1550155590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:00.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.222+0000 7f1557e55640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f155013e680 0x7f155015a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:00.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.222+0000 7f1557e55640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f155013ec60 0x7f155015ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:00.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.222+0000 7f1557e55640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f15500d0230 con 0x7f155006f6a0 2024-04-15T15:20:00.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.222+0000 7f1557e55640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f15500d00b0 con 0x7f155013e680 2024-04-15T15:20:00.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.222+0000 7f1557e55640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f15500cf590 con 0x7f155013ec60 2024-04-15T15:20:00.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.222+0000 7f1555bf1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f155006f6a0 0x7f1550155590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:00.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.222+0000 7f1555bf1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f155006f6a0 0x7f1550155590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47196/0 (socket says 172.21.15.105:47196) 2024-04-15T15:20:00.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.222+0000 7f1555bf1640 1 -- 172.21.15.105:0/3628342021 learned_addr learned my addr 172.21.15.105:0/3628342021 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:00.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.222+0000 7f15553f0640 1 --2- 172.21.15.105:0/3628342021 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f155013e680 0x7f155015a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:00.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.222+0000 7f1555bf1640 1 -- 172.21.15.105:0/3628342021 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f155013ec60 msgr2=0x7f155015ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:00.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.222+0000 7f1555bf1640 1 --2- 172.21.15.105:0/3628342021 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f155013ec60 0x7f155015ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:00.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.222+0000 7f1555bf1640 1 -- 172.21.15.105:0/3628342021 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f155013e680 msgr2=0x7f155015a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:00.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.222+0000 7f1555bf1640 1 --2- 172.21.15.105:0/3628342021 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f155013e680 0x7f155015a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:00.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.222+0000 7f1555bf1640 1 -- 172.21.15.105:0/3628342021 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f155012e290 con 0x7f155006f6a0 2024-04-15T15:20:00.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.222+0000 7f1555bf1640 1 --2- 172.21.15.105:0/3628342021 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f155006f6a0 0x7f1550155590 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1540000c00 tx=0x7f15400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:00.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.222+0000 7f154e7fc640 1 -- 172.21.15.105:0/3628342021 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f154003ad60 con 0x7f155006f6a0 2024-04-15T15:20:00.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.222+0000 7f154e7fc640 1 -- 172.21.15.105:0/3628342021 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15400443f0 con 0x7f155006f6a0 2024-04-15T15:20:00.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.222+0000 7f154e7fc640 1 -- 172.21.15.105:0/3628342021 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15400447c0 con 0x7f155006f6a0 2024-04-15T15:20:00.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.222+0000 7f1557e55640 1 -- 172.21.15.105:0/3628342021 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f155012e570 con 0x7f155006f6a0 2024-04-15T15:20:00.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.223+0000 7f1557e55640 1 -- 172.21.15.105:0/3628342021 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15501e7600 con 0x7f155006f6a0 2024-04-15T15:20:00.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.224+0000 7f1557e55640 1 -- 172.21.15.105:0/3628342021 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1550068a40 con 0x7f155006f6a0 2024-04-15T15:20:00.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.224+0000 7f154e7fc640 1 -- 172.21.15.105:0/3628342021 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f154004b060 con 0x7f155006f6a0 2024-04-15T15:20:00.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.224+0000 7f154e7fc640 1 --2- 172.21.15.105:0/3628342021 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f153c06c8f0 0x7f153c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:00.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.224+0000 7f154e7fc640 1 -- 172.21.15.105:0/3628342021 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f15400addd0 con 0x7f155006f6a0 2024-04-15T15:20:00.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.224+0000 7f15553f0640 1 --2- 172.21.15.105:0/3628342021 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f153c06c8f0 0x7f153c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:00.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.225+0000 7f15553f0640 1 --2- 172.21.15.105:0/3628342021 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f153c06c8f0 0x7f153c06edb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f15501e6050 tx=0x7f15440072f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:00.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.229+0000 7f154e7fc640 1 -- 172.21.15.105:0/3628342021 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f154003f070 con 0x7f155006f6a0 2024-04-15T15:20:00.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.403+0000 7f1557e55640 1 -- 172.21.15.105:0/3628342021 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f15500b5780 con 0x7f155006f6a0 2024-04-15T15:20:00.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.404+0000 7f154e7fc640 1 -- 172.21.15.105:0/3628342021 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16653 (secure 0 0 0) 0x7f15400af090 con 0x7f155006f6a0 2024-04-15T15:20:00.406 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:20:00.406 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":20,"fsid":"09e4d2a4-a324-4793-af77-644b25e520e1","created":"2024-04-15T15:19:01.524543+0000","modified":"2024-04-15T15:19:27.414890+0000","last_up_change":"2024-04-15T15:19:12.825815+0000","last_in_change":"2024-04-15T15:19:03.788126+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:19:13.748585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:19:23.789287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:19:24.722250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6f6292b-2d7e-4aa0-9d1c-6ddd0f97287c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6801","nonce":2873040054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6803","nonce":2873040054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6807","nonce":2873040054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6805","nonce":2873040054}]},"public_addr":"172.21.15.105:6801/2873040054","cluster_addr":"172.21.15.105:6803/2873040054","heartbeat_back_addr":"172.21.15.105:6807/2873040054","heartbeat_front_addr":"172.21.15.105:6805/2873040054","state":["exists","up"]},{"osd":1,"uuid":"8900966a-b393-41e3-8f93-e5a43bd23622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6818","nonce":2494994479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6821","nonce":2494994479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6828","nonce":2494994479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6823","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6825","nonce":2494994479}]},"public_addr":"172.21.15.105:6818/2494994479","cluster_addr":"172.21.15.105:6821/2494994479","heartbeat_back_addr":"172.21.15.105:6828/2494994479","heartbeat_front_addr":"172.21.15.105:6825/2494994479","state":["exists","up"]},{"osd":2,"uuid":"81a5dc7a-9325-4ff5-941d-a2d5fba21f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6809","nonce":3798408699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6811","nonce":3798408699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6815","nonce":3798408699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6813","nonce":3798408699}]},"public_addr":"172.21.15.105:6809/3798408699","cluster_addr":"172.21.15.105:6811/3798408699","heartbeat_back_addr":"172.21.15.105:6815/3798408699","heartbeat_front_addr":"172.21.15.105:6813/3798408699","state":["exists","up"]},{"osd":3,"uuid":"df766ed2-5c71-4805-96f4-3d4970edc960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6819","nonce":1920720186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6824","nonce":1920720186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6831","nonce":1920720186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6827","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6829","nonce":1920720186}]},"public_addr":"172.21.15.105:6819/1920720186","cluster_addr":"172.21.15.105:6824/1920720186","heartbeat_back_addr":"172.21.15.105:6831/1920720186","heartbeat_front_addr":"172.21.15.105:6829/1920720186","state":["exists","up"]},{"osd":4,"uuid":"e1039e03-28b8-49df-b951-c91930b93c9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6810","nonce":3516494456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6813","nonce":3516494456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6820","nonce":3516494456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6817","nonce":3516494456}]},"public_addr":"172.21.15.154:6810/3516494456","cluster_addr":"172.21.15.154:6813/3516494456","heartbeat_back_addr":"172.21.15.154:6820/3516494456","heartbeat_front_addr":"172.21.15.154:6817/3516494456","state":["exists","up"]},{"osd":5,"uuid":"a9f5eaf1-66ff-405a-a9e4-ec3340812a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6801","nonce":540367751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6803","nonce":540367751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6807","nonce":540367751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6805","nonce":540367751}]},"public_addr":"172.21.15.154:6801/540367751","cluster_addr":"172.21.15.154:6803/540367751","heartbeat_back_addr":"172.21.15.154:6807/540367751","heartbeat_front_addr":"172.21.15.154:6805/540367751","state":["exists","up"]},{"osd":6,"uuid":"267d02ea-675f-428f-aae8-e0375e4d4feb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6825","nonce":1550829005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6827","nonce":1550829005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6831","nonce":1550829005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6829","nonce":1550829005}]},"public_addr":"172.21.15.154:6825/1550829005","cluster_addr":"172.21.15.154:6827/1550829005","heartbeat_back_addr":"172.21.15.154:6831/1550829005","heartbeat_front_addr":"172.21.15.154:6829/1550829005","state":["exists","up"]},{"osd":7,"uuid":"759ac6cc-896a-423c-885f-9fe90d8dcb14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6809","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6811","nonce":3920089903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6816","nonce":3920089903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6823","nonce":3920089903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6819","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6821","nonce":3920089903}]},"public_addr":"172.21.15.154:6811/3920089903","cluster_addr":"172.21.15.154:6816/3920089903","heartbeat_back_addr":"172.21.15.154:6823/3920089903","heartbeat_front_addr":"172.21.15.154:6821/3920089903","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.949509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.984072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.907278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.947391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.975447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.998896+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:09.028739+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.894119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:20:00.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.406+0000 7f1557e55640 1 -- 172.21.15.105:0/3628342021 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f153c06c8f0 msgr2=0x7f153c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:00.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.406+0000 7f1557e55640 1 --2- 172.21.15.105:0/3628342021 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f153c06c8f0 0x7f153c06edb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f15501e6050 tx=0x7f15440072f0 comp rx=0 tx=0).stop 2024-04-15T15:20:00.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.406+0000 7f1557e55640 1 -- 172.21.15.105:0/3628342021 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f155006f6a0 msgr2=0x7f1550155590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:00.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.406+0000 7f1557e55640 1 --2- 172.21.15.105:0/3628342021 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f155006f6a0 0x7f1550155590 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1540000c00 tx=0x7f15400046c0 comp rx=0 tx=0).stop 2024-04-15T15:20:00.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.406+0000 7f1557e55640 1 -- 172.21.15.105:0/3628342021 shutdown_connections 2024-04-15T15:20:00.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.406+0000 7f1557e55640 1 --2- 172.21.15.105:0/3628342021 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f155013ec60 0x7f155015ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:00.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.406+0000 7f1557e55640 1 --2- 172.21.15.105:0/3628342021 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f153c06c8f0 0x7f153c06edb0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:00.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.406+0000 7f1557e55640 1 --2- 172.21.15.105:0/3628342021 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f155013e680 0x7f155015a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:00.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.406+0000 7f1557e55640 1 --2- 172.21.15.105:0/3628342021 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f155006f6a0 0x7f1550155590 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:00.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.406+0000 7f1557e55640 1 -- 172.21.15.105:0/3628342021 >> 172.21.15.105:0/3628342021 conn(0x7f1550067d50 msgr2=0x7f155006de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:00.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.406+0000 7f1557e55640 1 -- 172.21.15.105:0/3628342021 shutdown_connections 2024-04-15T15:20:00.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.406+0000 7f1557e55640 1 -- 172.21.15.105:0/3628342021 wait complete. 2024-04-15T15:20:00.419 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:19:13.748585+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-15T15:19:23.789287+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-15T15:19:24.722250+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:20:00.419 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:20:00.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.523+0000 7f46abb72640 1 Processor -- start 2024-04-15T15:20:00.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.523+0000 7f46abb72640 1 -- start start 2024-04-15T15:20:00.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.524+0000 7f46abb72640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f46a4151e30 0x7f46a406eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:00.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.524+0000 7f46abb72640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46a40726f0 0x7f46a4072b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:00.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.524+0000 7f46abb72640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46a415bd20 con 0x7f46a415b1c0 2024-04-15T15:20:00.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.524+0000 7f46abb72640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f46a406e670 con 0x7f46a4151e30 2024-04-15T15:20:00.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.524+0000 7f46abb72640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f46a4073070 con 0x7f46a40726f0 2024-04-15T15:20:00.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.524+0000 7f46a990e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f46a4151e30 0x7f46a406eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:00.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.524+0000 7f46a990e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f46a4151e30 0x7f46a406eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37334/0 (socket says 172.21.15.105:37334) 2024-04-15T15:20:00.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.524+0000 7f46a990e640 1 -- 172.21.15.105:0/2538736078 learned_addr learned my addr 172.21.15.105:0/2538736078 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:00.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.524+0000 7f46a990e640 1 -- 172.21.15.105:0/2538736078 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46a40726f0 msgr2=0x7f46a4072b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:00.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.524+0000 7f46a990e640 1 --2- 172.21.15.105:0/2538736078 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46a40726f0 0x7f46a4072b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:00.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.524+0000 7f46a990e640 1 -- 172.21.15.105:0/2538736078 >> v1:172.21.15.154:6789/0 conn(0x7f46a415b1c0 legacy=0x7f46a415b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:00.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.524+0000 7f46a990e640 1 -- 172.21.15.105:0/2538736078 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46a415bfa0 con 0x7f46a4151e30 2024-04-15T15:20:00.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.525+0000 7f46a990e640 1 --2- 172.21.15.105:0/2538736078 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46a4151e30 0x7f46a406eb00 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f46a0009e50 tx=0x7f46a002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=238601ffd7c6d2e server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:00.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.525+0000 7f46a890c640 1 -- 172.21.15.105:0/2538736078 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a002ca70 con 0x7f46a4151e30 2024-04-15T15:20:00.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.525+0000 7f46a890c640 1 -- 172.21.15.105:0/2538736078 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46a002cc10 con 0x7f46a4151e30 2024-04-15T15:20:00.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.525+0000 7f46a890c640 1 -- 172.21.15.105:0/2538736078 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a00343f0 con 0x7f46a4151e30 2024-04-15T15:20:00.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.526+0000 7f46abb72640 1 -- 172.21.15.105:0/2538736078 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46a4151e30 msgr2=0x7f46a406eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:00.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.526+0000 7f46abb72640 1 --2- 172.21.15.105:0/2538736078 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46a4151e30 0x7f46a406eb00 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f46a0009e50 tx=0x7f46a002bed0 comp rx=0 tx=0).stop 2024-04-15T15:20:00.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.526+0000 7f46abb72640 1 -- 172.21.15.105:0/2538736078 shutdown_connections 2024-04-15T15:20:00.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.526+0000 7f46abb72640 1 --2- 172.21.15.105:0/2538736078 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46a40726f0 0x7f46a4072b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:00.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.526+0000 7f46abb72640 1 --2- 172.21.15.105:0/2538736078 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46a4151e30 0x7f46a406eb00 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:00.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.526+0000 7f46abb72640 1 -- 172.21.15.105:0/2538736078 >> 172.21.15.105:0/2538736078 conn(0x7f46a4067d50 msgr2=0x7f46a406b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:00.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.526+0000 7f46abb72640 1 -- 172.21.15.105:0/2538736078 shutdown_connections 2024-04-15T15:20:00.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.526+0000 7f46abb72640 1 -- 172.21.15.105:0/2538736078 wait complete. 2024-04-15T15:20:00.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.527+0000 7f46abb72640 1 Processor -- start 2024-04-15T15:20:00.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.527+0000 7f46abb72640 1 -- start start 2024-04-15T15:20:00.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.528+0000 7f46abb72640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46a40726f0 0x7f46a4159850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:00.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.528+0000 7f46abb72640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46a4151e30 0x7f46a414a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:00.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.528+0000 7f46abb72640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46a415b1c0 0x7f46a415a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:00.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.528+0000 7f46abb72640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f46a40b3ab0 con 0x7f46a415b1c0 2024-04-15T15:20:00.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.528+0000 7f46abb72640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f46a406e300 con 0x7f46a4151e30 2024-04-15T15:20:00.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.528+0000 7f46abb72640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f46a40b3c30 con 0x7f46a40726f0 2024-04-15T15:20:00.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.528+0000 7f46aa10f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46a415b1c0 0x7f46a415a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:00.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.528+0000 7f46aa10f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46a415b1c0 0x7f46a415a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37336/0 (socket says 172.21.15.105:37336) 2024-04-15T15:20:00.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.528+0000 7f46aa10f640 1 -- 172.21.15.105:0/3975561702 learned_addr learned my addr 172.21.15.105:0/3975561702 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:00.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.528+0000 7f46a990e640 1 --2- 172.21.15.105:0/3975561702 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46a40726f0 0x7f46a4159850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:00.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.528+0000 7f46a910d640 1 --2- 172.21.15.105:0/3975561702 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46a4151e30 0x7f46a414a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:00.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.528+0000 7f46aa10f640 1 -- 172.21.15.105:0/3975561702 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46a40726f0 msgr2=0x7f46a4159850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:00.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.528+0000 7f46aa10f640 1 --2- 172.21.15.105:0/3975561702 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46a40726f0 0x7f46a4159850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:00.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.528+0000 7f46aa10f640 1 -- 172.21.15.105:0/3975561702 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46a4151e30 msgr2=0x7f46a414a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:00.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.528+0000 7f46aa10f640 1 --2- 172.21.15.105:0/3975561702 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46a4151e30 0x7f46a414a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:00.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.528+0000 7f46aa10f640 1 -- 172.21.15.105:0/3975561702 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46a41f10e0 con 0x7f46a415b1c0 2024-04-15T15:20:00.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.528+0000 7f46aa10f640 1 --2- 172.21.15.105:0/3975561702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46a415b1c0 0x7f46a415a060 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f469c00b490 tx=0x7f469c00b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:00.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.530+0000 7f469a7fc640 1 -- 172.21.15.105:0/3975561702 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469c015020 con 0x7f46a415b1c0 2024-04-15T15:20:00.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.530+0000 7f46abb72640 1 -- 172.21.15.105:0/3975561702 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46a415ad90 con 0x7f46a415b1c0 2024-04-15T15:20:00.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.530+0000 7f469a7fc640 1 -- 172.21.15.105:0/3975561702 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f469c011850 con 0x7f46a415b1c0 2024-04-15T15:20:00.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.530+0000 7f469a7fc640 1 -- 172.21.15.105:0/3975561702 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469c011c40 con 0x7f46a415b1c0 2024-04-15T15:20:00.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.530+0000 7f46abb72640 1 -- 172.21.15.105:0/3975561702 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46a4157690 con 0x7f46a415b1c0 2024-04-15T15:20:00.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.531+0000 7f46abb72640 1 -- 172.21.15.105:0/3975561702 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46a4068ab0 con 0x7f46a415b1c0 2024-04-15T15:20:00.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.532+0000 7f469a7fc640 1 -- 172.21.15.105:0/3975561702 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f469c018020 con 0x7f46a415b1c0 2024-04-15T15:20:00.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.532+0000 7f469a7fc640 1 --2- 172.21.15.105:0/3975561702 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f468806c8f0 0x7f468806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:00.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.533+0000 7f46a990e640 1 --2- 172.21.15.105:0/3975561702 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f468806c8f0 0x7f468806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:00.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.533+0000 7f469a7fc640 1 -- 172.21.15.105:0/3975561702 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f469c08ba50 con 0x7f46a415b1c0 2024-04-15T15:20:00.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.533+0000 7f46a990e640 1 --2- 172.21.15.105:0/3975561702 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f468806c8f0 0x7f468806edb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f46a0035000 tx=0x7f46a0030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:00.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.537+0000 7f469a7fc640 1 -- 172.21.15.105:0/3975561702 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f469c01f020 con 0x7f46a415b1c0 2024-04-15T15:20:00.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.710+0000 7f46abb72640 1 -- 172.21.15.105:0/3975561702 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f46a40cfbd0 con 0x7f46a415b1c0 2024-04-15T15:20:00.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.711+0000 7f469a7fc640 1 -- 172.21.15.105:0/3975561702 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f469c0172c0 con 0x7f46a415b1c0 2024-04-15T15:20:00.713 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-04-15T15:20:00.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.713+0000 7f46abb72640 1 -- 172.21.15.105:0/3975561702 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f468806c8f0 msgr2=0x7f468806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:00.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.713+0000 7f46abb72640 1 --2- 172.21.15.105:0/3975561702 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f468806c8f0 0x7f468806edb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f46a0035000 tx=0x7f46a0030000 comp rx=0 tx=0).stop 2024-04-15T15:20:00.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.713+0000 7f46abb72640 1 -- 172.21.15.105:0/3975561702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46a415b1c0 msgr2=0x7f46a415a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:00.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.713+0000 7f46abb72640 1 --2- 172.21.15.105:0/3975561702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46a415b1c0 0x7f46a415a060 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f469c00b490 tx=0x7f469c00b950 comp rx=0 tx=0).stop 2024-04-15T15:20:00.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.713+0000 7f46abb72640 1 -- 172.21.15.105:0/3975561702 shutdown_connections 2024-04-15T15:20:00.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.713+0000 7f46abb72640 1 --2- 172.21.15.105:0/3975561702 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f468806c8f0 0x7f468806edb0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:00.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.713+0000 7f46abb72640 1 --2- 172.21.15.105:0/3975561702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46a415b1c0 0x7f46a415a060 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:00.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.713+0000 7f46abb72640 1 --2- 172.21.15.105:0/3975561702 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46a4151e30 0x7f46a414a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:00.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.713+0000 7f46abb72640 1 --2- 172.21.15.105:0/3975561702 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46a40726f0 0x7f46a4159850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:00.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.713+0000 7f46abb72640 1 -- 172.21.15.105:0/3975561702 >> 172.21.15.105:0/3975561702 conn(0x7f46a4067d50 msgr2=0x7f46a40b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:00.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.713+0000 7f46abb72640 1 -- 172.21.15.105:0/3975561702 shutdown_connections 2024-04-15T15:20:00.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.713+0000 7f46abb72640 1 -- 172.21.15.105:0/3975561702 wait complete. 2024-04-15T15:20:00.726 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-15T15:20:00.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.830+0000 7ff9d3d0d640 1 Processor -- start 2024-04-15T15:20:00.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.830+0000 7ff9d3d0d640 1 -- start start 2024-04-15T15:20:00.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.831+0000 7ff9d3d0d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff9cc0d07c0 0x7ff9cc06b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:00.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.831+0000 7ff9d3d0d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff9cc0cd5a0 0x7ff9cc0d0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:00.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.831+0000 7ff9d3d0d640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9cc0cf150 con 0x7ff9cc0ce520 2024-04-15T15:20:00.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.831+0000 7ff9d3d0d640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7ff9cc0d10e0 con 0x7ff9cc0d07c0 2024-04-15T15:20:00.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.831+0000 7ff9d3d0d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7ff9cc06c030 con 0x7ff9cc0cd5a0 2024-04-15T15:20:00.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.831+0000 7ff9d1aa9640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff9cc0d07c0 0x7ff9cc06b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:00.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.831+0000 7ff9d1aa9640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff9cc0d07c0 0x7ff9cc06b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37340/0 (socket says 172.21.15.105:37340) 2024-04-15T15:20:00.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.831+0000 7ff9d1aa9640 1 -- 172.21.15.105:0/2515435940 learned_addr learned my addr 172.21.15.105:0/2515435940 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:00.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.831+0000 7ff9d12a8640 1 --2- 172.21.15.105:0/2515435940 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff9cc0cd5a0 0x7ff9cc0d0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:00.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.831+0000 7ff9d1aa9640 1 -- 172.21.15.105:0/2515435940 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff9cc0cd5a0 msgr2=0x7ff9cc0d0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:00.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.831+0000 7ff9d1aa9640 1 --2- 172.21.15.105:0/2515435940 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff9cc0cd5a0 0x7ff9cc0d0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:00.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.831+0000 7ff9d1aa9640 1 -- 172.21.15.105:0/2515435940 >> v1:172.21.15.154:6789/0 conn(0x7ff9cc0ce520 legacy=0x7ff9cc0cba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:00.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.831+0000 7ff9d1aa9640 1 -- 172.21.15.105:0/2515435940 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9cc06c310 con 0x7ff9cc0d07c0 2024-04-15T15:20:00.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.832+0000 7ff9d1aa9640 1 --2- 172.21.15.105:0/2515435940 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9cc0d07c0 0x7ff9cc06b9c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0009870 tx=0x7ff9c002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=825b1703cb1dc2c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:00.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.832+0000 7ff9d0aa7640 1 -- 172.21.15.105:0/2515435940 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c0034070 con 0x7ff9cc0d07c0 2024-04-15T15:20:00.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.832+0000 7ff9d0aa7640 1 -- 172.21.15.105:0/2515435940 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9c0030840 con 0x7ff9cc0d07c0 2024-04-15T15:20:00.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.832+0000 7ff9d0aa7640 1 -- 172.21.15.105:0/2515435940 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c0030c10 con 0x7ff9cc0d07c0 2024-04-15T15:20:00.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.832+0000 7ff9d3d0d640 1 -- 172.21.15.105:0/2515435940 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9cc0d07c0 msgr2=0x7ff9cc06b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:00.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.832+0000 7ff9d3d0d640 1 --2- 172.21.15.105:0/2515435940 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9cc0d07c0 0x7ff9cc06b9c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0009870 tx=0x7ff9c002b980 comp rx=0 tx=0).stop 2024-04-15T15:20:00.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.833+0000 7ff9d3d0d640 1 -- 172.21.15.105:0/2515435940 shutdown_connections 2024-04-15T15:20:00.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.833+0000 7ff9d3d0d640 1 --2- 172.21.15.105:0/2515435940 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff9cc0cd5a0 0x7ff9cc0d0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:00.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.833+0000 7ff9d3d0d640 1 --2- 172.21.15.105:0/2515435940 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9cc0d07c0 0x7ff9cc06b9c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:00.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.833+0000 7ff9d3d0d640 1 -- 172.21.15.105:0/2515435940 >> 172.21.15.105:0/2515435940 conn(0x7ff9cc0b2fc0 msgr2=0x7ff9cc0b33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:00.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.833+0000 7ff9d3d0d640 1 -- 172.21.15.105:0/2515435940 shutdown_connections 2024-04-15T15:20:00.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.833+0000 7ff9d3d0d640 1 -- 172.21.15.105:0/2515435940 wait complete. 2024-04-15T15:20:00.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.834+0000 7ff9d3d0d640 1 Processor -- start 2024-04-15T15:20:00.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.834+0000 7ff9d3d0d640 1 -- start start 2024-04-15T15:20:00.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.834+0000 7ff9d3d0d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9cc0cd5a0 0x7ff9cc09c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:00.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.834+0000 7ff9d3d0d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff9cc0ce520 0x7ff9cc0b6e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:00.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.834+0000 7ff9d3d0d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9cc0d07c0 0x7ff9cc155500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:00.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.834+0000 7ff9d3d0d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff9cc0ce000 con 0x7ff9cc0d07c0 2024-04-15T15:20:00.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.834+0000 7ff9d3d0d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff9cc0cdd20 con 0x7ff9cc0cd5a0 2024-04-15T15:20:00.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.834+0000 7ff9d3d0d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7ff9cc0cc190 con 0x7ff9cc0ce520 2024-04-15T15:20:00.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.835+0000 7ff9d22aa640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9cc0d07c0 0x7ff9cc155500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:00.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.835+0000 7ff9d22aa640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9cc0d07c0 0x7ff9cc155500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37342/0 (socket says 172.21.15.105:37342) 2024-04-15T15:20:00.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.835+0000 7ff9d22aa640 1 -- 172.21.15.105:0/3420958042 learned_addr learned my addr 172.21.15.105:0/3420958042 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:00.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.835+0000 7ff9d1aa9640 1 --2- 172.21.15.105:0/3420958042 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9cc0cd5a0 0x7ff9cc09c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:00.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.835+0000 7ff9d12a8640 1 --2- 172.21.15.105:0/3420958042 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff9cc0ce520 0x7ff9cc0b6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:00.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.835+0000 7ff9d22aa640 1 -- 172.21.15.105:0/3420958042 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff9cc0ce520 msgr2=0x7ff9cc0b6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:00.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.835+0000 7ff9d22aa640 1 --2- 172.21.15.105:0/3420958042 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff9cc0ce520 0x7ff9cc0b6e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:00.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.835+0000 7ff9d22aa640 1 -- 172.21.15.105:0/3420958042 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9cc0cd5a0 msgr2=0x7ff9cc09c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:00.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.835+0000 7ff9d22aa640 1 --2- 172.21.15.105:0/3420958042 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9cc0cd5a0 0x7ff9cc09c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:00.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.835+0000 7ff9d22aa640 1 -- 172.21.15.105:0/3420958042 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9cc0b48f0 con 0x7ff9cc0d07c0 2024-04-15T15:20:00.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.835+0000 7ff9d22aa640 1 --2- 172.21.15.105:0/3420958042 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9cc0d07c0 0x7ff9cc155500 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff9c400c7a0 tx=0x7ff9c400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:00.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.835+0000 7ff9be7fc640 1 -- 172.21.15.105:0/3420958042 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c4003b70 con 0x7ff9cc0d07c0 2024-04-15T15:20:00.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.835+0000 7ff9be7fc640 1 -- 172.21.15.105:0/3420958042 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9c4003d30 con 0x7ff9cc0d07c0 2024-04-15T15:20:00.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.835+0000 7ff9d3d0d640 1 -- 172.21.15.105:0/3420958042 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9cc0b4bd0 con 0x7ff9cc0d07c0 2024-04-15T15:20:00.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.835+0000 7ff9be7fc640 1 -- 172.21.15.105:0/3420958042 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c400aa90 con 0x7ff9cc0d07c0 2024-04-15T15:20:00.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.835+0000 7ff9d3d0d640 1 -- 172.21.15.105:0/3420958042 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9cc0b5190 con 0x7ff9cc0d07c0 2024-04-15T15:20:00.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.836+0000 7ff9d3d0d640 1 -- 172.21.15.105:0/3420958042 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9cc0b5390 con 0x7ff9cc0d07c0 2024-04-15T15:20:00.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.837+0000 7ff9be7fc640 1 -- 172.21.15.105:0/3420958042 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff9c400ac30 con 0x7ff9cc0d07c0 2024-04-15T15:20:00.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.837+0000 7ff9be7fc640 1 --2- 172.21.15.105:0/3420958042 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff9b006c8f0 0x7ff9b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:00.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.837+0000 7ff9be7fc640 1 -- 172.21.15.105:0/3420958042 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff9c408b8d0 con 0x7ff9cc0d07c0 2024-04-15T15:20:00.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.837+0000 7ff9d1aa9640 1 --2- 172.21.15.105:0/3420958042 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff9b006c8f0 0x7ff9b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:00.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.838+0000 7ff9d1aa9640 1 --2- 172.21.15.105:0/3420958042 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff9b006c8f0 0x7ff9b006edb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0036000 tx=0x7ff9c003b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:00.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:00.841+0000 7ff9be7fc640 1 -- 172.21.15.105:0/3420958042 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff9c4059740 con 0x7ff9cc0d07c0 2024-04-15T15:20:01.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.016+0000 7ff9d3d0d640 1 -- 172.21.15.105:0/3420958042 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff9cc06fad0 con 0x7ff9cc0d07c0 2024-04-15T15:20:01.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.017+0000 7ff9be7fc640 1 -- 172.21.15.105:0/3420958042 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7ff9c405da40 con 0x7ff9cc0d07c0 2024-04-15T15:20:01.019 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-04-15T15:20:01.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.019+0000 7ff9d3d0d640 1 -- 172.21.15.105:0/3420958042 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff9b006c8f0 msgr2=0x7ff9b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:01.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.019+0000 7ff9d3d0d640 1 --2- 172.21.15.105:0/3420958042 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff9b006c8f0 0x7ff9b006edb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0036000 tx=0x7ff9c003b040 comp rx=0 tx=0).stop 2024-04-15T15:20:01.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.019+0000 7ff9d3d0d640 1 -- 172.21.15.105:0/3420958042 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9cc0d07c0 msgr2=0x7ff9cc155500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:01.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.019+0000 7ff9d3d0d640 1 --2- 172.21.15.105:0/3420958042 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9cc0d07c0 0x7ff9cc155500 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff9c400c7a0 tx=0x7ff9c400cc60 comp rx=0 tx=0).stop 2024-04-15T15:20:01.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.019+0000 7ff9d3d0d640 1 -- 172.21.15.105:0/3420958042 shutdown_connections 2024-04-15T15:20:01.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.019+0000 7ff9d3d0d640 1 --2- 172.21.15.105:0/3420958042 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff9cc0ce520 0x7ff9cc0b6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:01.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.019+0000 7ff9d3d0d640 1 --2- 172.21.15.105:0/3420958042 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff9b006c8f0 0x7ff9b006edb0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:01.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.019+0000 7ff9d3d0d640 1 --2- 172.21.15.105:0/3420958042 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9cc0d07c0 0x7ff9cc155500 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:01.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.019+0000 7ff9d3d0d640 1 --2- 172.21.15.105:0/3420958042 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9cc0cd5a0 0x7ff9cc09c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:01.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.019+0000 7ff9d3d0d640 1 -- 172.21.15.105:0/3420958042 >> 172.21.15.105:0/3420958042 conn(0x7ff9cc0b2fc0 msgr2=0x7ff9cc0ae610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:01.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.019+0000 7ff9d3d0d640 1 -- 172.21.15.105:0/3420958042 shutdown_connections 2024-04-15T15:20:01.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.019+0000 7ff9d3d0d640 1 -- 172.21.15.105:0/3420958042 wait complete. 2024-04-15T15:20:01.032 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-15T15:20:01.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.136+0000 7fee7c8b3640 1 Processor -- start 2024-04-15T15:20:01.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.136+0000 7fee7c8b3640 1 -- start start 2024-04-15T15:20:01.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.137+0000 7fee7c8b3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fee7815e9f0 0x7fee7806e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:01.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.137+0000 7fee7c8b3640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fee780b59e0 0x7fee780b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:01.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.137+0000 7fee7c8b3640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee780aee60 con 0x7fee780b4890 2024-04-15T15:20:01.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.137+0000 7fee7c8b3640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fee780af140 con 0x7fee780b59e0 2024-04-15T15:20:01.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.137+0000 7fee7c8b3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fee780af420 con 0x7fee7815e9f0 2024-04-15T15:20:01.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.137+0000 7fee76d9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fee780b59e0 0x7fee780b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:01.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.137+0000 7fee76d9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fee780b59e0 0x7fee780b4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37344/0 (socket says 172.21.15.105:37344) 2024-04-15T15:20:01.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.137+0000 7fee76d9d640 1 -- 172.21.15.105:0/1455227750 learned_addr learned my addr 172.21.15.105:0/1455227750 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:01.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.137+0000 7fee7659c640 1 --2- 172.21.15.105:0/1455227750 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fee7815e9f0 0x7fee7806e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:01.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.137+0000 7fee76d9d640 1 -- 172.21.15.105:0/1455227750 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fee7815e9f0 msgr2=0x7fee7806e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:01.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.137+0000 7fee76d9d640 1 --2- 172.21.15.105:0/1455227750 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fee7815e9f0 0x7fee7806e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:01.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.137+0000 7fee76d9d640 1 -- 172.21.15.105:0/1455227750 >> v1:172.21.15.154:6789/0 conn(0x7fee780b4890 legacy=0x7fee780b52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:01.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.137+0000 7fee76d9d640 1 -- 172.21.15.105:0/1455227750 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee780af700 con 0x7fee780b59e0 2024-04-15T15:20:01.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.138+0000 7fee76d9d640 1 --2- 172.21.15.105:0/1455227750 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee780b59e0 0x7fee780b4cf0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fee70009e50 tx=0x7fee7002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eff92b0fdf1be15 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:01.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.138+0000 7fee75d9b640 1 -- 172.21.15.105:0/1455227750 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7002c9e0 con 0x7fee780b59e0 2024-04-15T15:20:01.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.138+0000 7fee75d9b640 1 -- 172.21.15.105:0/1455227750 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee7002cb80 con 0x7fee780b59e0 2024-04-15T15:20:01.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.138+0000 7fee75d9b640 1 -- 172.21.15.105:0/1455227750 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee700343f0 con 0x7fee780b59e0 2024-04-15T15:20:01.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.139+0000 7fee7c8b3640 1 -- 172.21.15.105:0/1455227750 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee780b59e0 msgr2=0x7fee780b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:01.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.139+0000 7fee7c8b3640 1 --2- 172.21.15.105:0/1455227750 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee780b59e0 0x7fee780b4cf0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fee70009e50 tx=0x7fee7002be40 comp rx=0 tx=0).stop 2024-04-15T15:20:01.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.139+0000 7fee7c8b3640 1 -- 172.21.15.105:0/1455227750 shutdown_connections 2024-04-15T15:20:01.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.139+0000 7fee7c8b3640 1 --2- 172.21.15.105:0/1455227750 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee780b59e0 0x7fee780b4cf0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:01.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.139+0000 7fee7c8b3640 1 --2- 172.21.15.105:0/1455227750 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fee7815e9f0 0x7fee7806e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:01.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.139+0000 7fee7c8b3640 1 -- 172.21.15.105:0/1455227750 >> 172.21.15.105:0/1455227750 conn(0x7fee78073570 msgr2=0x7fee7806b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:01.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.139+0000 7fee7c8b3640 1 -- 172.21.15.105:0/1455227750 shutdown_connections 2024-04-15T15:20:01.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.139+0000 7fee7c8b3640 1 -- 172.21.15.105:0/1455227750 wait complete. 2024-04-15T15:20:01.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.140+0000 7fee7c8b3640 1 Processor -- start 2024-04-15T15:20:01.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.140+0000 7fee7c8b3640 1 -- start start 2024-04-15T15:20:01.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.141+0000 7fee7c8b3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee780b4890 0x7fee78152dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:01.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.141+0000 7fee7c8b3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fee780b59e0 0x7fee78154cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:01.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.141+0000 7fee7c8b3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee7815e9f0 0x7fee780b1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:01.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.141+0000 7fee7c8b3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fee7806ec80 con 0x7fee7815e9f0 2024-04-15T15:20:01.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.141+0000 7fee7c8b3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fee7806eb00 con 0x7fee780b4890 2024-04-15T15:20:01.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.141+0000 7fee7c8b3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fee780aff00 con 0x7fee780b59e0 2024-04-15T15:20:01.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.141+0000 7fee76d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee7815e9f0 0x7fee780b1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:01.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.141+0000 7fee76d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee7815e9f0 0x7fee780b1df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37350/0 (socket says 172.21.15.105:37350) 2024-04-15T15:20:01.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.141+0000 7fee76d9d640 1 -- 172.21.15.105:0/2898780900 learned_addr learned my addr 172.21.15.105:0/2898780900 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:01.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.141+0000 7fee6dd9b640 1 --2- 172.21.15.105:0/2898780900 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fee780b59e0 0x7fee78154cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:01.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.141+0000 7fee76d9d640 1 -- 172.21.15.105:0/2898780900 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fee780b59e0 msgr2=0x7fee78154cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:01.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.141+0000 7fee76d9d640 1 --2- 172.21.15.105:0/2898780900 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fee780b59e0 0x7fee78154cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:01.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.141+0000 7fee7659c640 1 --2- 172.21.15.105:0/2898780900 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee780b4890 0x7fee78152dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:01.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.141+0000 7fee76d9d640 1 -- 172.21.15.105:0/2898780900 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee780b4890 msgr2=0x7fee78152dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:01.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.141+0000 7fee76d9d640 1 --2- 172.21.15.105:0/2898780900 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee780b4890 0x7fee78152dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:01.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.141+0000 7fee76d9d640 1 -- 172.21.15.105:0/2898780900 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee78141ed0 con 0x7fee7815e9f0 2024-04-15T15:20:01.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.141+0000 7fee6dd9b640 1 --2- 172.21.15.105:0/2898780900 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fee780b59e0 0x7fee78154cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:01.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.142+0000 7fee76d9d640 1 --2- 172.21.15.105:0/2898780900 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee7815e9f0 0x7fee780b1df0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fee70009e50 tx=0x7fee70004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:01.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.142+0000 7fee6f7fe640 1 -- 172.21.15.105:0/2898780900 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee70043020 con 0x7fee7815e9f0 2024-04-15T15:20:01.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.142+0000 7fee6f7fe640 1 -- 172.21.15.105:0/2898780900 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee7003e450 con 0x7fee7815e9f0 2024-04-15T15:20:01.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.142+0000 7fee6f7fe640 1 -- 172.21.15.105:0/2898780900 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee70009510 con 0x7fee7815e9f0 2024-04-15T15:20:01.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.142+0000 7fee7c8b3640 1 -- 172.21.15.105:0/2898780900 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee78155210 con 0x7fee7815e9f0 2024-04-15T15:20:01.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.142+0000 7fee7c8b3640 1 -- 172.21.15.105:0/2898780900 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee78142470 con 0x7fee7815e9f0 2024-04-15T15:20:01.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.143+0000 7fee7c8b3640 1 -- 172.21.15.105:0/2898780900 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee78068ab0 con 0x7fee7815e9f0 2024-04-15T15:20:01.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.143+0000 7fee6f7fe640 1 -- 172.21.15.105:0/2898780900 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fee70046020 con 0x7fee7815e9f0 2024-04-15T15:20:01.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.144+0000 7fee6f7fe640 1 --2- 172.21.15.105:0/2898780900 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fee5806c8f0 0x7fee5806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:01.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.144+0000 7fee6f7fe640 1 -- 172.21.15.105:0/2898780900 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fee700ae520 con 0x7fee7815e9f0 2024-04-15T15:20:01.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.144+0000 7fee7659c640 1 --2- 172.21.15.105:0/2898780900 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fee5806c8f0 0x7fee5806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:01.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.144+0000 7fee7659c640 1 --2- 172.21.15.105:0/2898780900 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fee5806c8f0 0x7fee5806edb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fee6800b7c0 tx=0x7fee68008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:01.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.148+0000 7fee6f7fe640 1 -- 172.21.15.105:0/2898780900 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fee7004d030 con 0x7fee7815e9f0 2024-04-15T15:20:01.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.322+0000 7fee7c8b3640 1 -- 172.21.15.105:0/2898780900 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fee7806d1b0 con 0x7fee7815e9f0 2024-04-15T15:20:01.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.322+0000 7fee6f7fe640 1 -- 172.21.15.105:0/2898780900 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fee7007c8c0 con 0x7fee7815e9f0 2024-04-15T15:20:01.324 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-04-15T15:20:01.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.324+0000 7fee7c8b3640 1 -- 172.21.15.105:0/2898780900 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fee5806c8f0 msgr2=0x7fee5806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:01.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.324+0000 7fee7c8b3640 1 --2- 172.21.15.105:0/2898780900 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fee5806c8f0 0x7fee5806edb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fee6800b7c0 tx=0x7fee68008040 comp rx=0 tx=0).stop 2024-04-15T15:20:01.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.324+0000 7fee7c8b3640 1 -- 172.21.15.105:0/2898780900 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee7815e9f0 msgr2=0x7fee780b1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:01.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.324+0000 7fee7c8b3640 1 --2- 172.21.15.105:0/2898780900 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee7815e9f0 0x7fee780b1df0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fee70009e50 tx=0x7fee70004a40 comp rx=0 tx=0).stop 2024-04-15T15:20:01.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.324+0000 7fee7c8b3640 1 -- 172.21.15.105:0/2898780900 shutdown_connections 2024-04-15T15:20:01.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.324+0000 7fee7c8b3640 1 --2- 172.21.15.105:0/2898780900 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fee780b59e0 0x7fee78154cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:01.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.324+0000 7fee7c8b3640 1 --2- 172.21.15.105:0/2898780900 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fee5806c8f0 0x7fee5806edb0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:01.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.324+0000 7fee7c8b3640 1 --2- 172.21.15.105:0/2898780900 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee7815e9f0 0x7fee780b1df0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:01.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.324+0000 7fee7c8b3640 1 --2- 172.21.15.105:0/2898780900 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee780b4890 0x7fee78152dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:01.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.324+0000 7fee7c8b3640 1 -- 172.21.15.105:0/2898780900 >> 172.21.15.105:0/2898780900 conn(0x7fee78073570 msgr2=0x7fee780b0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:01.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.325+0000 7fee7c8b3640 1 -- 172.21.15.105:0/2898780900 shutdown_connections 2024-04-15T15:20:01.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.325+0000 7fee7c8b3640 1 -- 172.21.15.105:0/2898780900 wait complete. 2024-04-15T15:20:01.337 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-15T15:20:01.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.440+0000 7f4a2a20d640 1 Processor -- start 2024-04-15T15:20:01.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.440+0000 7f4a2a20d640 1 -- start start 2024-04-15T15:20:01.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.440+0000 7f4a2a20d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4a240d1100 0x7f4a240b25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:01.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.440+0000 7f4a2a20d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4a240b2ae0 0x7f4a240cd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:01.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.440+0000 7f4a2a20d640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a240ccf40 con 0x7f4a2415c7b0 2024-04-15T15:20:01.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.440+0000 7f4a2a20d640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f4a240b32d0 con 0x7f4a240d1100 2024-04-15T15:20:01.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.440+0000 7f4a2a20d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f4a240b35b0 con 0x7f4a240b2ae0 2024-04-15T15:20:01.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.441+0000 7f4a237fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f4a2415c7b0 0x7f4a240d09f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:55078/0 (socket says 172.21.15.105:55078) 2024-04-15T15:20:01.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.441+0000 7f4a237fe640 1 -- 172.21.15.105:0/901071087 learned_addr learned my addr 172.21.15.105:0/901071087 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:20:01.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.441+0000 7f4a227fc640 1 -- 172.21.15.105:0/901071087 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2159419945 0 0) 0x7f4a240ccf40 con 0x7f4a2415c7b0 2024-04-15T15:20:01.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.441+0000 7f4a23fff640 1 --2- 172.21.15.105:0/901071087 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4a240b2ae0 0x7f4a240cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:01.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.441+0000 7f4a227fc640 1 -- 172.21.15.105:0/901071087 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4a10003520 con 0x7f4a2415c7b0 2024-04-15T15:20:01.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.441+0000 7f4a22ffd640 1 --2- 172.21.15.105:0/901071087 >> v2:172.21.15.154:3300/0 conn(0x7f4a240d1100 0x7f4a240b25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:01.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.441+0000 7f4a227fc640 1 -- 172.21.15.105:0/901071087 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 516263943 0 0) 0x7f4a10003520 con 0x7f4a2415c7b0 2024-04-15T15:20:01.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.441+0000 7f4a227fc640 1 -- 172.21.15.105:0/901071087 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4a240b2ae0 msgr2=0x7f4a240cd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:01.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.441+0000 7f4a227fc640 1 --2- 172.21.15.105:0/901071087 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4a240b2ae0 0x7f4a240cd420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:01.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.441+0000 7f4a227fc640 1 -- 172.21.15.105:0/901071087 >> v2:172.21.15.154:3300/0 conn(0x7f4a240d1100 msgr2=0x7f4a240b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:01.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.441+0000 7f4a227fc640 1 --2- 172.21.15.105:0/901071087 >> v2:172.21.15.154:3300/0 conn(0x7f4a240d1100 0x7f4a240b25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:01.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.441+0000 7f4a227fc640 1 -- 172.21.15.105:0/901071087 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a240b3890 con 0x7f4a2415c7b0 2024-04-15T15:20:01.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.441+0000 7f4a227fc640 1 -- 172.21.15.105:0/901071087 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7f4a140019f0 con 0x7f4a2415c7b0 2024-04-15T15:20:01.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.441+0000 7f4a227fc640 1 -- 172.21.15.105:0/901071087 >> v1:172.21.15.105:6789/0 conn(0x7f4a2415c7b0 legacy=0x7f4a240d09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:01.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.441+0000 7f4a227fc640 1 --2- 172.21.15.105:0/901071087 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4a240b2ae0 0x7f4a100039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:01.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.441+0000 7f4a227fc640 1 --2- 172.21.15.105:0/901071087 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4a10003fe0 0x7f4a10004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:01.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.441+0000 7f4a227fc640 1 --2- 172.21.15.105:0/901071087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a10004d40 0x7f4a100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:01.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.441+0000 7f4a227fc640 1 -- 172.21.15.105:0/901071087 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4a240b32d0 con 0x7f4a10004d40 2024-04-15T15:20:01.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.441+0000 7f4a227fc640 1 -- 172.21.15.105:0/901071087 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4a240b35b0 con 0x7f4a10003fe0 2024-04-15T15:20:01.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.441+0000 7f4a227fc640 1 -- 172.21.15.105:0/901071087 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f4a1000bac0 con 0x7f4a240b2ae0 2024-04-15T15:20:01.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.441+0000 7f4a23fff640 1 --2- 172.21.15.105:0/901071087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a10004d40 0x7f4a100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:01.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.441+0000 7f4a237fe640 1 --2- 172.21.15.105:0/901071087 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4a240b2ae0 0x7f4a100039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:01.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.441+0000 7f4a22ffd640 1 --2- 172.21.15.105:0/901071087 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4a10003fe0 0x7f4a10004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:01.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.441+0000 7f4a23fff640 1 -- 172.21.15.105:0/901071087 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4a240b2ae0 msgr2=0x7f4a100039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:01.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.441+0000 7f4a23fff640 1 --2- 172.21.15.105:0/901071087 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4a240b2ae0 0x7f4a100039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:01.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.441+0000 7f4a23fff640 1 -- 172.21.15.105:0/901071087 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4a10003fe0 msgr2=0x7f4a10004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:01.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.441+0000 7f4a23fff640 1 --2- 172.21.15.105:0/901071087 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4a10003fe0 0x7f4a10004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:01.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.441+0000 7f4a23fff640 1 -- 172.21.15.105:0/901071087 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a10003520 con 0x7f4a10004d40 2024-04-15T15:20:01.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.442+0000 7f4a23fff640 1 --2- 172.21.15.105:0/901071087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a10004d40 0x7f4a100051c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c00ba30 tx=0x7f4a10005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecc6dcddb484f589 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:01.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.442+0000 7f4a227fc640 1 -- 172.21.15.105:0/901071087 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c036020 con 0x7f4a10004d40 2024-04-15T15:20:01.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.442+0000 7f4a227fc640 1 -- 172.21.15.105:0/901071087 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a1c032640 con 0x7f4a10004d40 2024-04-15T15:20:01.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.442+0000 7f4a227fc640 1 -- 172.21.15.105:0/901071087 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c00b600 con 0x7f4a10004d40 2024-04-15T15:20:01.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.443+0000 7f4a2a20d640 1 -- 172.21.15.105:0/901071087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a10004d40 msgr2=0x7f4a100051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:01.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.443+0000 7f4a2a20d640 1 --2- 172.21.15.105:0/901071087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a10004d40 0x7f4a100051c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c00ba30 tx=0x7f4a10005830 comp rx=0 tx=0).stop 2024-04-15T15:20:01.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.443+0000 7f4a23fff640 1 -- 172.21.15.105:0/901071087 reap_dead start 2024-04-15T15:20:01.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.443+0000 7f4a2a20d640 1 -- 172.21.15.105:0/901071087 shutdown_connections 2024-04-15T15:20:01.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.443+0000 7f4a2a20d640 1 -- 172.21.15.105:0/901071087 >> 172.21.15.105:0/901071087 conn(0x7f4a240d25a0 msgr2=0x7f4a240d29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:01.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.443+0000 7f4a2a20d640 1 -- 172.21.15.105:0/901071087 shutdown_connections 2024-04-15T15:20:01.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.443+0000 7f4a2a20d640 1 -- 172.21.15.105:0/901071087 wait complete. 2024-04-15T15:20:01.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.444+0000 7f4a2a20d640 1 Processor -- start 2024-04-15T15:20:01.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.444+0000 7f4a2a20d640 1 -- start start 2024-04-15T15:20:01.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.445+0000 7f4a2a20d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4a10004d40 0x7f4a240c7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:01.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.445+0000 7f4a2a20d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4a240c8190 0x7f4a24148800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:01.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.445+0000 7f4a2a20d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a24148d40 0x7f4a241491f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:01.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.445+0000 7f4a2a20d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4a2415cb90 con 0x7f4a24148d40 2024-04-15T15:20:01.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.445+0000 7f4a2a20d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4a240b2f90 con 0x7f4a240c8190 2024-04-15T15:20:01.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.445+0000 7f4a2a20d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f4a2415cd10 con 0x7f4a10004d40 2024-04-15T15:20:01.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.445+0000 7f4a23fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a24148d40 0x7f4a241491f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:01.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.445+0000 7f4a23fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a24148d40 0x7f4a241491f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37368/0 (socket says 172.21.15.105:37368) 2024-04-15T15:20:01.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.445+0000 7f4a23fff640 1 -- 172.21.15.105:0/2604502358 learned_addr learned my addr 172.21.15.105:0/2604502358 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:01.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.445+0000 7f4a237fe640 1 --2- 172.21.15.105:0/2604502358 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4a10004d40 0x7f4a240c7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:01.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.445+0000 7f4a22ffd640 1 --2- 172.21.15.105:0/2604502358 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4a240c8190 0x7f4a24148800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:01.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.445+0000 7f4a23fff640 1 -- 172.21.15.105:0/2604502358 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4a10004d40 msgr2=0x7f4a240c7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:01.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.445+0000 7f4a23fff640 1 --2- 172.21.15.105:0/2604502358 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4a10004d40 0x7f4a240c7c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:01.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.445+0000 7f4a23fff640 1 -- 172.21.15.105:0/2604502358 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4a240c8190 msgr2=0x7f4a24148800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:01.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.445+0000 7f4a23fff640 1 --2- 172.21.15.105:0/2604502358 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4a240c8190 0x7f4a24148800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:01.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.445+0000 7f4a23fff640 1 -- 172.21.15.105:0/2604502358 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a2415dfe0 con 0x7f4a24148d40 2024-04-15T15:20:01.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.445+0000 7f4a22ffd640 1 --2- 172.21.15.105:0/2604502358 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4a240c8190 0x7f4a24148800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:01.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.446+0000 7f4a23fff640 1 --2- 172.21.15.105:0/2604502358 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a24148d40 0x7f4a241491f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c02bf90 tx=0x7f4a1c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:01.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.446+0000 7f4a1bfff640 1 -- 172.21.15.105:0/2604502358 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c042070 con 0x7f4a24148d40 2024-04-15T15:20:01.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.446+0000 7f4a1bfff640 1 -- 172.21.15.105:0/2604502358 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a1c004020 con 0x7f4a24148d40 2024-04-15T15:20:01.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.446+0000 7f4a2a20d640 1 -- 172.21.15.105:0/2604502358 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a240b8310 con 0x7f4a24148d40 2024-04-15T15:20:01.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.446+0000 7f4a1bfff640 1 -- 172.21.15.105:0/2604502358 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c036040 con 0x7f4a24148d40 2024-04-15T15:20:01.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.446+0000 7f4a2a20d640 1 -- 172.21.15.105:0/2604502358 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a240c8a80 con 0x7f4a24148d40 2024-04-15T15:20:01.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.447+0000 7f4a2a20d640 1 -- 172.21.15.105:0/2604502358 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a24069150 con 0x7f4a24148d40 2024-04-15T15:20:01.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.447+0000 7f4a1bfff640 1 -- 172.21.15.105:0/2604502358 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4a1c039020 con 0x7f4a24148d40 2024-04-15T15:20:01.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.448+0000 7f4a1bfff640 1 --2- 172.21.15.105:0/2604502358 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f4a0406c8f0 0x7f4a0406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:01.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.448+0000 7f4a1bfff640 1 -- 172.21.15.105:0/2604502358 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4a1c0ad980 con 0x7f4a24148d40 2024-04-15T15:20:01.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.448+0000 7f4a237fe640 1 --2- 172.21.15.105:0/2604502358 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f4a0406c8f0 0x7f4a0406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:01.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.448+0000 7f4a237fe640 1 --2- 172.21.15.105:0/2604502358 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f4a0406c8f0 0x7f4a0406edb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4a140013d0 tx=0x7f4a14009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:01.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.452+0000 7f4a1bfff640 1 -- 172.21.15.105:0/2604502358 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4a1c07bdb0 con 0x7f4a24148d40 2024-04-15T15:20:01.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.641+0000 7f4a2a20d640 1 -- 172.21.15.105:0/2604502358 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4a2415e1b0 con 0x7f4a24148d40 2024-04-15T15:20:01.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.641+0000 7f4a1bfff640 1 -- 172.21.15.105:0/2604502358 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7f4a1c07b6b0 con 0x7f4a24148d40 2024-04-15T15:20:01.643 INFO:teuthology.orchestra.run.smithi105.stderr:listed 0 entries 2024-04-15T15:20:01.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.643+0000 7f4a2a20d640 1 -- 172.21.15.105:0/2604502358 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f4a0406c8f0 msgr2=0x7f4a0406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:01.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.643+0000 7f4a2a20d640 1 --2- 172.21.15.105:0/2604502358 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f4a0406c8f0 0x7f4a0406edb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4a140013d0 tx=0x7f4a14009040 comp rx=0 tx=0).stop 2024-04-15T15:20:01.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.643+0000 7f4a2a20d640 1 -- 172.21.15.105:0/2604502358 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a24148d40 msgr2=0x7f4a241491f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:01.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.643+0000 7f4a2a20d640 1 --2- 172.21.15.105:0/2604502358 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a24148d40 0x7f4a241491f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c02bf90 tx=0x7f4a1c004730 comp rx=0 tx=0).stop 2024-04-15T15:20:01.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.643+0000 7f4a2a20d640 1 -- 172.21.15.105:0/2604502358 shutdown_connections 2024-04-15T15:20:01.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.643+0000 7f4a2a20d640 1 --2- 172.21.15.105:0/2604502358 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f4a0406c8f0 0x7f4a0406edb0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:01.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.643+0000 7f4a2a20d640 1 --2- 172.21.15.105:0/2604502358 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a24148d40 0x7f4a241491f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:01.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.643+0000 7f4a2a20d640 1 --2- 172.21.15.105:0/2604502358 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4a240c8190 0x7f4a24148800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:01.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.643+0000 7f4a2a20d640 1 --2- 172.21.15.105:0/2604502358 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4a10004d40 0x7f4a240c7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:01.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.643+0000 7f4a2a20d640 1 -- 172.21.15.105:0/2604502358 >> 172.21.15.105:0/2604502358 conn(0x7f4a240d25a0 msgr2=0x7f4a240cf2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:01.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.643+0000 7f4a2a20d640 1 -- 172.21.15.105:0/2604502358 shutdown_connections 2024-04-15T15:20:01.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.643+0000 7f4a2a20d640 1 -- 172.21.15.105:0/2604502358 wait complete. 2024-04-15T15:20:01.655 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi154.front.sepia.ceph.com... 2024-04-15T15:20:01.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:01.656 DEBUG:teuthology.orchestra.run.smithi154:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-15T15:20:01.691 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:20:01.691 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi154.front.sepia.ceph.com 2024-04-15T15:20:01.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:01.691 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-15T15:20:01.768 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-04-15T15:20:01.768 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:20:01.784 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-04-15T15:20:01.784 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:20:01.841 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:20:01.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.982+0000 7fc1df235640 1 Processor -- start 2024-04-15T15:20:01.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.982+0000 7fc1df235640 1 -- start start 2024-04-15T15:20:01.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.983+0000 7fc1df235640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc1d80d0f10 0x7fc1d8169c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:01.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.983+0000 7fc1df235640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1d806d2a0 con 0x7fc1d815ecf0 2024-04-15T15:20:01.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.983+0000 7fc1df235640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1d806d580 con 0x7fc1d806e060 2024-04-15T15:20:01.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.983+0000 7fc1df235640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fc1d806d860 con 0x7fc1d80d0f10 2024-04-15T15:20:01.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.983+0000 7fc1dcfd1640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fc1d815ecf0 0x7fc1d80ccc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:55082/0 (socket says 172.21.15.105:55082) 2024-04-15T15:20:01.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.983+0000 7fc1dcfd1640 1 -- 172.21.15.105:0/2984065888 learned_addr learned my addr 172.21.15.105:0/2984065888 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:20:01.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.984+0000 7fc1d77fe640 1 -- 172.21.15.105:0/2984065888 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3375034460 0 0) 0x7fc1d806d2a0 con 0x7fc1d815ecf0 2024-04-15T15:20:01.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.984+0000 7fc1dd7d2640 1 --2- 172.21.15.105:0/2984065888 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc1d80d0f10 0x7fc1d8169c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:01.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.984+0000 7fc1d77fe640 1 -- 172.21.15.105:0/2984065888 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc1c0003520 con 0x7fc1d815ecf0 2024-04-15T15:20:01.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.984+0000 7fc1d77fe640 1 -- 172.21.15.105:0/2984065888 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3403227338 0 0) 0x7fc1c0003520 con 0x7fc1d815ecf0 2024-04-15T15:20:01.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.984+0000 7fc1d77fe640 1 -- 172.21.15.105:0/2984065888 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc1d80d0f10 msgr2=0x7fc1d8169c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:01.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.984+0000 7fc1d77fe640 1 --2- 172.21.15.105:0/2984065888 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc1d80d0f10 0x7fc1d8169c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:01.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.984+0000 7fc1d77fe640 1 -- 172.21.15.105:0/2984065888 >> v1:172.21.15.154:6789/0 conn(0x7fc1d806e060 legacy=0x7fc1d80cd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:01.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.984+0000 7fc1d77fe640 1 -- 172.21.15.105:0/2984065888 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1d806db40 con 0x7fc1d815ecf0 2024-04-15T15:20:01.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.984+0000 7fc1d77fe640 1 -- 172.21.15.105:0/2984065888 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7fc1c80019f0 con 0x7fc1d815ecf0 2024-04-15T15:20:01.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.984+0000 7fc1d77fe640 1 -- 172.21.15.105:0/2984065888 >> v1:172.21.15.105:6789/0 conn(0x7fc1d815ecf0 legacy=0x7fc1d80ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:01.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.984+0000 7fc1d77fe640 1 --2- 172.21.15.105:0/2984065888 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1c00039e0 0x7fc1c0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:01.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.984+0000 7fc1d77fe640 1 --2- 172.21.15.105:0/2984065888 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1c0004800 0x7fc1c0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:01.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.984+0000 7fc1d77fe640 1 --2- 172.21.15.105:0/2984065888 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc1d80d0f10 0x7fc1c00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:01.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.984+0000 7fc1d77fe640 1 -- 172.21.15.105:0/2984065888 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d806d860 con 0x7fc1c0004800 2024-04-15T15:20:01.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.984+0000 7fc1d77fe640 1 -- 172.21.15.105:0/2984065888 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c000baa0 con 0x7fc1c00039e0 2024-04-15T15:20:01.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.984+0000 7fc1d77fe640 1 -- 172.21.15.105:0/2984065888 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fc1c000bca0 con 0x7fc1d80d0f10 2024-04-15T15:20:01.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.984+0000 7fc1d7fff640 1 --2- 172.21.15.105:0/2984065888 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1c0004800 0x7fc1c0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:01.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.984+0000 7fc1dcfd1640 1 --2- 172.21.15.105:0/2984065888 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1c00039e0 0x7fc1c0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:01.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.984+0000 7fc1dd7d2640 1 --2- 172.21.15.105:0/2984065888 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc1d80d0f10 0x7fc1c00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:01.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.984+0000 7fc1d7fff640 1 -- 172.21.15.105:0/2984065888 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc1d80d0f10 msgr2=0x7fc1c00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:01.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.984+0000 7fc1d7fff640 1 --2- 172.21.15.105:0/2984065888 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc1d80d0f10 0x7fc1c00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:01.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.984+0000 7fc1d7fff640 1 -- 172.21.15.105:0/2984065888 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1c00039e0 msgr2=0x7fc1c0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:01.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.984+0000 7fc1d7fff640 1 --2- 172.21.15.105:0/2984065888 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1c00039e0 0x7fc1c0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:01.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.984+0000 7fc1d7fff640 1 -- 172.21.15.105:0/2984065888 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1c0003520 con 0x7fc1c0004800 2024-04-15T15:20:01.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.985+0000 7fc1d7fff640 1 --2- 172.21.15.105:0/2984065888 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1c0004800 0x7fc1c0004c80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc001720 tx=0x7fc1c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c35a037387e2c23 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:01.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.985+0000 7fc1d77fe640 1 -- 172.21.15.105:0/2984065888 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc001370 con 0x7fc1c0004800 2024-04-15T15:20:01.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.985+0000 7fc1d77fe640 1 -- 172.21.15.105:0/2984065888 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1cc0052d0 con 0x7fc1c0004800 2024-04-15T15:20:01.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.985+0000 7fc1d77fe640 1 -- 172.21.15.105:0/2984065888 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc003830 con 0x7fc1c0004800 2024-04-15T15:20:01.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.986+0000 7fc1df235640 1 -- 172.21.15.105:0/2984065888 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1c0004800 msgr2=0x7fc1c0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:01.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.986+0000 7fc1df235640 1 --2- 172.21.15.105:0/2984065888 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1c0004800 0x7fc1c0004c80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc001720 tx=0x7fc1c0005830 comp rx=0 tx=0).stop 2024-04-15T15:20:01.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.986+0000 7fc1dd7d2640 1 -- 172.21.15.105:0/2984065888 reap_dead start 2024-04-15T15:20:01.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.986+0000 7fc1df235640 1 -- 172.21.15.105:0/2984065888 shutdown_connections 2024-04-15T15:20:01.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.986+0000 7fc1df235640 1 -- 172.21.15.105:0/2984065888 >> 172.21.15.105:0/2984065888 conn(0x7fc1d80d2600 msgr2=0x7fc1d80d2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:01.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.986+0000 7fc1df235640 1 -- 172.21.15.105:0/2984065888 shutdown_connections 2024-04-15T15:20:01.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.986+0000 7fc1df235640 1 -- 172.21.15.105:0/2984065888 wait complete. 2024-04-15T15:20:01.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.987+0000 7fc1df235640 1 Processor -- start 2024-04-15T15:20:01.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.987+0000 7fc1df235640 1 -- start start 2024-04-15T15:20:01.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.988+0000 7fc1df235640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1c0004800 0x7fc1d81f85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:01.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.988+0000 7fc1df235640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc1d81f5790 0x7fc1d80cbb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:01.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.988+0000 7fc1df235640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1d81f1590 0x7fc1d81f1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:01.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.988+0000 7fc1df235640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d815f0d0 con 0x7fc1d81f1590 2024-04-15T15:20:01.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.988+0000 7fc1df235640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d806cf90 con 0x7fc1c0004800 2024-04-15T15:20:01.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.988+0000 7fc1df235640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fc1d815f250 con 0x7fc1d81f5790 2024-04-15T15:20:01.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.988+0000 7fc1dd7d2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1d81f1590 0x7fc1d81f1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:01.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.988+0000 7fc1dd7d2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1d81f1590 0x7fc1d81f1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37388/0 (socket says 172.21.15.105:37388) 2024-04-15T15:20:01.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.988+0000 7fc1dd7d2640 1 -- 172.21.15.105:0/2667684285 learned_addr learned my addr 172.21.15.105:0/2667684285 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:01.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.988+0000 7fc1d7fff640 1 --2- 172.21.15.105:0/2667684285 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc1d81f5790 0x7fc1d80cbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:01.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.988+0000 7fc1dcfd1640 1 --2- 172.21.15.105:0/2667684285 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1c0004800 0x7fc1d81f85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:01.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.988+0000 7fc1dd7d2640 1 -- 172.21.15.105:0/2667684285 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc1d81f5790 msgr2=0x7fc1d80cbb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:01.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.988+0000 7fc1dd7d2640 1 --2- 172.21.15.105:0/2667684285 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc1d81f5790 0x7fc1d80cbb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:01.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.988+0000 7fc1dd7d2640 1 -- 172.21.15.105:0/2667684285 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1c0004800 msgr2=0x7fc1d81f85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:01.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.988+0000 7fc1dd7d2640 1 --2- 172.21.15.105:0/2667684285 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1c0004800 0x7fc1d81f85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:01.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.988+0000 7fc1dd7d2640 1 -- 172.21.15.105:0/2667684285 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1d81f8b10 con 0x7fc1d81f1590 2024-04-15T15:20:01.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.988+0000 7fc1dd7d2640 1 --2- 172.21.15.105:0/2667684285 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1d81f1590 0x7fc1d81f1a40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc1d806f730 tx=0x7fc1d0011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:01.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.988+0000 7fc1d57fa640 1 -- 172.21.15.105:0/2667684285 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d0019020 con 0x7fc1d81f1590 2024-04-15T15:20:01.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.989+0000 7fc1d57fa640 1 -- 172.21.15.105:0/2667684285 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1d0004240 con 0x7fc1d81f1590 2024-04-15T15:20:01.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.989+0000 7fc1d57fa640 1 -- 172.21.15.105:0/2667684285 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d000f770 con 0x7fc1d81f1590 2024-04-15T15:20:01.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.989+0000 7fc1df235640 1 -- 172.21.15.105:0/2667684285 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1d80c63e0 con 0x7fc1d81f1590 2024-04-15T15:20:01.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.989+0000 7fc1df235640 1 -- 172.21.15.105:0/2667684285 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1d80c69a0 con 0x7fc1d81f1590 2024-04-15T15:20:01.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.990+0000 7fc1df235640 1 -- 172.21.15.105:0/2667684285 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1d8068a40 con 0x7fc1d81f1590 2024-04-15T15:20:01.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.990+0000 7fc1d57fa640 1 -- 172.21.15.105:0/2667684285 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc1d001c020 con 0x7fc1d81f1590 2024-04-15T15:20:01.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.991+0000 7fc1d57fa640 1 --2- 172.21.15.105:0/2667684285 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fc1c406c8f0 0x7fc1c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:01.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.991+0000 7fc1d57fa640 1 -- 172.21.15.105:0/2667684285 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc1d008be20 con 0x7fc1d81f1590 2024-04-15T15:20:01.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.991+0000 7fc1dcfd1640 1 --2- 172.21.15.105:0/2667684285 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fc1c406c8f0 0x7fc1c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:01.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.991+0000 7fc1dcfd1640 1 --2- 172.21.15.105:0/2667684285 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fc1c406c8f0 0x7fc1c406edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc1c8002230 tx=0x7fc1c8006040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:01.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:01.995+0000 7fc1d57fa640 1 -- 172.21.15.105:0/2667684285 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc1d0023020 con 0x7fc1d81f1590 2024-04-15T15:20:02.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.170+0000 7fc1df235640 1 -- 172.21.15.105:0/2667684285 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc1d80d1ad0 con 0x7fc1d81f1590 2024-04-15T15:20:02.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.171+0000 7fc1d57fa640 1 -- 172.21.15.105:0/2667684285 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16653 (secure 0 0 0) 0x7fc1d81f8b10 con 0x7fc1d81f1590 2024-04-15T15:20:02.173 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:20:02.173 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":20,"fsid":"09e4d2a4-a324-4793-af77-644b25e520e1","created":"2024-04-15T15:19:01.524543+0000","modified":"2024-04-15T15:19:27.414890+0000","last_up_change":"2024-04-15T15:19:12.825815+0000","last_in_change":"2024-04-15T15:19:03.788126+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:19:13.748585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:19:23.789287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:19:24.722250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6f6292b-2d7e-4aa0-9d1c-6ddd0f97287c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6801","nonce":2873040054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6803","nonce":2873040054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6807","nonce":2873040054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6805","nonce":2873040054}]},"public_addr":"172.21.15.105:6801/2873040054","cluster_addr":"172.21.15.105:6803/2873040054","heartbeat_back_addr":"172.21.15.105:6807/2873040054","heartbeat_front_addr":"172.21.15.105:6805/2873040054","state":["exists","up"]},{"osd":1,"uuid":"8900966a-b393-41e3-8f93-e5a43bd23622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6818","nonce":2494994479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6821","nonce":2494994479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6828","nonce":2494994479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6823","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6825","nonce":2494994479}]},"public_addr":"172.21.15.105:6818/2494994479","cluster_addr":"172.21.15.105:6821/2494994479","heartbeat_back_addr":"172.21.15.105:6828/2494994479","heartbeat_front_addr":"172.21.15.105:6825/2494994479","state":["exists","up"]},{"osd":2,"uuid":"81a5dc7a-9325-4ff5-941d-a2d5fba21f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6809","nonce":3798408699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6811","nonce":3798408699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6815","nonce":3798408699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6813","nonce":3798408699}]},"public_addr":"172.21.15.105:6809/3798408699","cluster_addr":"172.21.15.105:6811/3798408699","heartbeat_back_addr":"172.21.15.105:6815/3798408699","heartbeat_front_addr":"172.21.15.105:6813/3798408699","state":["exists","up"]},{"osd":3,"uuid":"df766ed2-5c71-4805-96f4-3d4970edc960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6819","nonce":1920720186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6824","nonce":1920720186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6831","nonce":1920720186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6827","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6829","nonce":1920720186}]},"public_addr":"172.21.15.105:6819/1920720186","cluster_addr":"172.21.15.105:6824/1920720186","heartbeat_back_addr":"172.21.15.105:6831/1920720186","heartbeat_front_addr":"172.21.15.105:6829/1920720186","state":["exists","up"]},{"osd":4,"uuid":"e1039e03-28b8-49df-b951-c91930b93c9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6810","nonce":3516494456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6813","nonce":3516494456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6820","nonce":3516494456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6817","nonce":3516494456}]},"public_addr":"172.21.15.154:6810/3516494456","cluster_addr":"172.21.15.154:6813/3516494456","heartbeat_back_addr":"172.21.15.154:6820/3516494456","heartbeat_front_addr":"172.21.15.154:6817/3516494456","state":["exists","up"]},{"osd":5,"uuid":"a9f5eaf1-66ff-405a-a9e4-ec3340812a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6801","nonce":540367751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6803","nonce":540367751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6807","nonce":540367751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6805","nonce":540367751}]},"public_addr":"172.21.15.154:6801/540367751","cluster_addr":"172.21.15.154:6803/540367751","heartbeat_back_addr":"172.21.15.154:6807/540367751","heartbeat_front_addr":"172.21.15.154:6805/540367751","state":["exists","up"]},{"osd":6,"uuid":"267d02ea-675f-428f-aae8-e0375e4d4feb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6825","nonce":1550829005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6827","nonce":1550829005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6831","nonce":1550829005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6829","nonce":1550829005}]},"public_addr":"172.21.15.154:6825/1550829005","cluster_addr":"172.21.15.154:6827/1550829005","heartbeat_back_addr":"172.21.15.154:6831/1550829005","heartbeat_front_addr":"172.21.15.154:6829/1550829005","state":["exists","up"]},{"osd":7,"uuid":"759ac6cc-896a-423c-885f-9fe90d8dcb14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6809","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6811","nonce":3920089903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6816","nonce":3920089903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6823","nonce":3920089903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6819","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6821","nonce":3920089903}]},"public_addr":"172.21.15.154:6811/3920089903","cluster_addr":"172.21.15.154:6816/3920089903","heartbeat_back_addr":"172.21.15.154:6823/3920089903","heartbeat_front_addr":"172.21.15.154:6821/3920089903","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.949509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.984072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.907278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.947391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.975447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.998896+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:09.028739+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.894119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:20:02.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.173+0000 7fc1df235640 1 -- 172.21.15.105:0/2667684285 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fc1c406c8f0 msgr2=0x7fc1c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:02.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.173+0000 7fc1df235640 1 --2- 172.21.15.105:0/2667684285 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fc1c406c8f0 0x7fc1c406edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc1c8002230 tx=0x7fc1c8006040 comp rx=0 tx=0).stop 2024-04-15T15:20:02.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.173+0000 7fc1df235640 1 -- 172.21.15.105:0/2667684285 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1d81f1590 msgr2=0x7fc1d81f1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:02.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.173+0000 7fc1df235640 1 --2- 172.21.15.105:0/2667684285 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1d81f1590 0x7fc1d81f1a40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc1d806f730 tx=0x7fc1d0011cf0 comp rx=0 tx=0).stop 2024-04-15T15:20:02.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.173+0000 7fc1df235640 1 -- 172.21.15.105:0/2667684285 shutdown_connections 2024-04-15T15:20:02.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.173+0000 7fc1df235640 1 --2- 172.21.15.105:0/2667684285 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc1d81f5790 0x7fc1d80cbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:02.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.173+0000 7fc1df235640 1 --2- 172.21.15.105:0/2667684285 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fc1c406c8f0 0x7fc1c406edb0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:02.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.173+0000 7fc1df235640 1 --2- 172.21.15.105:0/2667684285 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1d81f1590 0x7fc1d81f1a40 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:02.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.173+0000 7fc1df235640 1 --2- 172.21.15.105:0/2667684285 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1c0004800 0x7fc1d81f85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:02.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.173+0000 7fc1df235640 1 -- 172.21.15.105:0/2667684285 >> 172.21.15.105:0/2667684285 conn(0x7fc1d80d2600 msgr2=0x7fc1d8070050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:02.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.173+0000 7fc1df235640 1 -- 172.21.15.105:0/2667684285 shutdown_connections 2024-04-15T15:20:02.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.173+0000 7fc1df235640 1 -- 172.21.15.105:0/2667684285 wait complete. 2024-04-15T15:20:02.186 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:19:13.748585+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-15T15:19:23.789287+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-15T15:19:24.722250+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:20:02.187 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:20:02.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.290+0000 7f9bfded9640 1 Processor -- start 2024-04-15T15:20:02.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.290+0000 7f9bfded9640 1 -- start start 2024-04-15T15:20:02.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.291+0000 7f9bfded9640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9bf8144c90 0x7f9bf80b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:02.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.291+0000 7f9bfded9640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9bf80cd8e0 0x7f9bf806f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:02.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.291+0000 7f9bfded9640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bf80b40a0 con 0x7f9bf806ef90 2024-04-15T15:20:02.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.291+0000 7f9bfded9640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f9bf80b3e50 con 0x7f9bf80cd8e0 2024-04-15T15:20:02.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.291+0000 7f9bfded9640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f9bf806f910 con 0x7f9bf8144c90 2024-04-15T15:20:02.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.291+0000 7f9bf6ffd640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f9bf806ef90 0x7f9bf80cd1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:55094/0 (socket says 172.21.15.105:55094) 2024-04-15T15:20:02.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.291+0000 7f9bf6ffd640 1 -- 172.21.15.105:0/1432105679 learned_addr learned my addr 172.21.15.105:0/1432105679 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:20:02.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.291+0000 7f9bf67fc640 1 -- 172.21.15.105:0/1432105679 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3599445100 0 0) 0x7f9bf80b40a0 con 0x7f9bf806ef90 2024-04-15T15:20:02.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.291+0000 7f9bf67fc640 1 -- 172.21.15.105:0/1432105679 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9be0003520 con 0x7f9bf806ef90 2024-04-15T15:20:02.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.291+0000 7f9bf7fff640 1 --2- 172.21.15.105:0/1432105679 >> v2:172.21.15.154:3300/0 conn(0x7f9bf80cd8e0 0x7f9bf806f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:02.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.291+0000 7f9bf77fe640 1 --2- 172.21.15.105:0/1432105679 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9bf8144c90 0x7f9bf80b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:02.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.292+0000 7f9bf67fc640 1 -- 172.21.15.105:0/1432105679 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2718823937 0 0) 0x7f9be0003520 con 0x7f9bf806ef90 2024-04-15T15:20:02.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.292+0000 7f9bf67fc640 1 -- 172.21.15.105:0/1432105679 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9bf8144c90 msgr2=0x7f9bf80b4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:02.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.292+0000 7f9bf67fc640 1 --2- 172.21.15.105:0/1432105679 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9bf8144c90 0x7f9bf80b4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:02.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.292+0000 7f9bf67fc640 1 -- 172.21.15.105:0/1432105679 >> v2:172.21.15.154:3300/0 conn(0x7f9bf80cd8e0 msgr2=0x7f9bf806f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:02.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.292+0000 7f9bf67fc640 1 --2- 172.21.15.105:0/1432105679 >> v2:172.21.15.154:3300/0 conn(0x7f9bf80cd8e0 0x7f9bf806f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:02.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.292+0000 7f9bf67fc640 1 -- 172.21.15.105:0/1432105679 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bf80b4380 con 0x7f9bf806ef90 2024-04-15T15:20:02.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.292+0000 7f9bf67fc640 1 -- 172.21.15.105:0/1432105679 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7f9bec001670 con 0x7f9bf806ef90 2024-04-15T15:20:02.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.292+0000 7f9bf67fc640 1 -- 172.21.15.105:0/1432105679 >> v1:172.21.15.105:6789/0 conn(0x7f9bf806ef90 legacy=0x7f9bf80cd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:02.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.292+0000 7f9bf67fc640 1 --2- 172.21.15.105:0/1432105679 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9be00039b0 0x7f9be0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:02.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.292+0000 7f9bf67fc640 1 --2- 172.21.15.105:0/1432105679 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9be0004800 0x7f9be0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:02.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.292+0000 7f9bf67fc640 1 --2- 172.21.15.105:0/1432105679 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9bf8144c90 0x7f9be00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:02.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.292+0000 7f9bf67fc640 1 -- 172.21.15.105:0/1432105679 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf80b3e50 con 0x7f9be00039b0 2024-04-15T15:20:02.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.292+0000 7f9bf67fc640 1 -- 172.21.15.105:0/1432105679 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf806f910 con 0x7f9be0004800 2024-04-15T15:20:02.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.292+0000 7f9bf67fc640 1 -- 172.21.15.105:0/1432105679 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f9be000b9c0 con 0x7f9bf8144c90 2024-04-15T15:20:02.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.292+0000 7f9bf77fe640 1 --2- 172.21.15.105:0/1432105679 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9be00039b0 0x7f9be0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:02.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.292+0000 7f9bf6ffd640 1 --2- 172.21.15.105:0/1432105679 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9be0004800 0x7f9be0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:02.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.292+0000 7f9bf7fff640 1 --2- 172.21.15.105:0/1432105679 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9bf8144c90 0x7f9be00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:02.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.292+0000 7f9bf77fe640 1 -- 172.21.15.105:0/1432105679 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9bf8144c90 msgr2=0x7f9be00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:02.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.292+0000 7f9bf77fe640 1 --2- 172.21.15.105:0/1432105679 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9bf8144c90 0x7f9be00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:02.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.292+0000 7f9bf77fe640 1 -- 172.21.15.105:0/1432105679 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9be0004800 msgr2=0x7f9be0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:02.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.292+0000 7f9bf77fe640 1 --2- 172.21.15.105:0/1432105679 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9be0004800 0x7f9be0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:02.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.292+0000 7f9bf77fe640 1 -- 172.21.15.105:0/1432105679 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9be0003520 con 0x7f9be00039b0 2024-04-15T15:20:02.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.293+0000 7f9bf77fe640 1 --2- 172.21.15.105:0/1432105679 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9be00039b0 0x7f9be0003dd0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9be8009af0 tx=0x7f9be0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67b6ea5d14cf0883 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:02.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.293+0000 7f9bf67fc640 1 -- 172.21.15.105:0/1432105679 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be8039070 con 0x7f9be00039b0 2024-04-15T15:20:02.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.293+0000 7f9bf67fc640 1 -- 172.21.15.105:0/1432105679 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9be802c6b0 con 0x7f9be00039b0 2024-04-15T15:20:02.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.293+0000 7f9bf67fc640 1 -- 172.21.15.105:0/1432105679 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be8009690 con 0x7f9be00039b0 2024-04-15T15:20:02.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.294+0000 7f9bfded9640 1 -- 172.21.15.105:0/1432105679 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9be00039b0 msgr2=0x7f9be0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:02.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.294+0000 7f9bfded9640 1 --2- 172.21.15.105:0/1432105679 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9be00039b0 0x7f9be0003dd0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9be8009af0 tx=0x7f9be0005830 comp rx=0 tx=0).stop 2024-04-15T15:20:02.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.294+0000 7f9bf7fff640 1 -- 172.21.15.105:0/1432105679 reap_dead start 2024-04-15T15:20:02.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.294+0000 7f9bfded9640 1 -- 172.21.15.105:0/1432105679 shutdown_connections 2024-04-15T15:20:02.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.294+0000 7f9bfded9640 1 -- 172.21.15.105:0/1432105679 >> 172.21.15.105:0/1432105679 conn(0x7f9bf8067d50 msgr2=0x7f9bf806b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:02.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.294+0000 7f9bfded9640 1 -- 172.21.15.105:0/1432105679 shutdown_connections 2024-04-15T15:20:02.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.294+0000 7f9bfded9640 1 -- 172.21.15.105:0/1432105679 wait complete. 2024-04-15T15:20:02.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.295+0000 7f9bfded9640 1 Processor -- start 2024-04-15T15:20:02.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.295+0000 7f9bfded9640 1 -- start start 2024-04-15T15:20:02.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.296+0000 7f9bfded9640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9be00039b0 0x7f9bf8155020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:02.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.296+0000 7f9bfded9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9bf815dd00 0x7f9bf815e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:02.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.296+0000 7f9bfded9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9bf815d060 0x7f9bf815d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:02.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.296+0000 7f9bfded9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf8145070 con 0x7f9bf815dd00 2024-04-15T15:20:02.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.296+0000 7f9bfded9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf80b4940 con 0x7f9bf815d060 2024-04-15T15:20:02.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.296+0000 7f9bfded9640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f9bf81451f0 con 0x7f9be00039b0 2024-04-15T15:20:02.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.296+0000 7f9bf6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9bf815dd00 0x7f9bf815e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:02.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.296+0000 7f9bf6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9bf815dd00 0x7f9bf815e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37412/0 (socket says 172.21.15.105:37412) 2024-04-15T15:20:02.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.296+0000 7f9bf6ffd640 1 -- 172.21.15.105:0/1426040815 learned_addr learned my addr 172.21.15.105:0/1426040815 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:02.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.296+0000 7f9bf77fe640 1 --2- 172.21.15.105:0/1426040815 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9be00039b0 0x7f9bf8155020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:02.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.296+0000 7f9bf7fff640 1 --2- 172.21.15.105:0/1426040815 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9bf815d060 0x7f9bf815d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:02.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.296+0000 7f9bf6ffd640 1 -- 172.21.15.105:0/1426040815 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9be00039b0 msgr2=0x7f9bf8155020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:02.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.296+0000 7f9bf6ffd640 1 --2- 172.21.15.105:0/1426040815 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9be00039b0 0x7f9bf8155020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:02.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.296+0000 7f9bf6ffd640 1 -- 172.21.15.105:0/1426040815 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9bf815d060 msgr2=0x7f9bf815d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:02.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.296+0000 7f9bf6ffd640 1 --2- 172.21.15.105:0/1426040815 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9bf815d060 0x7f9bf815d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:02.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.296+0000 7f9bf6ffd640 1 -- 172.21.15.105:0/1426040815 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bf815e7c0 con 0x7f9bf815dd00 2024-04-15T15:20:02.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.296+0000 7f9bf77fe640 1 --2- 172.21.15.105:0/1426040815 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9be00039b0 0x7f9bf8155020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:20:02.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.296+0000 7f9bf7fff640 1 --2- 172.21.15.105:0/1426040815 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9bf815d060 0x7f9bf815d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:20:02.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.296+0000 7f9bf6ffd640 1 --2- 172.21.15.105:0/1426040815 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9bf815dd00 0x7f9bf815e180 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9bec00e840 tx=0x7f9bec00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:02.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.296+0000 7f9bdffff640 1 -- 172.21.15.105:0/1426040815 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bec00c8d0 con 0x7f9bf815dd00 2024-04-15T15:20:02.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.296+0000 7f9bdffff640 1 -- 172.21.15.105:0/1426040815 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bec018070 con 0x7f9bf815dd00 2024-04-15T15:20:02.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.296+0000 7f9bfded9640 1 -- 172.21.15.105:0/1426040815 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bf81f8230 con 0x7f9bf815dd00 2024-04-15T15:20:02.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.297+0000 7f9bdffff640 1 -- 172.21.15.105:0/1426040815 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bec00ccd0 con 0x7f9bf815dd00 2024-04-15T15:20:02.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.297+0000 7f9bfded9640 1 -- 172.21.15.105:0/1426040815 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bf815b2f0 con 0x7f9bf815dd00 2024-04-15T15:20:02.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.298+0000 7f9bfded9640 1 -- 172.21.15.105:0/1426040815 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bf8068ab0 con 0x7f9bf815dd00 2024-04-15T15:20:02.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.298+0000 7f9bdffff640 1 -- 172.21.15.105:0/1426040815 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9bec013420 con 0x7f9bf815dd00 2024-04-15T15:20:02.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.299+0000 7f9bdffff640 1 --2- 172.21.15.105:0/1426040815 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f9be406c8c0 0x7f9be406ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:02.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.299+0000 7f9bf77fe640 1 --2- 172.21.15.105:0/1426040815 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f9be406c8c0 0x7f9be406ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:02.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.299+0000 7f9bdffff640 1 -- 172.21.15.105:0/1426040815 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9bec08b690 con 0x7f9bf815dd00 2024-04-15T15:20:02.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.299+0000 7f9bf77fe640 1 --2- 172.21.15.105:0/1426040815 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f9be406c8c0 0x7f9be406ed80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9be8036000 tx=0x7f9be8030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:02.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.303+0000 7f9bdffff640 1 -- 172.21.15.105:0/1426040815 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9bec059b50 con 0x7f9bf815dd00 2024-04-15T15:20:02.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.477+0000 7f9bfded9640 1 -- 172.21.15.105:0/1426040815 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9bf806eaa0 con 0x7f9bf815dd00 2024-04-15T15:20:02.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.477+0000 7f9bdffff640 1 -- 172.21.15.105:0/1426040815 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f9bec001670 con 0x7f9bf815dd00 2024-04-15T15:20:02.479 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-04-15T15:20:02.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.479+0000 7f9bfded9640 1 -- 172.21.15.105:0/1426040815 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f9be406c8c0 msgr2=0x7f9be406ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:02.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.479+0000 7f9bfded9640 1 --2- 172.21.15.105:0/1426040815 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f9be406c8c0 0x7f9be406ed80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9be8036000 tx=0x7f9be8030000 comp rx=0 tx=0).stop 2024-04-15T15:20:02.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.479+0000 7f9bfded9640 1 -- 172.21.15.105:0/1426040815 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9bf815dd00 msgr2=0x7f9bf815e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:02.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.479+0000 7f9bfded9640 1 --2- 172.21.15.105:0/1426040815 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9bf815dd00 0x7f9bf815e180 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9bec00e840 tx=0x7f9bec00ed00 comp rx=0 tx=0).stop 2024-04-15T15:20:02.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.479+0000 7f9bfded9640 1 -- 172.21.15.105:0/1426040815 shutdown_connections 2024-04-15T15:20:02.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.479+0000 7f9bfded9640 1 --2- 172.21.15.105:0/1426040815 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f9be406c8c0 0x7f9be406ed80 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:02.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.479+0000 7f9bfded9640 1 --2- 172.21.15.105:0/1426040815 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9bf815d060 0x7f9bf815d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:02.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.479+0000 7f9bfded9640 1 --2- 172.21.15.105:0/1426040815 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9bf815dd00 0x7f9bf815e180 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:02.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.479+0000 7f9bfded9640 1 --2- 172.21.15.105:0/1426040815 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9be00039b0 0x7f9bf8155020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:02.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.479+0000 7f9bfded9640 1 -- 172.21.15.105:0/1426040815 >> 172.21.15.105:0/1426040815 conn(0x7f9bf8067d50 msgr2=0x7f9bf806b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:02.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.479+0000 7f9bfded9640 1 -- 172.21.15.105:0/1426040815 shutdown_connections 2024-04-15T15:20:02.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.479+0000 7f9bfded9640 1 -- 172.21.15.105:0/1426040815 wait complete. 2024-04-15T15:20:02.492 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-15T15:20:02.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.598+0000 7fa35411c640 1 Processor -- start 2024-04-15T15:20:02.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.598+0000 7fa35411c640 1 -- start start 2024-04-15T15:20:02.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.599+0000 7fa35411c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa34c144e80 0x7fa34c06dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:02.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.599+0000 7fa35411c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa34c0b4910 0x7fa34c06fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:02.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.599+0000 7fa35411c640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa34c06c790 con 0x7fa34c06f860 2024-04-15T15:20:02.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.599+0000 7fa35411c640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fa34c0b3470 con 0x7fa34c144e80 2024-04-15T15:20:02.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.599+0000 7fa35411c640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fa34c0cf7b0 con 0x7fa34c0b4910 2024-04-15T15:20:02.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.599+0000 7fa351eb8640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa34c144e80 0x7fa34c06dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:02.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.599+0000 7fa351eb8640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa34c144e80 0x7fa34c06dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37416/0 (socket says 172.21.15.105:37416) 2024-04-15T15:20:02.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.599+0000 7fa351eb8640 1 -- 172.21.15.105:0/2084112839 learned_addr learned my addr 172.21.15.105:0/2084112839 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:02.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.599+0000 7fa3526b9640 1 --2- 172.21.15.105:0/2084112839 >> v2:172.21.15.154:3300/0 conn(0x7fa34c0b4910 0x7fa34c06fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:02.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.599+0000 7fa351eb8640 1 -- 172.21.15.105:0/2084112839 >> v2:172.21.15.154:3300/0 conn(0x7fa34c0b4910 msgr2=0x7fa34c06fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:02.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.599+0000 7fa351eb8640 1 --2- 172.21.15.105:0/2084112839 >> v2:172.21.15.154:3300/0 conn(0x7fa34c0b4910 0x7fa34c06fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:02.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.599+0000 7fa351eb8640 1 -- 172.21.15.105:0/2084112839 >> v1:172.21.15.154:6789/0 conn(0x7fa34c06f860 legacy=0x7fa34c15f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:02.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.599+0000 7fa351eb8640 1 -- 172.21.15.105:0/2084112839 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa34c0cfa90 con 0x7fa34c144e80 2024-04-15T15:20:02.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.599+0000 7fa3526b9640 1 --2- 172.21.15.105:0/2084112839 >> v2:172.21.15.154:3300/0 conn(0x7fa34c0b4910 0x7fa34c06fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:02.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.600+0000 7fa351eb8640 1 --2- 172.21.15.105:0/2084112839 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa34c144e80 0x7fa34c06dec0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa34c0ceeb0 tx=0x7fa34802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fea3357b1fa58d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:02.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.600+0000 7fa350eb6640 1 -- 172.21.15.105:0/2084112839 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34802e070 con 0x7fa34c144e80 2024-04-15T15:20:02.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.600+0000 7fa350eb6640 1 -- 172.21.15.105:0/2084112839 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa348037070 con 0x7fa34c144e80 2024-04-15T15:20:02.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.600+0000 7fa350eb6640 1 -- 172.21.15.105:0/2084112839 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa348031a00 con 0x7fa34c144e80 2024-04-15T15:20:02.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.601+0000 7fa35411c640 1 -- 172.21.15.105:0/2084112839 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa34c144e80 msgr2=0x7fa34c06dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:02.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.601+0000 7fa35411c640 1 --2- 172.21.15.105:0/2084112839 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa34c144e80 0x7fa34c06dec0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa34c0ceeb0 tx=0x7fa34802b880 comp rx=0 tx=0).stop 2024-04-15T15:20:02.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.601+0000 7fa35411c640 1 -- 172.21.15.105:0/2084112839 shutdown_connections 2024-04-15T15:20:02.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.601+0000 7fa35411c640 1 --2- 172.21.15.105:0/2084112839 >> v2:172.21.15.154:3300/0 conn(0x7fa34c0b4910 0x7fa34c06fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:02.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.601+0000 7fa35411c640 1 --2- 172.21.15.105:0/2084112839 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa34c144e80 0x7fa34c06dec0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:02.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.601+0000 7fa35411c640 1 -- 172.21.15.105:0/2084112839 >> 172.21.15.105:0/2084112839 conn(0x7fa34c06b9c0 msgr2=0x7fa34c06bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:02.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.601+0000 7fa35411c640 1 -- 172.21.15.105:0/2084112839 shutdown_connections 2024-04-15T15:20:02.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.601+0000 7fa35411c640 1 -- 172.21.15.105:0/2084112839 wait complete. 2024-04-15T15:20:02.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.602+0000 7fa35411c640 1 Processor -- start 2024-04-15T15:20:02.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.602+0000 7fa35411c640 1 -- start start 2024-04-15T15:20:02.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.603+0000 7fa35411c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa34c06f860 0x7fa34c06ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:02.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.603+0000 7fa35411c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa34c0b4910 0x7fa34c0ad810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:02.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.603+0000 7fa35411c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa34c144e80 0x7fa34c0ab0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:02.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.603+0000 7fa35411c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa34c06e400 con 0x7fa34c144e80 2024-04-15T15:20:02.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.603+0000 7fa35411c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa34c0d0050 con 0x7fa34c06f860 2024-04-15T15:20:02.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.603+0000 7fa35411c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fa34c145260 con 0x7fa34c0b4910 2024-04-15T15:20:02.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.603+0000 7fa3526b9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa34c144e80 0x7fa34c0ab0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:02.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.603+0000 7fa3526b9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa34c144e80 0x7fa34c0ab0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37422/0 (socket says 172.21.15.105:37422) 2024-04-15T15:20:02.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.603+0000 7fa3526b9640 1 -- 172.21.15.105:0/3645899922 learned_addr learned my addr 172.21.15.105:0/3645899922 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:02.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.603+0000 7fa3516b7640 1 --2- 172.21.15.105:0/3645899922 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa34c0b4910 0x7fa34c0ad810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:02.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.603+0000 7fa351eb8640 1 --2- 172.21.15.105:0/3645899922 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa34c06f860 0x7fa34c06ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:02.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.603+0000 7fa3526b9640 1 -- 172.21.15.105:0/3645899922 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa34c0b4910 msgr2=0x7fa34c0ad810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:02.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.603+0000 7fa3526b9640 1 --2- 172.21.15.105:0/3645899922 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa34c0b4910 0x7fa34c0ad810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:02.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.603+0000 7fa3526b9640 1 -- 172.21.15.105:0/3645899922 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa34c06f860 msgr2=0x7fa34c06ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:02.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.603+0000 7fa3526b9640 1 --2- 172.21.15.105:0/3645899922 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa34c06f860 0x7fa34c06ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:02.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.603+0000 7fa3526b9640 1 -- 172.21.15.105:0/3645899922 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa34c0adf80 con 0x7fa34c144e80 2024-04-15T15:20:02.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.603+0000 7fa3526b9640 1 --2- 172.21.15.105:0/3645899922 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa34c144e80 0x7fa34c0ab0e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa34400cef0 tx=0x7fa34400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:02.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.603+0000 7fa3427fc640 1 -- 172.21.15.105:0/3645899922 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34401a070 con 0x7fa34c144e80 2024-04-15T15:20:02.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.603+0000 7fa3427fc640 1 -- 172.21.15.105:0/3645899922 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa34400d860 con 0x7fa34c144e80 2024-04-15T15:20:02.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.603+0000 7fa35411c640 1 -- 172.21.15.105:0/3645899922 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa34c0ae260 con 0x7fa34c144e80 2024-04-15T15:20:02.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.603+0000 7fa3427fc640 1 -- 172.21.15.105:0/3645899922 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34400dc50 con 0x7fa34c144e80 2024-04-15T15:20:02.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.603+0000 7fa35411c640 1 -- 172.21.15.105:0/3645899922 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa34c0ae820 con 0x7fa34c144e80 2024-04-15T15:20:02.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.604+0000 7fa35411c640 1 -- 172.21.15.105:0/3645899922 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa34c068a40 con 0x7fa34c144e80 2024-04-15T15:20:02.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.605+0000 7fa3427fc640 1 -- 172.21.15.105:0/3645899922 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa34401d070 con 0x7fa34c144e80 2024-04-15T15:20:02.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.605+0000 7fa3427fc640 1 --2- 172.21.15.105:0/3645899922 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fa33006c8f0 0x7fa33006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:02.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.605+0000 7fa351eb8640 1 --2- 172.21.15.105:0/3645899922 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fa33006c8f0 0x7fa33006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:02.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.605+0000 7fa3427fc640 1 -- 172.21.15.105:0/3645899922 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa34408c2f0 con 0x7fa34c144e80 2024-04-15T15:20:02.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.606+0000 7fa351eb8640 1 --2- 172.21.15.105:0/3645899922 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fa33006c8f0 0x7fa33006edb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa348036000 tx=0x7fa34803d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:02.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.610+0000 7fa3427fc640 1 -- 172.21.15.105:0/3645899922 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa34405a160 con 0x7fa34c144e80 2024-04-15T15:20:02.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.782+0000 7fa35411c640 1 -- 172.21.15.105:0/3645899922 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa34c0730d0 con 0x7fa34c144e80 2024-04-15T15:20:02.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.783+0000 7fa3427fc640 1 -- 172.21.15.105:0/3645899922 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fa34405e460 con 0x7fa34c144e80 2024-04-15T15:20:02.785 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-04-15T15:20:02.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.785+0000 7fa35411c640 1 -- 172.21.15.105:0/3645899922 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fa33006c8f0 msgr2=0x7fa33006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:02.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.785+0000 7fa35411c640 1 --2- 172.21.15.105:0/3645899922 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fa33006c8f0 0x7fa33006edb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa348036000 tx=0x7fa34803d040 comp rx=0 tx=0).stop 2024-04-15T15:20:02.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.785+0000 7fa35411c640 1 -- 172.21.15.105:0/3645899922 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa34c144e80 msgr2=0x7fa34c0ab0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:02.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.785+0000 7fa35411c640 1 --2- 172.21.15.105:0/3645899922 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa34c144e80 0x7fa34c0ab0e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa34400cef0 tx=0x7fa34400a710 comp rx=0 tx=0).stop 2024-04-15T15:20:02.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.785+0000 7fa35411c640 1 -- 172.21.15.105:0/3645899922 shutdown_connections 2024-04-15T15:20:02.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.785+0000 7fa35411c640 1 --2- 172.21.15.105:0/3645899922 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa34c0b4910 0x7fa34c0ad810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:02.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.785+0000 7fa35411c640 1 --2- 172.21.15.105:0/3645899922 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fa33006c8f0 0x7fa33006edb0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:02.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.785+0000 7fa35411c640 1 --2- 172.21.15.105:0/3645899922 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa34c144e80 0x7fa34c0ab0e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:02.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.785+0000 7fa35411c640 1 --2- 172.21.15.105:0/3645899922 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa34c06f860 0x7fa34c06ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:02.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.785+0000 7fa35411c640 1 -- 172.21.15.105:0/3645899922 >> 172.21.15.105:0/3645899922 conn(0x7fa34c06b9c0 msgr2=0x7fa34c0b4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:02.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.785+0000 7fa35411c640 1 -- 172.21.15.105:0/3645899922 shutdown_connections 2024-04-15T15:20:02.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.785+0000 7fa35411c640 1 -- 172.21.15.105:0/3645899922 wait complete. 2024-04-15T15:20:02.798 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-15T15:20:02.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.902+0000 7fdd08b67640 1 Processor -- start 2024-04-15T15:20:02.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.902+0000 7fdd08b67640 1 -- start start 2024-04-15T15:20:02.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.903+0000 7fdd08b67640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fdd04072e00 0x7fdd0406eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:02.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.903+0000 7fdd08b67640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd0415bce0 con 0x7fdd0415b1c0 2024-04-15T15:20:02.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.903+0000 7fdd08b67640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd0415bfa0 con 0x7fdd04151e30 2024-04-15T15:20:02.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.903+0000 7fdd08b67640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fdd0406e670 con 0x7fdd04072e00 2024-04-15T15:20:02.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.903+0000 7fdd02d9d640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fdd0415b1c0 0x7fdd0415b5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:55096/0 (socket says 172.21.15.105:55096) 2024-04-15T15:20:02.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.903+0000 7fdd02d9d640 1 -- 172.21.15.105:0/2150474273 learned_addr learned my addr 172.21.15.105:0/2150474273 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:20:02.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.903+0000 7fdcf9d9b640 1 --2- 172.21.15.105:0/2150474273 >> v2:172.21.15.154:3300/0 conn(0x7fdd04072e00 0x7fdd0406eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:02.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.903+0000 7fdd01d9b640 1 -- 172.21.15.105:0/2150474273 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1713613810 0 0) 0x7fdd0415bce0 con 0x7fdd0415b1c0 2024-04-15T15:20:02.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.903+0000 7fdd01d9b640 1 -- 172.21.15.105:0/2150474273 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdcf0004630 con 0x7fdd0415b1c0 2024-04-15T15:20:02.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.903+0000 7fdcf9d9b640 1 -- 172.21.15.105:0/2150474273 >> v1:172.21.15.154:6789/0 conn(0x7fdd04151e30 legacy=0x7fdd040726f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:02.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.903+0000 7fdcf9d9b640 1 -- 172.21.15.105:0/2150474273 >> v1:172.21.15.105:6789/0 conn(0x7fdd0415b1c0 legacy=0x7fdd0415b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:02.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.903+0000 7fdcf9d9b640 1 -- 172.21.15.105:0/2150474273 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd0406e020 con 0x7fdd04072e00 2024-04-15T15:20:02.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.904+0000 7fdcf9d9b640 1 --2- 172.21.15.105:0/2150474273 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd04072e00 0x7fdd0406eb00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdcec0097a0 tx=0x7fdcec02b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a80cefad32658893 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:02.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.904+0000 7fdd01d9b640 1 -- 172.21.15.105:0/2150474273 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcec034070 con 0x7fdd04072e00 2024-04-15T15:20:02.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.904+0000 7fdd01d9b640 1 -- 172.21.15.105:0/2150474273 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcec030610 con 0x7fdd04072e00 2024-04-15T15:20:02.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.904+0000 7fdd01d9b640 1 -- 172.21.15.105:0/2150474273 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcec0309c0 con 0x7fdd04072e00 2024-04-15T15:20:02.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.905+0000 7fdd08b67640 1 -- 172.21.15.105:0/2150474273 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd04072e00 msgr2=0x7fdd0406eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:02.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.905+0000 7fdd08b67640 1 --2- 172.21.15.105:0/2150474273 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd04072e00 0x7fdd0406eb00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdcec0097a0 tx=0x7fdcec02b960 comp rx=0 tx=0).stop 2024-04-15T15:20:02.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.905+0000 7fdd08b67640 1 -- 172.21.15.105:0/2150474273 shutdown_connections 2024-04-15T15:20:02.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.905+0000 7fdd08b67640 1 --2- 172.21.15.105:0/2150474273 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd04072e00 0x7fdd0406eb00 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:02.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.905+0000 7fdd08b67640 1 -- 172.21.15.105:0/2150474273 >> 172.21.15.105:0/2150474273 conn(0x7fdd04067d50 msgr2=0x7fdd0406b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:02.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.905+0000 7fdd08b67640 1 -- 172.21.15.105:0/2150474273 shutdown_connections 2024-04-15T15:20:02.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.905+0000 7fdd08b67640 1 -- 172.21.15.105:0/2150474273 wait complete. 2024-04-15T15:20:02.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.906+0000 7fdd08b67640 1 Processor -- start 2024-04-15T15:20:02.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.906+0000 7fdd08b67640 1 -- start start 2024-04-15T15:20:02.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.907+0000 7fdd08b67640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd04072e00 0x7fdd0414ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:02.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.907+0000 7fdd08b67640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd04151e30 0x7fdd041558e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:02.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.907+0000 7fdd08b67640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdd0415b1c0 0x7fdd04153f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:02.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.907+0000 7fdd08b67640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdd04071fe0 con 0x7fdd04151e30 2024-04-15T15:20:02.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.907+0000 7fdd08b67640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdd04071e60 con 0x7fdd04072e00 2024-04-15T15:20:02.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.907+0000 7fdd08b67640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fdd04072160 con 0x7fdd0415b1c0 2024-04-15T15:20:02.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.907+0000 7fdcf9d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd04151e30 0x7fdd041558e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:02.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.907+0000 7fdcf9d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd04151e30 0x7fdd041558e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37428/0 (socket says 172.21.15.105:37428) 2024-04-15T15:20:02.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.907+0000 7fdcf9d9b640 1 -- 172.21.15.105:0/3030280244 learned_addr learned my addr 172.21.15.105:0/3030280244 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:02.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.907+0000 7fdd0259c640 1 --2- 172.21.15.105:0/3030280244 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd04072e00 0x7fdd0414ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:02.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.907+0000 7fdd02d9d640 1 --2- 172.21.15.105:0/3030280244 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdd0415b1c0 0x7fdd04153f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:02.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.907+0000 7fdcf9d9b640 1 -- 172.21.15.105:0/3030280244 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdd0415b1c0 msgr2=0x7fdd04153f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:02.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.907+0000 7fdcf9d9b640 1 --2- 172.21.15.105:0/3030280244 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdd0415b1c0 0x7fdd04153f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:02.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.907+0000 7fdcf9d9b640 1 -- 172.21.15.105:0/3030280244 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd04072e00 msgr2=0x7fdd0414ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:02.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.907+0000 7fdcf9d9b640 1 --2- 172.21.15.105:0/3030280244 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd04072e00 0x7fdd0414ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:02.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.908+0000 7fdcf9d9b640 1 -- 172.21.15.105:0/3030280244 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd04156110 con 0x7fdd04151e30 2024-04-15T15:20:02.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.908+0000 7fdcf9d9b640 1 --2- 172.21.15.105:0/3030280244 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd04151e30 0x7fdd041558e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fdcec000c00 tx=0x7fdcec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:02.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.908+0000 7fdcfb7fe640 1 -- 172.21.15.105:0/3030280244 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcec034040 con 0x7fdd04151e30 2024-04-15T15:20:02.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.908+0000 7fdcfb7fe640 1 -- 172.21.15.105:0/3030280244 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcec030410 con 0x7fdd04151e30 2024-04-15T15:20:02.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.908+0000 7fdd08b67640 1 -- 172.21.15.105:0/3030280244 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd041563f0 con 0x7fdd04151e30 2024-04-15T15:20:02.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.908+0000 7fdcfb7fe640 1 -- 172.21.15.105:0/3030280244 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcec004230 con 0x7fdd04151e30 2024-04-15T15:20:02.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.908+0000 7fdd08b67640 1 -- 172.21.15.105:0/3030280244 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd04157610 con 0x7fdd04151e30 2024-04-15T15:20:02.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.909+0000 7fdd08b67640 1 -- 172.21.15.105:0/3030280244 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd04068ab0 con 0x7fdd04151e30 2024-04-15T15:20:02.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.909+0000 7fdcfb7fe640 1 -- 172.21.15.105:0/3030280244 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdcec04c060 con 0x7fdd04151e30 2024-04-15T15:20:02.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.910+0000 7fdcfb7fe640 1 --2- 172.21.15.105:0/3030280244 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fdce406c920 0x7fdce406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:02.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.910+0000 7fdcfb7fe640 1 -- 172.21.15.105:0/3030280244 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fdcec0ad500 con 0x7fdd04151e30 2024-04-15T15:20:02.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.910+0000 7fdd0259c640 1 --2- 172.21.15.105:0/3030280244 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fdce406c920 0x7fdce406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:02.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.910+0000 7fdd0259c640 1 --2- 172.21.15.105:0/3030280244 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fdce406c920 0x7fdce406ede0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fdcf4000f60 tx=0x7fdcf4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:02.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:02.914+0000 7fdcfb7fe640 1 -- 172.21.15.105:0/3030280244 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdcec07b9c0 con 0x7fdd04151e30 2024-04-15T15:20:03.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.090+0000 7fdd08b67640 1 -- 172.21.15.105:0/3030280244 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdd0406bb60 con 0x7fdd04151e30 2024-04-15T15:20:03.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.090+0000 7fdcfb7fe640 1 -- 172.21.15.105:0/3030280244 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fdcec07b2c0 con 0x7fdd04151e30 2024-04-15T15:20:03.092 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-04-15T15:20:03.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.092+0000 7fdd08b67640 1 -- 172.21.15.105:0/3030280244 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fdce406c920 msgr2=0x7fdce406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:03.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.092+0000 7fdd08b67640 1 --2- 172.21.15.105:0/3030280244 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fdce406c920 0x7fdce406ede0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fdcf4000f60 tx=0x7fdcf4009040 comp rx=0 tx=0).stop 2024-04-15T15:20:03.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.092+0000 7fdd08b67640 1 -- 172.21.15.105:0/3030280244 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd04151e30 msgr2=0x7fdd041558e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:03.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.092+0000 7fdd08b67640 1 --2- 172.21.15.105:0/3030280244 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd04151e30 0x7fdd041558e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fdcec000c00 tx=0x7fdcec0046c0 comp rx=0 tx=0).stop 2024-04-15T15:20:03.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.092+0000 7fdd08b67640 1 -- 172.21.15.105:0/3030280244 shutdown_connections 2024-04-15T15:20:03.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.092+0000 7fdd08b67640 1 --2- 172.21.15.105:0/3030280244 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdd0415b1c0 0x7fdd04153f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:03.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.092+0000 7fdd08b67640 1 --2- 172.21.15.105:0/3030280244 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fdce406c920 0x7fdce406ede0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:03.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.092+0000 7fdd08b67640 1 --2- 172.21.15.105:0/3030280244 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd04151e30 0x7fdd041558e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:03.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.092+0000 7fdd08b67640 1 --2- 172.21.15.105:0/3030280244 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd04072e00 0x7fdd0414ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:03.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.092+0000 7fdd08b67640 1 -- 172.21.15.105:0/3030280244 >> 172.21.15.105:0/3030280244 conn(0x7fdd04067d50 msgr2=0x7fdd04150e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:03.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.092+0000 7fdd08b67640 1 -- 172.21.15.105:0/3030280244 shutdown_connections 2024-04-15T15:20:03.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.092+0000 7fdd08b67640 1 -- 172.21.15.105:0/3030280244 wait complete. 2024-04-15T15:20:03.105 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-15T15:20:03.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.207+0000 7f1b19bba640 1 Processor -- start 2024-04-15T15:20:03.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.207+0000 7f1b19bba640 1 -- start start 2024-04-15T15:20:03.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.208+0000 7f1b19bba640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1b140d0f10 0x7f1b14169c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:03.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.208+0000 7f1b19bba640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b1406d2a0 con 0x7f1b1415ecf0 2024-04-15T15:20:03.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.208+0000 7f1b19bba640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b1406d580 con 0x7f1b1406e060 2024-04-15T15:20:03.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.208+0000 7f1b19bba640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f1b1406d860 con 0x7f1b140d0f10 2024-04-15T15:20:03.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.208+0000 7f1b137fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f1b1415ecf0 0x7f1b140ccc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:55110/0 (socket says 172.21.15.105:55110) 2024-04-15T15:20:03.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.208+0000 7f1b137fe640 1 -- 172.21.15.105:0/2258250847 learned_addr learned my addr 172.21.15.105:0/2258250847 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:20:03.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.208+0000 7f1b13fff640 1 --2- 172.21.15.105:0/2258250847 >> v2:172.21.15.154:3300/0 conn(0x7f1b140d0f10 0x7f1b14169c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:03.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.208+0000 7f1b127fc640 1 -- 172.21.15.105:0/2258250847 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 795169419 0 0) 0x7f1b1406d2a0 con 0x7f1b1415ecf0 2024-04-15T15:20:03.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.208+0000 7f1b127fc640 1 -- 172.21.15.105:0/2258250847 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1afc003520 con 0x7f1b1415ecf0 2024-04-15T15:20:03.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.209+0000 7f1b127fc640 1 -- 172.21.15.105:0/2258250847 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3097894250 0 0) 0x7f1b1406d580 con 0x7f1b1406e060 2024-04-15T15:20:03.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.209+0000 7f1b127fc640 1 -- 172.21.15.105:0/2258250847 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b1406d2a0 con 0x7f1b1406e060 2024-04-15T15:20:03.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.209+0000 7f1b13fff640 1 -- 172.21.15.105:0/2258250847 >> v1:172.21.15.154:6789/0 conn(0x7f1b1406e060 legacy=0x7f1b140cd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:03.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.209+0000 7f1b127fc640 1 -- 172.21.15.105:0/2258250847 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2847764331 0 0) 0x7f1afc003520 con 0x7f1b1415ecf0 2024-04-15T15:20:03.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.209+0000 7f1b13fff640 1 -- 172.21.15.105:0/2258250847 >> v1:172.21.15.105:6789/0 conn(0x7f1b1415ecf0 legacy=0x7f1b140ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:03.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.209+0000 7f1b13fff640 1 -- 172.21.15.105:0/2258250847 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b1406db40 con 0x7f1b140d0f10 2024-04-15T15:20:03.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.210+0000 7f1b13fff640 1 --2- 172.21.15.105:0/2258250847 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b140d0f10 0x7f1b14169c90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c0097a0 tx=0x7f1b0c02b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=181044dfc7f2e316 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:03.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.210+0000 7f1b127fc640 1 -- 172.21.15.105:0/2258250847 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c038070 con 0x7f1b140d0f10 2024-04-15T15:20:03.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.210+0000 7f1b127fc640 1 -- 172.21.15.105:0/2258250847 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b0c031500 con 0x7f1b140d0f10 2024-04-15T15:20:03.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.210+0000 7f1b127fc640 1 -- 172.21.15.105:0/2258250847 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b040019f0 con 0x7f1b140d0f10 2024-04-15T15:20:03.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.210+0000 7f1b19bba640 1 -- 172.21.15.105:0/2258250847 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b140d0f10 msgr2=0x7f1b14169c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:03.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.210+0000 7f1b19bba640 1 --2- 172.21.15.105:0/2258250847 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b140d0f10 0x7f1b14169c90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c0097a0 tx=0x7f1b0c02b820 comp rx=0 tx=0).stop 2024-04-15T15:20:03.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.210+0000 7f1b19bba640 1 -- 172.21.15.105:0/2258250847 shutdown_connections 2024-04-15T15:20:03.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.210+0000 7f1b19bba640 1 --2- 172.21.15.105:0/2258250847 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b140d0f10 0x7f1b14169c90 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:03.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.210+0000 7f1b19bba640 1 -- 172.21.15.105:0/2258250847 >> 172.21.15.105:0/2258250847 conn(0x7f1b140d2600 msgr2=0x7f1b140d2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:03.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.210+0000 7f1b19bba640 1 -- 172.21.15.105:0/2258250847 shutdown_connections 2024-04-15T15:20:03.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.210+0000 7f1b19bba640 1 -- 172.21.15.105:0/2258250847 wait complete. 2024-04-15T15:20:03.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.211+0000 7f1b19bba640 1 Processor -- start 2024-04-15T15:20:03.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.212+0000 7f1b19bba640 1 -- start start 2024-04-15T15:20:03.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.212+0000 7f1b19bba640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b1406e060 0x7f1b140cb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:03.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.212+0000 7f1b19bba640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b140d0f10 0x7f1b141548e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:03.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.212+0000 7f1b19bba640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1b1415ecf0 0x7f1b14154e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:03.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.212+0000 7f1b19bba640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1b1415f0d0 con 0x7f1b1415ecf0 2024-04-15T15:20:03.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.212+0000 7f1b19bba640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1b1406cf90 con 0x7f1b140d0f10 2024-04-15T15:20:03.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.212+0000 7f1b19bba640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f1b1415f250 con 0x7f1b1406e060 2024-04-15T15:20:03.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.212+0000 7f1b13fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1b1415ecf0 0x7f1b14154e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:03.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.212+0000 7f1b13fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1b1415ecf0 0x7f1b14154e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37434/0 (socket says 172.21.15.105:37434) 2024-04-15T15:20:03.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.213+0000 7f1b13fff640 1 -- 172.21.15.105:0/3388354930 learned_addr learned my addr 172.21.15.105:0/3388354930 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:03.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.213+0000 7f1b12ffd640 1 --2- 172.21.15.105:0/3388354930 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b140d0f10 0x7f1b141548e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:03.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.213+0000 7f1b137fe640 1 --2- 172.21.15.105:0/3388354930 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b1406e060 0x7f1b140cb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:03.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.213+0000 7f1b13fff640 1 -- 172.21.15.105:0/3388354930 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b1406e060 msgr2=0x7f1b140cb4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:03.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.213+0000 7f1b13fff640 1 --2- 172.21.15.105:0/3388354930 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b1406e060 0x7f1b140cb4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:03.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.213+0000 7f1b13fff640 1 -- 172.21.15.105:0/3388354930 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b140d0f10 msgr2=0x7f1b141548e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:03.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.213+0000 7f1b13fff640 1 --2- 172.21.15.105:0/3388354930 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b140d0f10 0x7f1b141548e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:03.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.213+0000 7f1b13fff640 1 -- 172.21.15.105:0/3388354930 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b140ca550 con 0x7f1b1415ecf0 2024-04-15T15:20:03.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.213+0000 7f1b13fff640 1 --2- 172.21.15.105:0/3388354930 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1b1415ecf0 0x7f1b14154e20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c0097a0 tx=0x7f1b0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:03.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.213+0000 7f1afbfff640 1 -- 172.21.15.105:0/3388354930 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c042070 con 0x7f1b1415ecf0 2024-04-15T15:20:03.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.213+0000 7f1afbfff640 1 -- 172.21.15.105:0/3388354930 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b0c03d890 con 0x7f1b1415ecf0 2024-04-15T15:20:03.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.213+0000 7f1b19bba640 1 -- 172.21.15.105:0/3388354930 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b140ca830 con 0x7f1b1415ecf0 2024-04-15T15:20:03.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.213+0000 7f1afbfff640 1 -- 172.21.15.105:0/3388354930 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c038040 con 0x7f1b1415ecf0 2024-04-15T15:20:03.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.213+0000 7f1b19bba640 1 -- 172.21.15.105:0/3388354930 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b140b52c0 con 0x7f1b1415ecf0 2024-04-15T15:20:03.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.214+0000 7f1b19bba640 1 -- 172.21.15.105:0/3388354930 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b14068a40 con 0x7f1b1415ecf0 2024-04-15T15:20:03.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.215+0000 7f1afbfff640 1 -- 172.21.15.105:0/3388354930 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1b0c045070 con 0x7f1b1415ecf0 2024-04-15T15:20:03.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.215+0000 7f1afbfff640 1 --2- 172.21.15.105:0/3388354930 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f1b0006c8f0 0x7f1b0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:03.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.215+0000 7f1afbfff640 1 -- 172.21.15.105:0/3388354930 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1b0c0ad6b0 con 0x7f1b1415ecf0 2024-04-15T15:20:03.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.215+0000 7f1b137fe640 1 --2- 172.21.15.105:0/3388354930 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f1b0006c8f0 0x7f1b0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:03.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.216+0000 7f1b137fe640 1 --2- 172.21.15.105:0/3388354930 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f1b0006c8f0 0x7f1b0006edb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1b04001b60 tx=0x7f1b04009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:03.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.219+0000 7f1afbfff640 1 -- 172.21.15.105:0/3388354930 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1b0c030070 con 0x7f1b1415ecf0 2024-04-15T15:20:03.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.408+0000 7f1b19bba640 1 -- 172.21.15.105:0/3388354930 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1b140cb010 con 0x7f1b1415ecf0 2024-04-15T15:20:03.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.408+0000 7f1afbfff640 1 -- 172.21.15.105:0/3388354930 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7f1b0c07bae0 con 0x7f1b1415ecf0 2024-04-15T15:20:03.410 INFO:teuthology.orchestra.run.smithi105.stderr:listed 0 entries 2024-04-15T15:20:03.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.410+0000 7f1b19bba640 1 -- 172.21.15.105:0/3388354930 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f1b0006c8f0 msgr2=0x7f1b0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:03.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.410+0000 7f1b19bba640 1 --2- 172.21.15.105:0/3388354930 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f1b0006c8f0 0x7f1b0006edb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1b04001b60 tx=0x7f1b04009040 comp rx=0 tx=0).stop 2024-04-15T15:20:03.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.410+0000 7f1b19bba640 1 -- 172.21.15.105:0/3388354930 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1b1415ecf0 msgr2=0x7f1b14154e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:03.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.410+0000 7f1b19bba640 1 --2- 172.21.15.105:0/3388354930 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1b1415ecf0 0x7f1b14154e20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c0097a0 tx=0x7f1b0c0046c0 comp rx=0 tx=0).stop 2024-04-15T15:20:03.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.410+0000 7f1b19bba640 1 -- 172.21.15.105:0/3388354930 shutdown_connections 2024-04-15T15:20:03.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.410+0000 7f1b19bba640 1 --2- 172.21.15.105:0/3388354930 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f1b0006c8f0 0x7f1b0006edb0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:03.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.410+0000 7f1b19bba640 1 --2- 172.21.15.105:0/3388354930 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1b1415ecf0 0x7f1b14154e20 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:03.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.410+0000 7f1b19bba640 1 --2- 172.21.15.105:0/3388354930 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b140d0f10 0x7f1b141548e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:03.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.410+0000 7f1b19bba640 1 --2- 172.21.15.105:0/3388354930 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b1406e060 0x7f1b140cb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:03.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.410+0000 7f1b19bba640 1 -- 172.21.15.105:0/3388354930 >> 172.21.15.105:0/3388354930 conn(0x7f1b140d2600 msgr2=0x7f1b140ccc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:03.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.410+0000 7f1b19bba640 1 -- 172.21.15.105:0/3388354930 shutdown_connections 2024-04-15T15:20:03.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.410+0000 7f1b19bba640 1 -- 172.21.15.105:0/3388354930 wait complete. 2024-04-15T15:20:03.423 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-04-15T15:20:03.423 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:20:03.439 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:20:03.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.581+0000 7f0299674640 1 Processor -- start 2024-04-15T15:20:03.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.581+0000 7f0299674640 1 -- start start 2024-04-15T15:20:03.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.581+0000 7f0299674640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f029415eb60 0x7f02940d11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:03.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.581+0000 7f0299674640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f0294162d40 0x7f02940d3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:03.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.581+0000 7f0299674640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02940d1b30 con 0x7f02940d1710 2024-04-15T15:20:03.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.581+0000 7f0299674640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f0294163540 con 0x7f0294162d40 2024-04-15T15:20:03.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.581+0000 7f0299674640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f02940d2fc0 con 0x7f029415eb60 2024-04-15T15:20:03.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.582+0000 7f028a7fc640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f02940d1710 0x7f0294162630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:55124/0 (socket says 172.21.15.105:55124) 2024-04-15T15:20:03.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.582+0000 7f028a7fc640 1 -- 172.21.15.105:0/3869664317 learned_addr learned my addr 172.21.15.105:0/3869664317 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:20:03.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.582+0000 7f02927fc640 1 -- 172.21.15.105:0/3869664317 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 373006121 0 0) 0x7f02940d1b30 con 0x7f02940d1710 2024-04-15T15:20:03.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.582+0000 7f0292ffd640 1 --2- 172.21.15.105:0/3869664317 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f029415eb60 0x7f02940d11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:03.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.582+0000 7f02937fe640 1 --2- 172.21.15.105:0/3869664317 >> v2:172.21.15.154:3300/0 conn(0x7f0294162d40 0x7f02940d3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:03.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.582+0000 7f02927fc640 1 -- 172.21.15.105:0/3869664317 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0280003520 con 0x7f02940d1710 2024-04-15T15:20:03.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.582+0000 7f02927fc640 1 -- 172.21.15.105:0/3869664317 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 638566854 0 0) 0x7f0280003520 con 0x7f02940d1710 2024-04-15T15:20:03.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.582+0000 7f02927fc640 1 -- 172.21.15.105:0/3869664317 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f029415eb60 msgr2=0x7f02940d11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:03.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.582+0000 7f02927fc640 1 --2- 172.21.15.105:0/3869664317 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f029415eb60 0x7f02940d11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:03.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.582+0000 7f02927fc640 1 -- 172.21.15.105:0/3869664317 >> v2:172.21.15.154:3300/0 conn(0x7f0294162d40 msgr2=0x7f02940d3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:03.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.582+0000 7f02927fc640 1 --2- 172.21.15.105:0/3869664317 >> v2:172.21.15.154:3300/0 conn(0x7f0294162d40 0x7f02940d3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:03.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.582+0000 7f02927fc640 1 -- 172.21.15.105:0/3869664317 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f029415ef40 con 0x7f02940d1710 2024-04-15T15:20:03.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.582+0000 7f02927fc640 1 -- 172.21.15.105:0/3869664317 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7f027c001670 con 0x7f02940d1710 2024-04-15T15:20:03.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.582+0000 7f02927fc640 1 -- 172.21.15.105:0/3869664317 >> v1:172.21.15.105:6789/0 conn(0x7f02940d1710 legacy=0x7f0294162630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:03.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.582+0000 7f02927fc640 1 --2- 172.21.15.105:0/3869664317 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f029415eb60 0x7f02800039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:03.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.582+0000 7f02927fc640 1 --2- 172.21.15.105:0/3869664317 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0280003f30 0x7f0280004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:03.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.582+0000 7f02927fc640 1 --2- 172.21.15.105:0/3869664317 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0280004d40 0x7f02800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:03.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.582+0000 7f02927fc640 1 -- 172.21.15.105:0/3869664317 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0294163540 con 0x7f0280003f30 2024-04-15T15:20:03.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.582+0000 7f02927fc640 1 -- 172.21.15.105:0/3869664317 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f02940d2fc0 con 0x7f0280004d40 2024-04-15T15:20:03.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.582+0000 7f02927fc640 1 -- 172.21.15.105:0/3869664317 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f028000b9c0 con 0x7f029415eb60 2024-04-15T15:20:03.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.582+0000 7f028a7fc640 1 --2- 172.21.15.105:0/3869664317 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0280003f30 0x7f0280004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:03.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.582+0000 7f0292ffd640 1 --2- 172.21.15.105:0/3869664317 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f029415eb60 0x7f02800039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:03.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.583+0000 7f02937fe640 1 --2- 172.21.15.105:0/3869664317 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0280004d40 0x7f02800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:03.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.583+0000 7f028a7fc640 1 -- 172.21.15.105:0/3869664317 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f029415eb60 msgr2=0x7f02800039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:03.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.583+0000 7f028a7fc640 1 --2- 172.21.15.105:0/3869664317 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f029415eb60 0x7f02800039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:03.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.583+0000 7f028a7fc640 1 -- 172.21.15.105:0/3869664317 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0280004d40 msgr2=0x7f02800051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:03.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.583+0000 7f028a7fc640 1 --2- 172.21.15.105:0/3869664317 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0280004d40 0x7f02800051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:03.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.583+0000 7f028a7fc640 1 -- 172.21.15.105:0/3869664317 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0280003520 con 0x7f0280003f30 2024-04-15T15:20:03.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.583+0000 7f028a7fc640 1 --2- 172.21.15.105:0/3869664317 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0280003f30 0x7f0280004800 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f027c001050 tx=0x7f0280005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=507a1c58bfcf15e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:03.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.584+0000 7f02927fc640 1 -- 172.21.15.105:0/3869664317 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027c007d70 con 0x7f0280003f30 2024-04-15T15:20:03.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.584+0000 7f02927fc640 1 -- 172.21.15.105:0/3869664317 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f027c030070 con 0x7f0280003f30 2024-04-15T15:20:03.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.584+0000 7f02927fc640 1 -- 172.21.15.105:0/3869664317 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027c02cd70 con 0x7f0280003f30 2024-04-15T15:20:03.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.584+0000 7f0299674640 1 -- 172.21.15.105:0/3869664317 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0280003f30 msgr2=0x7f0280004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:03.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.584+0000 7f0299674640 1 --2- 172.21.15.105:0/3869664317 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0280003f30 0x7f0280004800 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f027c001050 tx=0x7f0280005830 comp rx=0 tx=0).stop 2024-04-15T15:20:03.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.584+0000 7f02937fe640 1 -- 172.21.15.105:0/3869664317 reap_dead start 2024-04-15T15:20:03.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.584+0000 7f0299674640 1 -- 172.21.15.105:0/3869664317 shutdown_connections 2024-04-15T15:20:03.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.584+0000 7f0299674640 1 -- 172.21.15.105:0/3869664317 >> 172.21.15.105:0/3869664317 conn(0x7f0294067720 msgr2=0x7f0294067b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:03.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.584+0000 7f0299674640 1 -- 172.21.15.105:0/3869664317 shutdown_connections 2024-04-15T15:20:03.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.584+0000 7f0299674640 1 -- 172.21.15.105:0/3869664317 wait complete. 2024-04-15T15:20:03.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.585+0000 7f0299674640 1 Processor -- start 2024-04-15T15:20:03.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.586+0000 7f0299674640 1 -- start start 2024-04-15T15:20:03.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.586+0000 7f0299674640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0280003f30 0x7f02940b4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:03.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.586+0000 7f0299674640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f02941f4680 0x7f029414bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:03.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.586+0000 7f0299674640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f02940d43a0 0x7f02940d4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:03.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.586+0000 7f0299674640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f02940d2c50 con 0x7f0280003f30 2024-04-15T15:20:03.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.586+0000 7f0299674640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f02940d2970 con 0x7f02941f4680 2024-04-15T15:20:03.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.586+0000 7f0299674640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f02940b78f0 con 0x7f02940d43a0 2024-04-15T15:20:03.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.586+0000 7f0292ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0280003f30 0x7f02940b4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:03.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.586+0000 7f0292ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0280003f30 0x7f02940b4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37448/0 (socket says 172.21.15.105:37448) 2024-04-15T15:20:03.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.586+0000 7f0292ffd640 1 -- 172.21.15.105:0/2122928502 learned_addr learned my addr 172.21.15.105:0/2122928502 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:03.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.587+0000 7f02937fe640 1 --2- 172.21.15.105:0/2122928502 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f02940d43a0 0x7f02940d4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:03.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.587+0000 7f028a7fc640 1 --2- 172.21.15.105:0/2122928502 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f02941f4680 0x7f029414bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:03.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.587+0000 7f0292ffd640 1 -- 172.21.15.105:0/2122928502 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f02940d43a0 msgr2=0x7f02940d4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:03.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.587+0000 7f0292ffd640 1 --2- 172.21.15.105:0/2122928502 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f02940d43a0 0x7f02940d4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:03.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.587+0000 7f0292ffd640 1 -- 172.21.15.105:0/2122928502 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f02941f4680 msgr2=0x7f029414bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:03.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.587+0000 7f0292ffd640 1 --2- 172.21.15.105:0/2122928502 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f02941f4680 0x7f029414bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:03.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.587+0000 7f0292ffd640 1 -- 172.21.15.105:0/2122928502 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02940ccd90 con 0x7f0280003f30 2024-04-15T15:20:03.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.587+0000 7f02937fe640 1 --2- 172.21.15.105:0/2122928502 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f02940d43a0 0x7f02940d4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:03.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.587+0000 7f028a7fc640 1 --2- 172.21.15.105:0/2122928502 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f02941f4680 0x7f029414bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:03.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.587+0000 7f0292ffd640 1 --2- 172.21.15.105:0/2122928502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0280003f30 0x7f02940b4830 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f02840118e0 tx=0x7f0284011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:03.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.587+0000 7f028bfff640 1 -- 172.21.15.105:0/2122928502 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028400faf0 con 0x7f0280003f30 2024-04-15T15:20:03.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.587+0000 7f028bfff640 1 -- 172.21.15.105:0/2122928502 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f028400fc90 con 0x7f0280003f30 2024-04-15T15:20:03.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.587+0000 7f0299674640 1 -- 172.21.15.105:0/2122928502 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02941f4f40 con 0x7f0280003f30 2024-04-15T15:20:03.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.587+0000 7f028bfff640 1 -- 172.21.15.105:0/2122928502 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0284014440 con 0x7f0280003f30 2024-04-15T15:20:03.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.587+0000 7f0299674640 1 -- 172.21.15.105:0/2122928502 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02941f5500 con 0x7f0280003f30 2024-04-15T15:20:03.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.588+0000 7f0299674640 1 -- 172.21.15.105:0/2122928502 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0294068a40 con 0x7f0280003f30 2024-04-15T15:20:03.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.588+0000 7f028bfff640 1 -- 172.21.15.105:0/2122928502 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0284012040 con 0x7f0280003f30 2024-04-15T15:20:03.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.589+0000 7f028bfff640 1 --2- 172.21.15.105:0/2122928502 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f027406c920 0x7f027406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:03.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.589+0000 7f028bfff640 1 -- 172.21.15.105:0/2122928502 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f028408c120 con 0x7f0280003f30 2024-04-15T15:20:03.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.589+0000 7f028a7fc640 1 --2- 172.21.15.105:0/2122928502 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f027406c920 0x7f027406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:03.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.589+0000 7f028a7fc640 1 --2- 172.21.15.105:0/2122928502 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f027406c920 0x7f027406ede0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f027c034000 tx=0x7f027c007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:03.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.593+0000 7f028bfff640 1 -- 172.21.15.105:0/2122928502 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0284019030 con 0x7f0280003f30 2024-04-15T15:20:03.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.769+0000 7f0299674640 1 -- 172.21.15.105:0/2122928502 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f02940d23f0 con 0x7f0280003f30 2024-04-15T15:20:03.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.770+0000 7f028bfff640 1 -- 172.21.15.105:0/2122928502 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16653 (secure 0 0 0) 0x7f028405a5e0 con 0x7f0280003f30 2024-04-15T15:20:03.772 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:20:03.773 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":20,"fsid":"09e4d2a4-a324-4793-af77-644b25e520e1","created":"2024-04-15T15:19:01.524543+0000","modified":"2024-04-15T15:19:27.414890+0000","last_up_change":"2024-04-15T15:19:12.825815+0000","last_in_change":"2024-04-15T15:19:03.788126+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:19:13.748585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:19:23.789287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:19:24.722250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6f6292b-2d7e-4aa0-9d1c-6ddd0f97287c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6801","nonce":2873040054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6803","nonce":2873040054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6807","nonce":2873040054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6805","nonce":2873040054}]},"public_addr":"172.21.15.105:6801/2873040054","cluster_addr":"172.21.15.105:6803/2873040054","heartbeat_back_addr":"172.21.15.105:6807/2873040054","heartbeat_front_addr":"172.21.15.105:6805/2873040054","state":["exists","up"]},{"osd":1,"uuid":"8900966a-b393-41e3-8f93-e5a43bd23622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6818","nonce":2494994479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6821","nonce":2494994479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6828","nonce":2494994479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6823","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6825","nonce":2494994479}]},"public_addr":"172.21.15.105:6818/2494994479","cluster_addr":"172.21.15.105:6821/2494994479","heartbeat_back_addr":"172.21.15.105:6828/2494994479","heartbeat_front_addr":"172.21.15.105:6825/2494994479","state":["exists","up"]},{"osd":2,"uuid":"81a5dc7a-9325-4ff5-941d-a2d5fba21f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6809","nonce":3798408699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6811","nonce":3798408699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6815","nonce":3798408699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6813","nonce":3798408699}]},"public_addr":"172.21.15.105:6809/3798408699","cluster_addr":"172.21.15.105:6811/3798408699","heartbeat_back_addr":"172.21.15.105:6815/3798408699","heartbeat_front_addr":"172.21.15.105:6813/3798408699","state":["exists","up"]},{"osd":3,"uuid":"df766ed2-5c71-4805-96f4-3d4970edc960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6819","nonce":1920720186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6824","nonce":1920720186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6831","nonce":1920720186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6827","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6829","nonce":1920720186}]},"public_addr":"172.21.15.105:6819/1920720186","cluster_addr":"172.21.15.105:6824/1920720186","heartbeat_back_addr":"172.21.15.105:6831/1920720186","heartbeat_front_addr":"172.21.15.105:6829/1920720186","state":["exists","up"]},{"osd":4,"uuid":"e1039e03-28b8-49df-b951-c91930b93c9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6810","nonce":3516494456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6813","nonce":3516494456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6820","nonce":3516494456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6817","nonce":3516494456}]},"public_addr":"172.21.15.154:6810/3516494456","cluster_addr":"172.21.15.154:6813/3516494456","heartbeat_back_addr":"172.21.15.154:6820/3516494456","heartbeat_front_addr":"172.21.15.154:6817/3516494456","state":["exists","up"]},{"osd":5,"uuid":"a9f5eaf1-66ff-405a-a9e4-ec3340812a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6801","nonce":540367751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6803","nonce":540367751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6807","nonce":540367751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6805","nonce":540367751}]},"public_addr":"172.21.15.154:6801/540367751","cluster_addr":"172.21.15.154:6803/540367751","heartbeat_back_addr":"172.21.15.154:6807/540367751","heartbeat_front_addr":"172.21.15.154:6805/540367751","state":["exists","up"]},{"osd":6,"uuid":"267d02ea-675f-428f-aae8-e0375e4d4feb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6825","nonce":1550829005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6827","nonce":1550829005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6831","nonce":1550829005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6829","nonce":1550829005}]},"public_addr":"172.21.15.154:6825/1550829005","cluster_addr":"172.21.15.154:6827/1550829005","heartbeat_back_addr":"172.21.15.154:6831/1550829005","heartbeat_front_addr":"172.21.15.154:6829/1550829005","state":["exists","up"]},{"osd":7,"uuid":"759ac6cc-896a-423c-885f-9fe90d8dcb14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6809","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6811","nonce":3920089903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6816","nonce":3920089903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6823","nonce":3920089903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6819","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6821","nonce":3920089903}]},"public_addr":"172.21.15.154:6811/3920089903","cluster_addr":"172.21.15.154:6816/3920089903","heartbeat_back_addr":"172.21.15.154:6823/3920089903","heartbeat_front_addr":"172.21.15.154:6821/3920089903","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.949509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.984072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.907278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.947391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.975447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.998896+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:09.028739+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.894119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:20:03.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.772+0000 7f0299674640 1 -- 172.21.15.105:0/2122928502 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f027406c920 msgr2=0x7f027406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:03.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.772+0000 7f0299674640 1 --2- 172.21.15.105:0/2122928502 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f027406c920 0x7f027406ede0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f027c034000 tx=0x7f027c007000 comp rx=0 tx=0).stop 2024-04-15T15:20:03.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.772+0000 7f0299674640 1 -- 172.21.15.105:0/2122928502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0280003f30 msgr2=0x7f02940b4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:03.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.772+0000 7f0299674640 1 --2- 172.21.15.105:0/2122928502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0280003f30 0x7f02940b4830 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f02840118e0 tx=0x7f0284011da0 comp rx=0 tx=0).stop 2024-04-15T15:20:03.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.772+0000 7f0299674640 1 -- 172.21.15.105:0/2122928502 shutdown_connections 2024-04-15T15:20:03.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.772+0000 7f0299674640 1 --2- 172.21.15.105:0/2122928502 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f02940d43a0 0x7f02940d4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:03.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.772+0000 7f0299674640 1 --2- 172.21.15.105:0/2122928502 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f027406c920 0x7f027406ede0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:03.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.772+0000 7f0299674640 1 --2- 172.21.15.105:0/2122928502 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f02941f4680 0x7f029414bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:03.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.772+0000 7f0299674640 1 --2- 172.21.15.105:0/2122928502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0280003f30 0x7f02940b4830 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:03.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.772+0000 7f0299674640 1 -- 172.21.15.105:0/2122928502 >> 172.21.15.105:0/2122928502 conn(0x7f0294067720 msgr2=0x7f029415d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:03.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.773+0000 7f0299674640 1 -- 172.21.15.105:0/2122928502 shutdown_connections 2024-04-15T15:20:03.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.773+0000 7f0299674640 1 -- 172.21.15.105:0/2122928502 wait complete. 2024-04-15T15:20:03.786 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:19:13.748585+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-15T15:19:23.789287+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-15T15:19:24.722250+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:20:03.786 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:20:03.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.889+0000 7f9410d7c640 1 Processor -- start 2024-04-15T15:20:03.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.889+0000 7f9410d7c640 1 -- start start 2024-04-15T15:20:03.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.890+0000 7f9410d7c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f940c072570 0x7f940c06de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:03.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.890+0000 7f9410d7c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f940c06e3c0 0x7f940c15d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:03.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.890+0000 7f9410d7c640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f940c06ebb0 con 0x7f940c145f00 2024-04-15T15:20:03.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.890+0000 7f9410d7c640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f940c1462e0 con 0x7f940c06e3c0 2024-04-15T15:20:03.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.890+0000 7f9410d7c640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f940c06ee90 con 0x7f940c072570 2024-04-15T15:20:03.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.891+0000 7f940ad9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f940c06e3c0 0x7f940c15d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:03.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.891+0000 7f940ad9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f940c06e3c0 0x7f940c15d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37456/0 (socket says 172.21.15.105:37456) 2024-04-15T15:20:03.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.891+0000 7f940ad9d640 1 -- 172.21.15.105:0/4274788587 learned_addr learned my addr 172.21.15.105:0/4274788587 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:03.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.891+0000 7f9409d9b640 1 --2- 172.21.15.105:0/4274788587 >> v2:172.21.15.154:3300/0 conn(0x7f940c072570 0x7f940c06de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:03.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.891+0000 7f940ad9d640 1 -- 172.21.15.105:0/4274788587 >> v2:172.21.15.154:3300/0 conn(0x7f940c072570 msgr2=0x7f940c06de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:03.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.891+0000 7f940ad9d640 1 --2- 172.21.15.105:0/4274788587 >> v2:172.21.15.154:3300/0 conn(0x7f940c072570 0x7f940c06de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:03.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.891+0000 7f940ad9d640 1 -- 172.21.15.105:0/4274788587 >> v1:172.21.15.154:6789/0 conn(0x7f940c145f00 legacy=0x7f940c071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:03.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.891+0000 7f940ad9d640 1 -- 172.21.15.105:0/4274788587 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f940c15cde0 con 0x7f940c06e3c0 2024-04-15T15:20:03.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.891+0000 7f9409d9b640 1 --2- 172.21.15.105:0/4274788587 >> v2:172.21.15.154:3300/0 conn(0x7f940c072570 0x7f940c06de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:03.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.892+0000 7f940ad9d640 1 --2- 172.21.15.105:0/4274788587 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f940c06e3c0 0x7f940c15d5e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f940c0a94e0 tx=0x7f940402bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0b305600f825fc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:03.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.892+0000 7f940959a640 1 -- 172.21.15.105:0/4274788587 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940402c960 con 0x7f940c06e3c0 2024-04-15T15:20:03.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.892+0000 7f940959a640 1 -- 172.21.15.105:0/4274788587 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f940402cb00 con 0x7f940c06e3c0 2024-04-15T15:20:03.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.892+0000 7f940959a640 1 -- 172.21.15.105:0/4274788587 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94040343f0 con 0x7f940c06e3c0 2024-04-15T15:20:03.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.892+0000 7f9410d7c640 1 -- 172.21.15.105:0/4274788587 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f940c06e3c0 msgr2=0x7f940c15d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:03.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.892+0000 7f9410d7c640 1 --2- 172.21.15.105:0/4274788587 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f940c06e3c0 0x7f940c15d5e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f940c0a94e0 tx=0x7f940402bdc0 comp rx=0 tx=0).stop 2024-04-15T15:20:03.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.892+0000 7f9410d7c640 1 -- 172.21.15.105:0/4274788587 shutdown_connections 2024-04-15T15:20:03.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.892+0000 7f9410d7c640 1 --2- 172.21.15.105:0/4274788587 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f940c06e3c0 0x7f940c15d5e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:03.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.893+0000 7f9410d7c640 1 --2- 172.21.15.105:0/4274788587 >> v2:172.21.15.154:3300/0 conn(0x7f940c072570 0x7f940c06de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:03.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.893+0000 7f9410d7c640 1 -- 172.21.15.105:0/4274788587 >> 172.21.15.105:0/4274788587 conn(0x7f940c067d50 msgr2=0x7f940c06b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:03.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.893+0000 7f9410d7c640 1 -- 172.21.15.105:0/4274788587 shutdown_connections 2024-04-15T15:20:03.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.893+0000 7f9410d7c640 1 -- 172.21.15.105:0/4274788587 wait complete. 2024-04-15T15:20:03.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.894+0000 7f9410d7c640 1 Processor -- start 2024-04-15T15:20:03.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.894+0000 7f9410d7c640 1 -- start start 2024-04-15T15:20:03.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.895+0000 7f9410d7c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f940c06e3c0 0x7f940c13dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:03.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.895+0000 7f9410d7c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f940c072570 0x7f940c0c6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:03.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.895+0000 7f9410d7c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f940c145f00 0x7f940c0c6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:03.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.895+0000 7f9410d7c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f940c15db20 con 0x7f940c145f00 2024-04-15T15:20:03.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.895+0000 7f9410d7c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f940c06e830 con 0x7f940c072570 2024-04-15T15:20:03.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.895+0000 7f9410d7c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f940c15dca0 con 0x7f940c06e3c0 2024-04-15T15:20:03.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.895+0000 7f940ad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f940c145f00 0x7f940c0c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:03.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.895+0000 7f940ad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f940c145f00 0x7f940c0c6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37470/0 (socket says 172.21.15.105:37470) 2024-04-15T15:20:03.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.895+0000 7f940ad9d640 1 -- 172.21.15.105:0/3234326576 learned_addr learned my addr 172.21.15.105:0/3234326576 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:03.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.895+0000 7f940a59c640 1 --2- 172.21.15.105:0/3234326576 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f940c06e3c0 0x7f940c13dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:03.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.895+0000 7f9409d9b640 1 --2- 172.21.15.105:0/3234326576 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f940c072570 0x7f940c0c6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:03.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.895+0000 7f940ad9d640 1 -- 172.21.15.105:0/3234326576 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f940c06e3c0 msgr2=0x7f940c13dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:03.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.895+0000 7f940ad9d640 1 --2- 172.21.15.105:0/3234326576 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f940c06e3c0 0x7f940c13dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:03.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.895+0000 7f940ad9d640 1 -- 172.21.15.105:0/3234326576 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f940c072570 msgr2=0x7f940c0c6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:03.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.895+0000 7f940ad9d640 1 --2- 172.21.15.105:0/3234326576 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f940c072570 0x7f940c0c6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:03.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.895+0000 7f940ad9d640 1 -- 172.21.15.105:0/3234326576 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f940c1e59c0 con 0x7f940c145f00 2024-04-15T15:20:03.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.895+0000 7f9409d9b640 1 --2- 172.21.15.105:0/3234326576 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f940c072570 0x7f940c0c6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:20:03.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.895+0000 7f940ad9d640 1 --2- 172.21.15.105:0/3234326576 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f940c145f00 0x7f940c0c6770 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9404009f40 tx=0x7f9404004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:03.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.895+0000 7f9402ffd640 1 -- 172.21.15.105:0/3234326576 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9404043020 con 0x7f940c145f00 2024-04-15T15:20:03.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.895+0000 7f9402ffd640 1 -- 172.21.15.105:0/3234326576 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f940403e450 con 0x7f940c145f00 2024-04-15T15:20:03.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.895+0000 7f9410d7c640 1 -- 172.21.15.105:0/3234326576 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f940c1e5ca0 con 0x7f940c145f00 2024-04-15T15:20:03.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.895+0000 7f9402ffd640 1 -- 172.21.15.105:0/3234326576 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940403e820 con 0x7f940c145f00 2024-04-15T15:20:03.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.895+0000 7f9410d7c640 1 -- 172.21.15.105:0/3234326576 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f940c1e6260 con 0x7f940c145f00 2024-04-15T15:20:03.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.896+0000 7f9410d7c640 1 -- 172.21.15.105:0/3234326576 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f940c068ab0 con 0x7f940c145f00 2024-04-15T15:20:03.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.897+0000 7f9402ffd640 1 -- 172.21.15.105:0/3234326576 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9404046020 con 0x7f940c145f00 2024-04-15T15:20:03.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.897+0000 7f9402ffd640 1 --2- 172.21.15.105:0/3234326576 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f93ec06c8f0 0x7f93ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:03.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.897+0000 7f9402ffd640 1 -- 172.21.15.105:0/3234326576 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f94040ae420 con 0x7f940c145f00 2024-04-15T15:20:03.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.897+0000 7f940a59c640 1 --2- 172.21.15.105:0/3234326576 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f93ec06c8f0 0x7f93ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:03.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.898+0000 7f940a59c640 1 --2- 172.21.15.105:0/3234326576 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f93ec06c8f0 0x7f93ec06edb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f93f4006fd0 tx=0x7f93f4009060 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:03.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:03.901+0000 7f9402ffd640 1 -- 172.21.15.105:0/3234326576 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f940404d030 con 0x7f940c145f00 2024-04-15T15:20:04.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.076+0000 7f9410d7c640 1 -- 172.21.15.105:0/3234326576 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f940c072fb0 con 0x7f940c145f00 2024-04-15T15:20:04.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.076+0000 7f9402ffd640 1 -- 172.21.15.105:0/3234326576 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f940407c8e0 con 0x7f940c145f00 2024-04-15T15:20:04.078 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-04-15T15:20:04.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.078+0000 7f9410d7c640 1 -- 172.21.15.105:0/3234326576 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f93ec06c8f0 msgr2=0x7f93ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:04.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.078+0000 7f9410d7c640 1 --2- 172.21.15.105:0/3234326576 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f93ec06c8f0 0x7f93ec06edb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f93f4006fd0 tx=0x7f93f4009060 comp rx=0 tx=0).stop 2024-04-15T15:20:04.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.078+0000 7f9410d7c640 1 -- 172.21.15.105:0/3234326576 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f940c145f00 msgr2=0x7f940c0c6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:04.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.078+0000 7f9410d7c640 1 --2- 172.21.15.105:0/3234326576 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f940c145f00 0x7f940c0c6770 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9404009f40 tx=0x7f9404004a40 comp rx=0 tx=0).stop 2024-04-15T15:20:04.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.078+0000 7f9410d7c640 1 -- 172.21.15.105:0/3234326576 shutdown_connections 2024-04-15T15:20:04.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.078+0000 7f9410d7c640 1 --2- 172.21.15.105:0/3234326576 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f93ec06c8f0 0x7f93ec06edb0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:04.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.078+0000 7f9410d7c640 1 --2- 172.21.15.105:0/3234326576 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f940c145f00 0x7f940c0c6770 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:04.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.078+0000 7f9410d7c640 1 --2- 172.21.15.105:0/3234326576 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f940c072570 0x7f940c0c6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:04.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.078+0000 7f9410d7c640 1 --2- 172.21.15.105:0/3234326576 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f940c06e3c0 0x7f940c13dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:04.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.078+0000 7f9410d7c640 1 -- 172.21.15.105:0/3234326576 >> 172.21.15.105:0/3234326576 conn(0x7f940c067d50 msgr2=0x7f940c15e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:04.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.078+0000 7f9410d7c640 1 -- 172.21.15.105:0/3234326576 shutdown_connections 2024-04-15T15:20:04.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.078+0000 7f9410d7c640 1 -- 172.21.15.105:0/3234326576 wait complete. 2024-04-15T15:20:04.091 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-15T15:20:04.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.195+0000 7fd54aa28640 1 Processor -- start 2024-04-15T15:20:04.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.195+0000 7fd54aa28640 1 -- start start 2024-04-15T15:20:04.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.196+0000 7fd54aa28640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd54406f860 0x7fd54406fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:04.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.196+0000 7fd54aa28640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd5440b3900 0x7fd5440b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:04.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.196+0000 7fd54aa28640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd54406c790 con 0x7fd544144e80 2024-04-15T15:20:04.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.196+0000 7fd54aa28640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fd5440b3470 con 0x7fd5440b3900 2024-04-15T15:20:04.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.196+0000 7fd54aa28640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fd5440cf7b0 con 0x7fd54406f860 2024-04-15T15:20:04.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.196+0000 7fd548fc5640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd5440b3900 0x7fd5440b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:04.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.196+0000 7fd548fc5640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd5440b3900 0x7fd5440b3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37486/0 (socket says 172.21.15.105:37486) 2024-04-15T15:20:04.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.196+0000 7fd543fff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fd544144e80 0x7fd54406dec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:55134/0 (socket says 172.21.15.105:55134) 2024-04-15T15:20:04.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.196+0000 7fd548fc5640 1 -- 172.21.15.105:0/1887876680 learned_addr learned my addr 172.21.15.105:0/1887876680 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:04.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.196+0000 7fd5437fe640 1 --2- 172.21.15.105:0/1887876680 >> v2:172.21.15.154:3300/0 conn(0x7fd54406f860 0x7fd54406fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:04.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.196+0000 7fd548fc5640 1 -- 172.21.15.105:0/1887876680 >> v2:172.21.15.154:3300/0 conn(0x7fd54406f860 msgr2=0x7fd54406fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:04.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.196+0000 7fd548fc5640 1 --2- 172.21.15.105:0/1887876680 >> v2:172.21.15.154:3300/0 conn(0x7fd54406f860 0x7fd54406fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:04.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.196+0000 7fd548fc5640 1 -- 172.21.15.105:0/1887876680 >> v1:172.21.15.105:6789/0 conn(0x7fd544144e80 legacy=0x7fd54406dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:04.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.196+0000 7fd548fc5640 1 -- 172.21.15.105:0/1887876680 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5440cfa90 con 0x7fd5440b3900 2024-04-15T15:20:04.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.196+0000 7fd542ffd640 1 -- 172.21.15.105:0/1887876680 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1991811904 0 0) 0x7fd54406c790 con 0x7fd544144e80 2024-04-15T15:20:04.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.197+0000 7fd548fc5640 1 --2- 172.21.15.105:0/1887876680 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5440b3900 0x7fd5440b3d30 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd5440cdf50 tx=0x7fd53c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7f714e19d1f3e2b server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:04.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.197+0000 7fd542ffd640 1 -- 172.21.15.105:0/1887876680 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd53c02ca80 con 0x7fd5440b3900 2024-04-15T15:20:04.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.197+0000 7fd542ffd640 1 -- 172.21.15.105:0/1887876680 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd53c02cc20 con 0x7fd5440b3900 2024-04-15T15:20:04.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.197+0000 7fd542ffd640 1 -- 172.21.15.105:0/1887876680 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd53c0343f0 con 0x7fd5440b3900 2024-04-15T15:20:04.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.198+0000 7fd54aa28640 1 -- 172.21.15.105:0/1887876680 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5440b3900 msgr2=0x7fd5440b3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:04.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.198+0000 7fd54aa28640 1 --2- 172.21.15.105:0/1887876680 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5440b3900 0x7fd5440b3d30 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd5440cdf50 tx=0x7fd53c02bee0 comp rx=0 tx=0).stop 2024-04-15T15:20:04.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.198+0000 7fd54aa28640 1 -- 172.21.15.105:0/1887876680 shutdown_connections 2024-04-15T15:20:04.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.198+0000 7fd54aa28640 1 --2- 172.21.15.105:0/1887876680 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5440b3900 0x7fd5440b3d30 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:04.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.198+0000 7fd54aa28640 1 --2- 172.21.15.105:0/1887876680 >> v2:172.21.15.154:3300/0 conn(0x7fd54406f860 0x7fd54406fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:04.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.198+0000 7fd54aa28640 1 -- 172.21.15.105:0/1887876680 >> 172.21.15.105:0/1887876680 conn(0x7fd54406b9c0 msgr2=0x7fd54406bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:04.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.198+0000 7fd54aa28640 1 -- 172.21.15.105:0/1887876680 shutdown_connections 2024-04-15T15:20:04.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.198+0000 7fd54aa28640 1 -- 172.21.15.105:0/1887876680 wait complete. 2024-04-15T15:20:04.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.199+0000 7fd54aa28640 1 Processor -- start 2024-04-15T15:20:04.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.199+0000 7fd54aa28640 1 -- start start 2024-04-15T15:20:04.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.200+0000 7fd54aa28640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd54406f860 0x7fd544138fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:04.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.200+0000 7fd54aa28640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd5440b3900 0x7fd54413c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:04.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.200+0000 7fd54aa28640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd544144e80 0x7fd5440affa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:04.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.200+0000 7fd54aa28640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd544145260 con 0x7fd544144e80 2024-04-15T15:20:04.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.200+0000 7fd54aa28640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd5440d0050 con 0x7fd54406f860 2024-04-15T15:20:04.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.200+0000 7fd54aa28640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fd5441453e0 con 0x7fd5440b3900 2024-04-15T15:20:04.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.200+0000 7fd548fc5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd544144e80 0x7fd5440affa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:04.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.200+0000 7fd548fc5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd544144e80 0x7fd5440affa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37498/0 (socket says 172.21.15.105:37498) 2024-04-15T15:20:04.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.200+0000 7fd548fc5640 1 -- 172.21.15.105:0/1121539894 learned_addr learned my addr 172.21.15.105:0/1121539894 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:04.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.200+0000 7fd5437fe640 1 --2- 172.21.15.105:0/1121539894 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd5440b3900 0x7fd54413c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:04.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.200+0000 7fd543fff640 1 --2- 172.21.15.105:0/1121539894 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd54406f860 0x7fd544138fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:04.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.200+0000 7fd548fc5640 1 -- 172.21.15.105:0/1121539894 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd5440b3900 msgr2=0x7fd54413c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:04.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.200+0000 7fd548fc5640 1 --2- 172.21.15.105:0/1121539894 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd5440b3900 0x7fd54413c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:04.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.200+0000 7fd548fc5640 1 -- 172.21.15.105:0/1121539894 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd54406f860 msgr2=0x7fd544138fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:04.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.200+0000 7fd548fc5640 1 --2- 172.21.15.105:0/1121539894 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd54406f860 0x7fd544138fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:04.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.200+0000 7fd548fc5640 1 -- 172.21.15.105:0/1121539894 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd54413abf0 con 0x7fd544144e80 2024-04-15T15:20:04.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.200+0000 7fd5437fe640 1 --2- 172.21.15.105:0/1121539894 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd5440b3900 0x7fd54413c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:04.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.200+0000 7fd543fff640 1 --2- 172.21.15.105:0/1121539894 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd54406f860 0x7fd544138fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:04.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.200+0000 7fd548fc5640 1 --2- 172.21.15.105:0/1121539894 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd544144e80 0x7fd5440affa0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd53c036040 tx=0x7fd53c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:04.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.201+0000 7fd540ff9640 1 -- 172.21.15.105:0/1121539894 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd53c043030 con 0x7fd544144e80 2024-04-15T15:20:04.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.201+0000 7fd540ff9640 1 -- 172.21.15.105:0/1121539894 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd53c003990 con 0x7fd544144e80 2024-04-15T15:20:04.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.201+0000 7fd54aa28640 1 -- 172.21.15.105:0/1121539894 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd54413aed0 con 0x7fd544144e80 2024-04-15T15:20:04.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.201+0000 7fd540ff9640 1 -- 172.21.15.105:0/1121539894 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd53c009510 con 0x7fd544144e80 2024-04-15T15:20:04.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.201+0000 7fd54aa28640 1 -- 172.21.15.105:0/1121539894 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd54413b490 con 0x7fd544144e80 2024-04-15T15:20:04.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.202+0000 7fd54aa28640 1 -- 172.21.15.105:0/1121539894 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd544068a40 con 0x7fd544144e80 2024-04-15T15:20:04.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.202+0000 7fd540ff9640 1 -- 172.21.15.105:0/1121539894 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd53c02ca90 con 0x7fd544144e80 2024-04-15T15:20:04.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.203+0000 7fd540ff9640 1 --2- 172.21.15.105:0/1121539894 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fd53006c920 0x7fd53006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:04.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.203+0000 7fd540ff9640 1 -- 172.21.15.105:0/1121539894 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd53c0ae040 con 0x7fd544144e80 2024-04-15T15:20:04.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.203+0000 7fd543fff640 1 --2- 172.21.15.105:0/1121539894 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fd53006c920 0x7fd53006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:04.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.203+0000 7fd543fff640 1 --2- 172.21.15.105:0/1121539894 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fd53006c920 0x7fd53006ede0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd534000f60 tx=0x7fd534009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:04.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.207+0000 7fd540ff9640 1 -- 172.21.15.105:0/1121539894 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd53c07c500 con 0x7fd544144e80 2024-04-15T15:20:04.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.382+0000 7fd54aa28640 1 -- 172.21.15.105:0/1121539894 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd5440ce6a0 con 0x7fd544144e80 2024-04-15T15:20:04.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.382+0000 7fd540ff9640 1 -- 172.21.15.105:0/1121539894 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fd53c07be00 con 0x7fd544144e80 2024-04-15T15:20:04.384 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-04-15T15:20:04.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.384+0000 7fd54aa28640 1 -- 172.21.15.105:0/1121539894 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fd53006c920 msgr2=0x7fd53006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:04.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.384+0000 7fd54aa28640 1 --2- 172.21.15.105:0/1121539894 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fd53006c920 0x7fd53006ede0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd534000f60 tx=0x7fd534009040 comp rx=0 tx=0).stop 2024-04-15T15:20:04.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.384+0000 7fd54aa28640 1 -- 172.21.15.105:0/1121539894 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd544144e80 msgr2=0x7fd5440affa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:04.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.384+0000 7fd54aa28640 1 --2- 172.21.15.105:0/1121539894 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd544144e80 0x7fd5440affa0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd53c036040 tx=0x7fd53c004a40 comp rx=0 tx=0).stop 2024-04-15T15:20:04.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.384+0000 7fd54aa28640 1 -- 172.21.15.105:0/1121539894 shutdown_connections 2024-04-15T15:20:04.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.384+0000 7fd54aa28640 1 --2- 172.21.15.105:0/1121539894 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd5440b3900 0x7fd54413c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:04.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.384+0000 7fd54aa28640 1 --2- 172.21.15.105:0/1121539894 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fd53006c920 0x7fd53006ede0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:04.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.384+0000 7fd54aa28640 1 --2- 172.21.15.105:0/1121539894 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd544144e80 0x7fd5440affa0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:04.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.384+0000 7fd54aa28640 1 --2- 172.21.15.105:0/1121539894 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd54406f860 0x7fd544138fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:04.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.384+0000 7fd54aa28640 1 -- 172.21.15.105:0/1121539894 >> 172.21.15.105:0/1121539894 conn(0x7fd54406b9c0 msgr2=0x7fd5440b3ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:04.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.384+0000 7fd54aa28640 1 -- 172.21.15.105:0/1121539894 shutdown_connections 2024-04-15T15:20:04.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.384+0000 7fd54aa28640 1 -- 172.21.15.105:0/1121539894 wait complete. 2024-04-15T15:20:04.397 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-15T15:20:04.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.502+0000 7f4e84675640 1 Processor -- start 2024-04-15T15:20:04.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.502+0000 7f4e84675640 1 -- start start 2024-04-15T15:20:04.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.503+0000 7f4e84675640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4e7c162040 0x7f4e7c06d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:04.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.503+0000 7f4e84675640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4e7c0ce430 0x7f4e7c06d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:04.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.503+0000 7f4e84675640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e7c0ccc40 con 0x7f4e7c09aa40 2024-04-15T15:20:04.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.503+0000 7f4e84675640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f4e7c09af40 con 0x7f4e7c162040 2024-04-15T15:20:04.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.503+0000 7f4e84675640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f4e7c09b220 con 0x7f4e7c0ce430 2024-04-15T15:20:04.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.503+0000 7f4e81c10640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f4e7c09aa40 0x7f4e7c0d06f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:55136/0 (socket says 172.21.15.105:55136) 2024-04-15T15:20:04.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.503+0000 7f4e81c10640 1 -- 172.21.15.105:0/4153968551 learned_addr learned my addr 172.21.15.105:0/4153968551 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:20:04.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.504+0000 7f4e82c12640 1 --2- 172.21.15.105:0/4153968551 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4e7c0ce430 0x7f4e7c06d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:04.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.504+0000 7f4e8140f640 1 -- 172.21.15.105:0/4153968551 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2685837358 0 0) 0x7f4e7c0ccc40 con 0x7f4e7c09aa40 2024-04-15T15:20:04.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.504+0000 7f4e82411640 1 --2- 172.21.15.105:0/4153968551 >> v2:172.21.15.154:3300/0 conn(0x7f4e7c162040 0x7f4e7c06d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:04.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.504+0000 7f4e8140f640 1 -- 172.21.15.105:0/4153968551 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4e68003520 con 0x7f4e7c09aa40 2024-04-15T15:20:04.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.504+0000 7f4e8140f640 1 -- 172.21.15.105:0/4153968551 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1710236950 0 0) 0x7f4e68003520 con 0x7f4e7c09aa40 2024-04-15T15:20:04.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.504+0000 7f4e8140f640 1 -- 172.21.15.105:0/4153968551 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4e7c0ce430 msgr2=0x7f4e7c06d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:04.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.504+0000 7f4e8140f640 1 --2- 172.21.15.105:0/4153968551 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4e7c0ce430 0x7f4e7c06d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:04.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.504+0000 7f4e8140f640 1 -- 172.21.15.105:0/4153968551 >> v2:172.21.15.154:3300/0 conn(0x7f4e7c162040 msgr2=0x7f4e7c06d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:04.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.504+0000 7f4e8140f640 1 --2- 172.21.15.105:0/4153968551 >> v2:172.21.15.154:3300/0 conn(0x7f4e7c162040 0x7f4e7c06d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:04.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.504+0000 7f4e8140f640 1 -- 172.21.15.105:0/4153968551 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e7c09b500 con 0x7f4e7c09aa40 2024-04-15T15:20:04.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.504+0000 7f4e8140f640 1 -- 172.21.15.105:0/4153968551 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7f4e6c001670 con 0x7f4e7c09aa40 2024-04-15T15:20:04.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.504+0000 7f4e82411640 1 --2- 172.21.15.105:0/4153968551 >> v2:172.21.15.154:3300/0 conn(0x7f4e7c162040 0x7f4e7c06d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:20:04.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.504+0000 7f4e8140f640 1 -- 172.21.15.105:0/4153968551 >> v1:172.21.15.105:6789/0 conn(0x7f4e7c09aa40 legacy=0x7f4e7c0d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:04.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.504+0000 7f4e8140f640 1 --2- 172.21.15.105:0/4153968551 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4e7c0ce430 0x7f4e680039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:04.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.504+0000 7f4e8140f640 1 --2- 172.21.15.105:0/4153968551 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e68003f30 0x7f4e68004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:04.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.504+0000 7f4e8140f640 1 --2- 172.21.15.105:0/4153968551 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e68004d40 0x7f4e680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:04.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.504+0000 7f4e8140f640 1 -- 172.21.15.105:0/4153968551 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7c09af40 con 0x7f4e68003f30 2024-04-15T15:20:04.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.504+0000 7f4e8140f640 1 -- 172.21.15.105:0/4153968551 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7c09b220 con 0x7f4e68004d40 2024-04-15T15:20:04.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.504+0000 7f4e8140f640 1 -- 172.21.15.105:0/4153968551 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f4e6800ba10 con 0x7f4e7c0ce430 2024-04-15T15:20:04.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.504+0000 7f4e81c10640 1 --2- 172.21.15.105:0/4153968551 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e68003f30 0x7f4e68004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:04.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.504+0000 7f4e82411640 1 --2- 172.21.15.105:0/4153968551 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4e7c0ce430 0x7f4e680039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:04.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.504+0000 7f4e82c12640 1 --2- 172.21.15.105:0/4153968551 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e68004d40 0x7f4e680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:04.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.504+0000 7f4e81c10640 1 -- 172.21.15.105:0/4153968551 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4e7c0ce430 msgr2=0x7f4e680039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:04.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.504+0000 7f4e81c10640 1 --2- 172.21.15.105:0/4153968551 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4e7c0ce430 0x7f4e680039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:04.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.504+0000 7f4e81c10640 1 -- 172.21.15.105:0/4153968551 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e68004d40 msgr2=0x7f4e680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:04.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.504+0000 7f4e81c10640 1 --2- 172.21.15.105:0/4153968551 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e68004d40 0x7f4e680051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:04.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.504+0000 7f4e81c10640 1 -- 172.21.15.105:0/4153968551 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e68003520 con 0x7f4e68003f30 2024-04-15T15:20:04.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.505+0000 7f4e81c10640 1 --2- 172.21.15.105:0/4153968551 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e68003f30 0x7f4e68004800 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c001050 tx=0x7f4e68005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbfca0fb9d6a4add server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:04.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.505+0000 7f4e8140f640 1 -- 172.21.15.105:0/4153968551 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c007d70 con 0x7f4e68003f30 2024-04-15T15:20:04.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.505+0000 7f4e8140f640 1 -- 172.21.15.105:0/4153968551 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e6c030070 con 0x7f4e68003f30 2024-04-15T15:20:04.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.505+0000 7f4e8140f640 1 -- 172.21.15.105:0/4153968551 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c02cd80 con 0x7f4e68003f30 2024-04-15T15:20:04.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.506+0000 7f4e84675640 1 -- 172.21.15.105:0/4153968551 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e68003f30 msgr2=0x7f4e68004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:04.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.506+0000 7f4e84675640 1 --2- 172.21.15.105:0/4153968551 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e68003f30 0x7f4e68004800 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c001050 tx=0x7f4e68005830 comp rx=0 tx=0).stop 2024-04-15T15:20:04.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.506+0000 7f4e82c12640 1 -- 172.21.15.105:0/4153968551 reap_dead start 2024-04-15T15:20:04.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.506+0000 7f4e84675640 1 -- 172.21.15.105:0/4153968551 shutdown_connections 2024-04-15T15:20:04.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.506+0000 7f4e84675640 1 -- 172.21.15.105:0/4153968551 >> 172.21.15.105:0/4153968551 conn(0x7f4e7c0d25f0 msgr2=0x7f4e7c0d2a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:04.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.506+0000 7f4e84675640 1 -- 172.21.15.105:0/4153968551 shutdown_connections 2024-04-15T15:20:04.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.506+0000 7f4e84675640 1 -- 172.21.15.105:0/4153968551 wait complete. 2024-04-15T15:20:04.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.507+0000 7f4e84675640 1 Processor -- start 2024-04-15T15:20:04.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.507+0000 7f4e84675640 1 -- start start 2024-04-15T15:20:04.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.508+0000 7f4e84675640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4e68003f30 0x7f4e7c152960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:04.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.508+0000 7f4e84675640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e7c160320 0x7f4e7c0c6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:04.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.508+0000 7f4e84675640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e7c15d3b0 0x7f4e7c15d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:04.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.508+0000 7f4e84675640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7c0d3390 con 0x7f4e7c160320 2024-04-15T15:20:04.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.508+0000 7f4e84675640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7c162420 con 0x7f4e7c15d3b0 2024-04-15T15:20:04.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.508+0000 7f4e84675640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f4e7c0d3510 con 0x7f4e68003f30 2024-04-15T15:20:04.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.508+0000 7f4e81c10640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e7c160320 0x7f4e7c0c6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:04.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.508+0000 7f4e81c10640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e7c160320 0x7f4e7c0c6b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37514/0 (socket says 172.21.15.105:37514) 2024-04-15T15:20:04.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.508+0000 7f4e81c10640 1 -- 172.21.15.105:0/3186717356 learned_addr learned my addr 172.21.15.105:0/3186717356 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:04.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.508+0000 7f4e81c10640 1 -- 172.21.15.105:0/3186717356 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4e68003f30 msgr2=0x7f4e7c152960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:04.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.508+0000 7f4e82411640 1 --2- 172.21.15.105:0/3186717356 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4e68003f30 0x7f4e7c152960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:04.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.508+0000 7f4e82c12640 1 --2- 172.21.15.105:0/3186717356 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e7c15d3b0 0x7f4e7c15d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:04.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.508+0000 7f4e81c10640 1 --2- 172.21.15.105:0/3186717356 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4e68003f30 0x7f4e7c152960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:04.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.508+0000 7f4e81c10640 1 -- 172.21.15.105:0/3186717356 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e7c15d3b0 msgr2=0x7f4e7c15d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:04.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.508+0000 7f4e81c10640 1 --2- 172.21.15.105:0/3186717356 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e7c15d3b0 0x7f4e7c15d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:04.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.508+0000 7f4e81c10640 1 -- 172.21.15.105:0/3186717356 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e7c0c8520 con 0x7f4e7c160320 2024-04-15T15:20:04.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.508+0000 7f4e82411640 1 --2- 172.21.15.105:0/3186717356 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4e68003f30 0x7f4e7c152960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:04.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.508+0000 7f4e82c12640 1 --2- 172.21.15.105:0/3186717356 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e7c15d3b0 0x7f4e7c15d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:04.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.508+0000 7f4e81c10640 1 --2- 172.21.15.105:0/3186717356 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e7c160320 0x7f4e7c0c6b10 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c02c590 tx=0x7f4e6c000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:04.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.508+0000 7f4e72ffd640 1 -- 172.21.15.105:0/3186717356 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c045070 con 0x7f4e7c160320 2024-04-15T15:20:04.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.508+0000 7f4e72ffd640 1 -- 172.21.15.105:0/3186717356 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e6c004bf0 con 0x7f4e7c160320 2024-04-15T15:20:04.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.508+0000 7f4e84675640 1 -- 172.21.15.105:0/3186717356 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e7c0c7480 con 0x7f4e7c160320 2024-04-15T15:20:04.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.508+0000 7f4e72ffd640 1 -- 172.21.15.105:0/3186717356 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c007d50 con 0x7f4e7c160320 2024-04-15T15:20:04.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.509+0000 7f4e84675640 1 -- 172.21.15.105:0/3186717356 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e7c0c7a40 con 0x7f4e7c160320 2024-04-15T15:20:04.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.510+0000 7f4e84675640 1 -- 172.21.15.105:0/3186717356 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e7c068ab0 con 0x7f4e7c160320 2024-04-15T15:20:04.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.510+0000 7f4e72ffd640 1 -- 172.21.15.105:0/3186717356 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4e6c007070 con 0x7f4e7c160320 2024-04-15T15:20:04.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.510+0000 7f4e72ffd640 1 --2- 172.21.15.105:0/3186717356 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f4e6006c920 0x7f4e6006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:04.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.511+0000 7f4e72ffd640 1 -- 172.21.15.105:0/3186717356 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4e6c0accc0 con 0x7f4e7c160320 2024-04-15T15:20:04.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.511+0000 7f4e82411640 1 --2- 172.21.15.105:0/3186717356 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f4e6006c920 0x7f4e6006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:04.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.511+0000 7f4e82411640 1 --2- 172.21.15.105:0/3186717356 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f4e6006c920 0x7f4e6006ede0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c0b1c20 tx=0x7f4e7800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:04.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.515+0000 7f4e72ffd640 1 -- 172.21.15.105:0/3186717356 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4e6c07ab80 con 0x7f4e7c160320 2024-04-15T15:20:04.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.692+0000 7f4e84675640 1 -- 172.21.15.105:0/3186717356 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4e7c0d1b40 con 0x7f4e7c160320 2024-04-15T15:20:04.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.692+0000 7f4e72ffd640 1 -- 172.21.15.105:0/3186717356 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f4e6c07ee80 con 0x7f4e7c160320 2024-04-15T15:20:04.694 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-04-15T15:20:04.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.694+0000 7f4e84675640 1 -- 172.21.15.105:0/3186717356 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f4e6006c920 msgr2=0x7f4e6006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:04.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.694+0000 7f4e84675640 1 --2- 172.21.15.105:0/3186717356 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f4e6006c920 0x7f4e6006ede0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c0b1c20 tx=0x7f4e7800b040 comp rx=0 tx=0).stop 2024-04-15T15:20:04.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.694+0000 7f4e84675640 1 -- 172.21.15.105:0/3186717356 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e7c160320 msgr2=0x7f4e7c0c6b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:04.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.694+0000 7f4e84675640 1 --2- 172.21.15.105:0/3186717356 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e7c160320 0x7f4e7c0c6b10 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c02c590 tx=0x7f4e6c000fe0 comp rx=0 tx=0).stop 2024-04-15T15:20:04.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.694+0000 7f4e84675640 1 -- 172.21.15.105:0/3186717356 shutdown_connections 2024-04-15T15:20:04.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.694+0000 7f4e84675640 1 --2- 172.21.15.105:0/3186717356 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f4e6006c920 0x7f4e6006ede0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:04.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.694+0000 7f4e84675640 1 --2- 172.21.15.105:0/3186717356 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e7c15d3b0 0x7f4e7c15d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:04.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.694+0000 7f4e84675640 1 --2- 172.21.15.105:0/3186717356 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e7c160320 0x7f4e7c0c6b10 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:04.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.694+0000 7f4e84675640 1 --2- 172.21.15.105:0/3186717356 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4e68003f30 0x7f4e7c152960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:04.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.694+0000 7f4e84675640 1 -- 172.21.15.105:0/3186717356 >> 172.21.15.105:0/3186717356 conn(0x7f4e7c0d25f0 msgr2=0x7f4e7c0b28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:04.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.694+0000 7f4e84675640 1 -- 172.21.15.105:0/3186717356 shutdown_connections 2024-04-15T15:20:04.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.694+0000 7f4e84675640 1 -- 172.21.15.105:0/3186717356 wait complete. 2024-04-15T15:20:04.707 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-15T15:20:04.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.809+0000 7f677d967640 1 Processor -- start 2024-04-15T15:20:04.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.809+0000 7f677d967640 1 -- start start 2024-04-15T15:20:04.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.810+0000 7f677d967640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f67780ae410 0x7f677806b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:04.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.810+0000 7f677d967640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67780b2ca0 con 0x7f677815e9e0 2024-04-15T15:20:04.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.810+0000 7f677d967640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67780b6130 con 0x7f67780ae870 2024-04-15T15:20:04.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.810+0000 7f677d967640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f67780b5f00 con 0x7f67780ae410 2024-04-15T15:20:04.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.810+0000 7f6776ffd640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f677815e9e0 0x7f67780b4430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:55144/0 (socket says 172.21.15.105:55144) 2024-04-15T15:20:04.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.810+0000 7f6776ffd640 1 -- 172.21.15.105:0/3788960911 learned_addr learned my addr 172.21.15.105:0/3788960911 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:20:04.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.811+0000 7f6775ffb640 1 -- 172.21.15.105:0/3788960911 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2524463593 0 0) 0x7f67780b2ca0 con 0x7f677815e9e0 2024-04-15T15:20:04.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.811+0000 7f6775ffb640 1 -- 172.21.15.105:0/3788960911 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6764003520 con 0x7f677815e9e0 2024-04-15T15:20:04.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.811+0000 7f67767fc640 1 --2- 172.21.15.105:0/3788960911 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f67780ae410 0x7f677806b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:04.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.811+0000 7f6775ffb640 1 -- 172.21.15.105:0/3788960911 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3223283958 0 0) 0x7f6764003520 con 0x7f677815e9e0 2024-04-15T15:20:04.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.811+0000 7f6775ffb640 1 -- 172.21.15.105:0/3788960911 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f67780ae410 msgr2=0x7f677806b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:04.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.811+0000 7f6775ffb640 1 --2- 172.21.15.105:0/3788960911 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f67780ae410 0x7f677806b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:04.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.811+0000 7f6775ffb640 1 -- 172.21.15.105:0/3788960911 >> v1:172.21.15.154:6789/0 conn(0x7f67780ae870 legacy=0x7f67780b17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:04.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.811+0000 7f6775ffb640 1 -- 172.21.15.105:0/3788960911 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67780b6410 con 0x7f677815e9e0 2024-04-15T15:20:04.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.811+0000 7f6775ffb640 1 -- 172.21.15.105:0/3788960911 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7f67680019f0 con 0x7f677815e9e0 2024-04-15T15:20:04.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.811+0000 7f67767fc640 1 --2- 172.21.15.105:0/3788960911 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f67780ae410 0x7f677806b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:20:04.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.811+0000 7f6775ffb640 1 -- 172.21.15.105:0/3788960911 >> v1:172.21.15.105:6789/0 conn(0x7f677815e9e0 legacy=0x7f67780b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:04.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.811+0000 7f6775ffb640 1 --2- 172.21.15.105:0/3788960911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67640039e0 0x7f6764003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:04.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.811+0000 7f6775ffb640 1 --2- 172.21.15.105:0/3788960911 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6764004800 0x7f6764004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:04.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.811+0000 7f6775ffb640 1 --2- 172.21.15.105:0/3788960911 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f67780ae410 0x7f67640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:04.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.811+0000 7f6775ffb640 1 -- 172.21.15.105:0/3788960911 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f676400bac0 con 0x7f67640039e0 2024-04-15T15:20:04.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.811+0000 7f6775ffb640 1 -- 172.21.15.105:0/3788960911 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f676400bc70 con 0x7f6764004800 2024-04-15T15:20:04.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.811+0000 7f6775ffb640 1 -- 172.21.15.105:0/3788960911 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f676400be70 con 0x7f67780ae410 2024-04-15T15:20:04.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.811+0000 7f6776ffd640 1 --2- 172.21.15.105:0/3788960911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67640039e0 0x7f6764003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:04.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.811+0000 7f67767fc640 1 --2- 172.21.15.105:0/3788960911 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6764004800 0x7f6764004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:04.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.811+0000 7f67777fe640 1 --2- 172.21.15.105:0/3788960911 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f67780ae410 0x7f67640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:04.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.811+0000 7f6776ffd640 1 -- 172.21.15.105:0/3788960911 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f67780ae410 msgr2=0x7f67640051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:04.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.811+0000 7f6776ffd640 1 --2- 172.21.15.105:0/3788960911 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f67780ae410 0x7f67640051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:04.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.811+0000 7f6776ffd640 1 -- 172.21.15.105:0/3788960911 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6764004800 msgr2=0x7f6764004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:04.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.811+0000 7f6776ffd640 1 --2- 172.21.15.105:0/3788960911 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6764004800 0x7f6764004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:04.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.811+0000 7f6776ffd640 1 -- 172.21.15.105:0/3788960911 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6764003520 con 0x7f67640039e0 2024-04-15T15:20:04.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.811+0000 7f67767fc640 1 --2- 172.21.15.105:0/3788960911 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6764004800 0x7f6764004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:20:04.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.812+0000 7f6776ffd640 1 --2- 172.21.15.105:0/3788960911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67640039e0 0x7f6764003e00 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f676800a850 tx=0x7f6764005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1a21ed6eea8049c server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:04.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.812+0000 7f6775ffb640 1 -- 172.21.15.105:0/3788960911 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676800a4a0 con 0x7f67640039e0 2024-04-15T15:20:04.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.812+0000 7f6775ffb640 1 -- 172.21.15.105:0/3788960911 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6768034650 con 0x7f67640039e0 2024-04-15T15:20:04.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.812+0000 7f6775ffb640 1 -- 172.21.15.105:0/3788960911 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6768034a40 con 0x7f67640039e0 2024-04-15T15:20:04.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.813+0000 7f677d967640 1 -- 172.21.15.105:0/3788960911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67640039e0 msgr2=0x7f6764003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:04.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.813+0000 7f677d967640 1 --2- 172.21.15.105:0/3788960911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67640039e0 0x7f6764003e00 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f676800a850 tx=0x7f6764005830 comp rx=0 tx=0).stop 2024-04-15T15:20:04.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.813+0000 7f67777fe640 1 -- 172.21.15.105:0/3788960911 reap_dead start 2024-04-15T15:20:04.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.813+0000 7f677d967640 1 -- 172.21.15.105:0/3788960911 shutdown_connections 2024-04-15T15:20:04.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.813+0000 7f677d967640 1 -- 172.21.15.105:0/3788960911 >> 172.21.15.105:0/3788960911 conn(0x7f677809aa40 msgr2=0x7f67780b6950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:04.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.813+0000 7f677d967640 1 -- 172.21.15.105:0/3788960911 shutdown_connections 2024-04-15T15:20:04.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.813+0000 7f677d967640 1 -- 172.21.15.105:0/3788960911 wait complete. 2024-04-15T15:20:04.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.814+0000 7f677d967640 1 Processor -- start 2024-04-15T15:20:04.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.814+0000 7f677d967640 1 -- start start 2024-04-15T15:20:04.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.815+0000 7f677d967640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f67640039e0 0x7f677814c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:04.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.815+0000 7f677d967640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f677815d850 0x7f677815c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:04.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.815+0000 7f677d967640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67781578f0 0x7f6778157d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:04.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.815+0000 7f677d967640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f677815edc0 con 0x7f677815d850 2024-04-15T15:20:04.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.815+0000 7f677d967640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f67780b2f80 con 0x7f67781578f0 2024-04-15T15:20:04.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.815+0000 7f677d967640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f67780b72a0 con 0x7f67640039e0 2024-04-15T15:20:04.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.815+0000 7f67767fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f677815d850 0x7f677815c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:04.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.815+0000 7f67767fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f677815d850 0x7f677815c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37526/0 (socket says 172.21.15.105:37526) 2024-04-15T15:20:04.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.815+0000 7f67767fc640 1 -- 172.21.15.105:0/777533202 learned_addr learned my addr 172.21.15.105:0/777533202 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:04.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.815+0000 7f6776ffd640 1 --2- 172.21.15.105:0/777533202 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f67640039e0 0x7f677814c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:04.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.815+0000 7f67777fe640 1 --2- 172.21.15.105:0/777533202 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67781578f0 0x7f6778157d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:04.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.815+0000 7f67767fc640 1 -- 172.21.15.105:0/777533202 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f67640039e0 msgr2=0x7f677814c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:04.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.815+0000 7f67767fc640 1 --2- 172.21.15.105:0/777533202 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f67640039e0 0x7f677814c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:04.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.815+0000 7f67767fc640 1 -- 172.21.15.105:0/777533202 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67781578f0 msgr2=0x7f6778157d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:04.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.815+0000 7f67767fc640 1 --2- 172.21.15.105:0/777533202 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67781578f0 0x7f6778157d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:04.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.815+0000 7f67767fc640 1 -- 172.21.15.105:0/777533202 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6778154bf0 con 0x7f677815d850 2024-04-15T15:20:04.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.815+0000 7f67777fe640 1 --2- 172.21.15.105:0/777533202 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67781578f0 0x7f6778157d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:04.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.815+0000 7f67767fc640 1 --2- 172.21.15.105:0/777533202 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f677815d850 0x7f677815c9e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f67600108c0 tx=0x7f6760010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:04.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.816+0000 7f676ffff640 1 -- 172.21.15.105:0/777533202 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676001a020 con 0x7f677815d850 2024-04-15T15:20:04.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.816+0000 7f676ffff640 1 -- 172.21.15.105:0/777533202 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f676000eb60 con 0x7f677815d850 2024-04-15T15:20:04.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.816+0000 7f677d967640 1 -- 172.21.15.105:0/777533202 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6778154ed0 con 0x7f677815d850 2024-04-15T15:20:04.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.816+0000 7f676ffff640 1 -- 172.21.15.105:0/777533202 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676000f040 con 0x7f677815d850 2024-04-15T15:20:04.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.816+0000 7f677d967640 1 -- 172.21.15.105:0/777533202 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6778155490 con 0x7f677815d850 2024-04-15T15:20:04.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.817+0000 7f677d967640 1 -- 172.21.15.105:0/777533202 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6778159c20 con 0x7f677815d850 2024-04-15T15:20:04.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.817+0000 7f676ffff640 1 -- 172.21.15.105:0/777533202 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f676001d020 con 0x7f677815d850 2024-04-15T15:20:04.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.817+0000 7f676ffff640 1 --2- 172.21.15.105:0/777533202 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f675806c8f0 0x7f675806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:04.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.818+0000 7f676ffff640 1 -- 172.21.15.105:0/777533202 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f676008c340 con 0x7f677815d850 2024-04-15T15:20:04.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.818+0000 7f6776ffd640 1 --2- 172.21.15.105:0/777533202 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f675806c8f0 0x7f675806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:04.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.818+0000 7f6776ffd640 1 --2- 172.21.15.105:0/777533202 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f675806c8f0 0x7f675806edb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6768034000 tx=0x7f676802f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:04.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:04.822+0000 7f676ffff640 1 -- 172.21.15.105:0/777533202 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6778159c20 con 0x7f677815d850 2024-04-15T15:20:05.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.009+0000 7f677d967640 1 -- 172.21.15.105:0/777533202 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6778069150 con 0x7f677815d850 2024-04-15T15:20:05.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.009+0000 7f676ffff640 1 -- 172.21.15.105:0/777533202 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7f6760024020 con 0x7f677815d850 2024-04-15T15:20:05.011 INFO:teuthology.orchestra.run.smithi105.stderr:listed 0 entries 2024-04-15T15:20:05.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.011+0000 7f677d967640 1 -- 172.21.15.105:0/777533202 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f675806c8f0 msgr2=0x7f675806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:05.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.011+0000 7f677d967640 1 --2- 172.21.15.105:0/777533202 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f675806c8f0 0x7f675806edb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6768034000 tx=0x7f676802f000 comp rx=0 tx=0).stop 2024-04-15T15:20:05.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.011+0000 7f677d967640 1 -- 172.21.15.105:0/777533202 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f677815d850 msgr2=0x7f677815c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:05.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.011+0000 7f677d967640 1 --2- 172.21.15.105:0/777533202 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f677815d850 0x7f677815c9e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f67600108c0 tx=0x7f6760010d80 comp rx=0 tx=0).stop 2024-04-15T15:20:05.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.011+0000 7f677d967640 1 -- 172.21.15.105:0/777533202 shutdown_connections 2024-04-15T15:20:05.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.011+0000 7f677d967640 1 --2- 172.21.15.105:0/777533202 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f675806c8f0 0x7f675806edb0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:05.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.011+0000 7f677d967640 1 --2- 172.21.15.105:0/777533202 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67781578f0 0x7f6778157d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:05.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.011+0000 7f677d967640 1 --2- 172.21.15.105:0/777533202 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f677815d850 0x7f677815c9e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:05.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.011+0000 7f677d967640 1 --2- 172.21.15.105:0/777533202 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f67640039e0 0x7f677814c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:05.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.011+0000 7f677d967640 1 -- 172.21.15.105:0/777533202 >> 172.21.15.105:0/777533202 conn(0x7f677809aa40 msgr2=0x7f67780d1a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:05.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.012+0000 7f677d967640 1 -- 172.21.15.105:0/777533202 shutdown_connections 2024-04-15T15:20:05.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.012+0000 7f677d967640 1 -- 172.21.15.105:0/777533202 wait complete. 2024-04-15T15:20:05.024 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi105.front.sepia.ceph.com... 2024-04-15T15:20:05.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:05.024 DEBUG:teuthology.orchestra.run.smithi105:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-04-15T15:20:05.052 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:20:05.053 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi105.front.sepia.ceph.com 2024-04-15T15:20:05.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:05.053 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-15T15:20:05.108 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-15T15:20:05.108 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:20:05.128 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-15T15:20:05.128 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:20:05.188 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:20:05.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.289+0000 7ff97d67e640 1 Processor -- start 2024-04-15T15:20:05.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.289+0000 7ff97d67e640 1 -- start start 2024-04-15T15:20:05.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.289+0000 7ff97d67e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff9780d18e0 0x7ff978162630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:05.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.289+0000 7ff97d67e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff978162b70 0x7ff9780d3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:05.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.289+0000 7ff97d67e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff97815ef40 con 0x7ff97815eb60 2024-04-15T15:20:05.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.289+0000 7ff97d67e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7ff978163540 con 0x7ff978162b70 2024-04-15T15:20:05.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.289+0000 7ff97d67e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7ff9780d2fc0 con 0x7ff9780d18e0 2024-04-15T15:20:05.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.290+0000 7ff9777fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff978162b70 0x7ff9780d3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:05.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.290+0000 7ff9767fc640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff9780d18e0 0x7ff978162630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:05.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.290+0000 7ff9777fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff978162b70 0x7ff9780d3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.105:58396/0 (socket says 172.21.15.105:58396) 2024-04-15T15:20:05.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.290+0000 7ff976ffd640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7ff97815eb60 0x7ff9780d11d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.105:49178/0 (socket says 172.21.15.105:49178) 2024-04-15T15:20:05.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.290+0000 7ff9777fe640 1 -- 172.21.15.105:0/2820836002 learned_addr learned my addr 172.21.15.105:0/2820836002 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:05.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.290+0000 7ff9767fc640 1 -- 172.21.15.105:0/2820836002 >> v2:172.21.15.154:3300/0 conn(0x7ff978162b70 msgr2=0x7ff9780d3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:05.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.290+0000 7ff9767fc640 1 --2- 172.21.15.105:0/2820836002 >> v2:172.21.15.154:3300/0 conn(0x7ff978162b70 0x7ff9780d3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:05.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.290+0000 7ff9767fc640 1 -- 172.21.15.105:0/2820836002 >> v1:172.21.15.154:6789/0 conn(0x7ff97815eb60 legacy=0x7ff9780d11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:05.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.290+0000 7ff9767fc640 1 -- 172.21.15.105:0/2820836002 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff978162fc0 con 0x7ff9780d18e0 2024-04-15T15:20:05.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.290+0000 7ff9777fe640 1 --2- 172.21.15.105:0/2820836002 >> v2:172.21.15.154:3300/0 conn(0x7ff978162b70 0x7ff9780d3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:20:05.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.291+0000 7ff9767fc640 1 --2- 172.21.15.105:0/2820836002 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff9780d18e0 0x7ff978162630 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff96c0097a0 tx=0x7ff96c02b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2090cd1b82d2ccc server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:05.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.291+0000 7ff975ffb640 1 -- 172.21.15.105:0/2820836002 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96c035020 con 0x7ff9780d18e0 2024-04-15T15:20:05.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.291+0000 7ff975ffb640 1 -- 172.21.15.105:0/2820836002 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff96c031830 con 0x7ff9780d18e0 2024-04-15T15:20:05.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.291+0000 7ff975ffb640 1 -- 172.21.15.105:0/2820836002 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96c031c00 con 0x7ff9780d18e0 2024-04-15T15:20:05.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.292+0000 7ff97d67e640 1 -- 172.21.15.105:0/2820836002 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff9780d18e0 msgr2=0x7ff978162630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:05.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.292+0000 7ff97d67e640 1 --2- 172.21.15.105:0/2820836002 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff9780d18e0 0x7ff978162630 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff96c0097a0 tx=0x7ff96c02b8d0 comp rx=0 tx=0).stop 2024-04-15T15:20:05.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.292+0000 7ff97d67e640 1 -- 172.21.15.105:0/2820836002 shutdown_connections 2024-04-15T15:20:05.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.292+0000 7ff97d67e640 1 --2- 172.21.15.105:0/2820836002 >> v2:172.21.15.154:3300/0 conn(0x7ff978162b70 0x7ff9780d3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:05.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.292+0000 7ff97d67e640 1 --2- 172.21.15.105:0/2820836002 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff9780d18e0 0x7ff978162630 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:05.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.292+0000 7ff97d67e640 1 -- 172.21.15.105:0/2820836002 >> 172.21.15.105:0/2820836002 conn(0x7ff978067720 msgr2=0x7ff978067b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:05.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.292+0000 7ff97d67e640 1 -- 172.21.15.105:0/2820836002 shutdown_connections 2024-04-15T15:20:05.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.292+0000 7ff97d67e640 1 -- 172.21.15.105:0/2820836002 wait complete. 2024-04-15T15:20:05.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.293+0000 7ff97d67e640 1 Processor -- start 2024-04-15T15:20:05.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.293+0000 7ff97d67e640 1 -- start start 2024-04-15T15:20:05.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.294+0000 7ff97d67e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9780d18e0 0x7ff97815f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:05.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.294+0000 7ff97d67e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff97815eb60 0x7ff97815ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:05.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.294+0000 7ff97d67e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff978162b70 0x7ff9780cb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:05.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.294+0000 7ff97d67e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff9780d2c50 con 0x7ff9780d18e0 2024-04-15T15:20:05.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.294+0000 7ff97d67e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff9780d2970 con 0x7ff97815eb60 2024-04-15T15:20:05.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.294+0000 7ff97d67e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7ff9780b78f0 con 0x7ff978162b70 2024-04-15T15:20:05.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.294+0000 7ff976ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9780d18e0 0x7ff97815f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:05.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.294+0000 7ff976ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9780d18e0 0x7ff97815f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37530/0 (socket says 172.21.15.105:37530) 2024-04-15T15:20:05.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.294+0000 7ff976ffd640 1 -- 172.21.15.105:0/1276692294 learned_addr learned my addr 172.21.15.105:0/1276692294 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:05.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.294+0000 7ff9777fe640 1 --2- 172.21.15.105:0/1276692294 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff978162b70 0x7ff9780cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:05.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.294+0000 7ff976ffd640 1 -- 172.21.15.105:0/1276692294 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff978162b70 msgr2=0x7ff9780cb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:05.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.294+0000 7ff976ffd640 1 --2- 172.21.15.105:0/1276692294 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff978162b70 0x7ff9780cb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:05.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.294+0000 7ff976ffd640 1 -- 172.21.15.105:0/1276692294 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff97815eb60 msgr2=0x7ff97815ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:05.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.294+0000 7ff9767fc640 1 --2- 172.21.15.105:0/1276692294 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff97815eb60 0x7ff97815ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:05.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.294+0000 7ff976ffd640 1 --2- 172.21.15.105:0/1276692294 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff97815eb60 0x7ff97815ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:05.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.294+0000 7ff976ffd640 1 -- 172.21.15.105:0/1276692294 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff97815faf0 con 0x7ff9780d18e0 2024-04-15T15:20:05.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.294+0000 7ff9777fe640 1 --2- 172.21.15.105:0/1276692294 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff978162b70 0x7ff9780cb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:05.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.295+0000 7ff976ffd640 1 --2- 172.21.15.105:0/1276692294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9780d18e0 0x7ff97815f5b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff96800c790 tx=0x7ff96800cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:05.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.295+0000 7ff95f7fe640 1 -- 172.21.15.105:0/1276692294 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff968017070 con 0x7ff9780d18e0 2024-04-15T15:20:05.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.295+0000 7ff95f7fe640 1 -- 172.21.15.105:0/1276692294 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff96800a640 con 0x7ff9780d18e0 2024-04-15T15:20:05.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.295+0000 7ff97d67e640 1 -- 172.21.15.105:0/1276692294 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff97815fd90 con 0x7ff9780d18e0 2024-04-15T15:20:05.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.295+0000 7ff95f7fe640 1 -- 172.21.15.105:0/1276692294 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96800a9f0 con 0x7ff9780d18e0 2024-04-15T15:20:05.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.295+0000 7ff97d67e640 1 -- 172.21.15.105:0/1276692294 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff978160350 con 0x7ff9780d18e0 2024-04-15T15:20:05.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.296+0000 7ff97d67e640 1 -- 172.21.15.105:0/1276692294 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff978068a40 con 0x7ff9780d18e0 2024-04-15T15:20:05.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.296+0000 7ff95f7fe640 1 -- 172.21.15.105:0/1276692294 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff96801a070 con 0x7ff9780d18e0 2024-04-15T15:20:05.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.297+0000 7ff95f7fe640 1 --2- 172.21.15.105:0/1276692294 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff96406c8f0 0x7ff96406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:05.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.297+0000 7ff95f7fe640 1 -- 172.21.15.105:0/1276692294 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff96808b010 con 0x7ff9780d18e0 2024-04-15T15:20:05.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.297+0000 7ff9767fc640 1 --2- 172.21.15.105:0/1276692294 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff96406c8f0 0x7ff96406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:05.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.297+0000 7ff9767fc640 1 --2- 172.21.15.105:0/1276692294 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff96406c8f0 0x7ff96406edb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff96c036000 tx=0x7ff96c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:05.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.301+0000 7ff95f7fe640 1 -- 172.21.15.105:0/1276692294 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff968058e80 con 0x7ff9780d18e0 2024-04-15T15:20:05.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.481+0000 7ff97d67e640 1 -- 172.21.15.105:0/1276692294 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff9780cb1b0 con 0x7ff9780d18e0 2024-04-15T15:20:05.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.482+0000 7ff95f7fe640 1 -- 172.21.15.105:0/1276692294 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16653 (secure 0 0 0) 0x7ff96805d180 con 0x7ff9780d18e0 2024-04-15T15:20:05.484 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:20:05.484 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":20,"fsid":"09e4d2a4-a324-4793-af77-644b25e520e1","created":"2024-04-15T15:19:01.524543+0000","modified":"2024-04-15T15:19:27.414890+0000","last_up_change":"2024-04-15T15:19:12.825815+0000","last_in_change":"2024-04-15T15:19:03.788126+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:19:13.748585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:19:23.789287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:19:24.722250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6f6292b-2d7e-4aa0-9d1c-6ddd0f97287c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6801","nonce":2873040054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6803","nonce":2873040054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6807","nonce":2873040054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6805","nonce":2873040054}]},"public_addr":"172.21.15.105:6801/2873040054","cluster_addr":"172.21.15.105:6803/2873040054","heartbeat_back_addr":"172.21.15.105:6807/2873040054","heartbeat_front_addr":"172.21.15.105:6805/2873040054","state":["exists","up"]},{"osd":1,"uuid":"8900966a-b393-41e3-8f93-e5a43bd23622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6818","nonce":2494994479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6821","nonce":2494994479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6828","nonce":2494994479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6823","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6825","nonce":2494994479}]},"public_addr":"172.21.15.105:6818/2494994479","cluster_addr":"172.21.15.105:6821/2494994479","heartbeat_back_addr":"172.21.15.105:6828/2494994479","heartbeat_front_addr":"172.21.15.105:6825/2494994479","state":["exists","up"]},{"osd":2,"uuid":"81a5dc7a-9325-4ff5-941d-a2d5fba21f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6809","nonce":3798408699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6811","nonce":3798408699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6815","nonce":3798408699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6813","nonce":3798408699}]},"public_addr":"172.21.15.105:6809/3798408699","cluster_addr":"172.21.15.105:6811/3798408699","heartbeat_back_addr":"172.21.15.105:6815/3798408699","heartbeat_front_addr":"172.21.15.105:6813/3798408699","state":["exists","up"]},{"osd":3,"uuid":"df766ed2-5c71-4805-96f4-3d4970edc960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6819","nonce":1920720186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6824","nonce":1920720186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6831","nonce":1920720186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6827","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6829","nonce":1920720186}]},"public_addr":"172.21.15.105:6819/1920720186","cluster_addr":"172.21.15.105:6824/1920720186","heartbeat_back_addr":"172.21.15.105:6831/1920720186","heartbeat_front_addr":"172.21.15.105:6829/1920720186","state":["exists","up"]},{"osd":4,"uuid":"e1039e03-28b8-49df-b951-c91930b93c9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6810","nonce":3516494456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6813","nonce":3516494456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6820","nonce":3516494456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6817","nonce":3516494456}]},"public_addr":"172.21.15.154:6810/3516494456","cluster_addr":"172.21.15.154:6813/3516494456","heartbeat_back_addr":"172.21.15.154:6820/3516494456","heartbeat_front_addr":"172.21.15.154:6817/3516494456","state":["exists","up"]},{"osd":5,"uuid":"a9f5eaf1-66ff-405a-a9e4-ec3340812a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6801","nonce":540367751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6803","nonce":540367751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6807","nonce":540367751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6805","nonce":540367751}]},"public_addr":"172.21.15.154:6801/540367751","cluster_addr":"172.21.15.154:6803/540367751","heartbeat_back_addr":"172.21.15.154:6807/540367751","heartbeat_front_addr":"172.21.15.154:6805/540367751","state":["exists","up"]},{"osd":6,"uuid":"267d02ea-675f-428f-aae8-e0375e4d4feb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6825","nonce":1550829005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6827","nonce":1550829005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6831","nonce":1550829005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6829","nonce":1550829005}]},"public_addr":"172.21.15.154:6825/1550829005","cluster_addr":"172.21.15.154:6827/1550829005","heartbeat_back_addr":"172.21.15.154:6831/1550829005","heartbeat_front_addr":"172.21.15.154:6829/1550829005","state":["exists","up"]},{"osd":7,"uuid":"759ac6cc-896a-423c-885f-9fe90d8dcb14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6809","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6811","nonce":3920089903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6816","nonce":3920089903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6823","nonce":3920089903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6819","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6821","nonce":3920089903}]},"public_addr":"172.21.15.154:6811/3920089903","cluster_addr":"172.21.15.154:6816/3920089903","heartbeat_back_addr":"172.21.15.154:6823/3920089903","heartbeat_front_addr":"172.21.15.154:6821/3920089903","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.949509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.984072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.907278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.947391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.975447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.998896+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:09.028739+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.894119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:20:05.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.484+0000 7ff97d67e640 1 -- 172.21.15.105:0/1276692294 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff96406c8f0 msgr2=0x7ff96406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:05.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.484+0000 7ff97d67e640 1 --2- 172.21.15.105:0/1276692294 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff96406c8f0 0x7ff96406edb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff96c036000 tx=0x7ff96c008030 comp rx=0 tx=0).stop 2024-04-15T15:20:05.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.484+0000 7ff97d67e640 1 -- 172.21.15.105:0/1276692294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9780d18e0 msgr2=0x7ff97815f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:05.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.484+0000 7ff97d67e640 1 --2- 172.21.15.105:0/1276692294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9780d18e0 0x7ff97815f5b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff96800c790 tx=0x7ff96800cc50 comp rx=0 tx=0).stop 2024-04-15T15:20:05.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.484+0000 7ff97d67e640 1 -- 172.21.15.105:0/1276692294 shutdown_connections 2024-04-15T15:20:05.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.484+0000 7ff97d67e640 1 --2- 172.21.15.105:0/1276692294 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff978162b70 0x7ff9780cb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:05.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.484+0000 7ff97d67e640 1 --2- 172.21.15.105:0/1276692294 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff96406c8f0 0x7ff96406edb0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:05.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.484+0000 7ff97d67e640 1 --2- 172.21.15.105:0/1276692294 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff97815eb60 0x7ff97815ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:05.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.484+0000 7ff97d67e640 1 --2- 172.21.15.105:0/1276692294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9780d18e0 0x7ff97815f5b0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:05.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.484+0000 7ff97d67e640 1 -- 172.21.15.105:0/1276692294 >> 172.21.15.105:0/1276692294 conn(0x7ff978067720 msgr2=0x7ff9780b4b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:05.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.484+0000 7ff97d67e640 1 -- 172.21.15.105:0/1276692294 shutdown_connections 2024-04-15T15:20:05.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.484+0000 7ff97d67e640 1 -- 172.21.15.105:0/1276692294 wait complete. 2024-04-15T15:20:05.497 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:19:13.748585+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-15T15:19:23.789287+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-15T15:19:24.722250+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:20:05.497 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:20:05.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.602+0000 7f755dffa640 1 Processor -- start 2024-04-15T15:20:05.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.602+0000 7f755dffa640 1 -- start start 2024-04-15T15:20:05.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.603+0000 7f755dffa640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f75580b3c20 0x7f7558163490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:05.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.603+0000 7f755dffa640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75580cfc70 con 0x7f75580cf7f0 2024-04-15T15:20:05.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.603+0000 7f755dffa640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75580cff20 con 0x7f75580b6580 2024-04-15T15:20:05.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.603+0000 7f755dffa640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f75580d0200 con 0x7f75580b3c20 2024-04-15T15:20:05.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.603+0000 7f7556ffd640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f75580cf7f0 0x7f75580b7ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:55158/0 (socket says 172.21.15.105:55158) 2024-04-15T15:20:05.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.603+0000 7f7556ffd640 1 -- 172.21.15.105:0/891547249 learned_addr learned my addr 172.21.15.105:0/891547249 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:20:05.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.604+0000 7f75577fe640 1 --2- 172.21.15.105:0/891547249 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f75580b3c20 0x7f7558163490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:05.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.604+0000 7f75567fc640 1 -- 172.21.15.105:0/891547249 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 273899625 0 0) 0x7f75580cfc70 con 0x7f75580cf7f0 2024-04-15T15:20:05.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.604+0000 7f75567fc640 1 -- 172.21.15.105:0/891547249 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7540003520 con 0x7f75580cf7f0 2024-04-15T15:20:05.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.604+0000 7f75567fc640 1 -- 172.21.15.105:0/891547249 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2224823899 0 0) 0x7f7540003520 con 0x7f75580cf7f0 2024-04-15T15:20:05.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.604+0000 7f75567fc640 1 -- 172.21.15.105:0/891547249 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f75580b3c20 msgr2=0x7f7558163490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:05.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.604+0000 7f75567fc640 1 --2- 172.21.15.105:0/891547249 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f75580b3c20 0x7f7558163490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:05.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.604+0000 7f75567fc640 1 -- 172.21.15.105:0/891547249 >> v1:172.21.15.154:6789/0 conn(0x7f75580b6580 legacy=0x7f75580d3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:05.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.604+0000 7f75567fc640 1 -- 172.21.15.105:0/891547249 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75580d04e0 con 0x7f75580cf7f0 2024-04-15T15:20:05.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.604+0000 7f75567fc640 1 -- 172.21.15.105:0/891547249 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7f754c001670 con 0x7f75580cf7f0 2024-04-15T15:20:05.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.604+0000 7f75567fc640 1 -- 172.21.15.105:0/891547249 >> v1:172.21.15.105:6789/0 conn(0x7f75580cf7f0 legacy=0x7f75580b7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:05.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.604+0000 7f75567fc640 1 --2- 172.21.15.105:0/891547249 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f75580b3c20 0x7f75400039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:05.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.604+0000 7f75567fc640 1 --2- 172.21.15.105:0/891547249 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7540003f60 0x7f7540004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:05.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.604+0000 7f75567fc640 1 --2- 172.21.15.105:0/891547249 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7540004d40 0x7f75400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:05.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.604+0000 7f75567fc640 1 -- 172.21.15.105:0/891547249 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f75580d0200 con 0x7f7540004d40 2024-04-15T15:20:05.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.604+0000 7f75567fc640 1 -- 172.21.15.105:0/891547249 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f754000baa0 con 0x7f7540003f60 2024-04-15T15:20:05.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.604+0000 7f75567fc640 1 -- 172.21.15.105:0/891547249 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f754000bca0 con 0x7f75580b3c20 2024-04-15T15:20:05.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.604+0000 7f7557fff640 1 --2- 172.21.15.105:0/891547249 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7540004d40 0x7f75400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:05.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.604+0000 7f75577fe640 1 --2- 172.21.15.105:0/891547249 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f75580b3c20 0x7f75400039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:05.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.604+0000 7f7556ffd640 1 --2- 172.21.15.105:0/891547249 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7540003f60 0x7f7540004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:05.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.604+0000 7f7557fff640 1 -- 172.21.15.105:0/891547249 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f75580b3c20 msgr2=0x7f75400039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:05.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.604+0000 7f7557fff640 1 --2- 172.21.15.105:0/891547249 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f75580b3c20 0x7f75400039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:05.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.604+0000 7f7557fff640 1 -- 172.21.15.105:0/891547249 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7540003f60 msgr2=0x7f7540004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:05.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.604+0000 7f7557fff640 1 --2- 172.21.15.105:0/891547249 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7540003f60 0x7f7540004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:05.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.604+0000 7f7557fff640 1 -- 172.21.15.105:0/891547249 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7540003520 con 0x7f7540004d40 2024-04-15T15:20:05.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.605+0000 7f7557fff640 1 --2- 172.21.15.105:0/891547249 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7540004d40 0x7f75400051c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7550001750 tx=0x7f7540005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0e8dce56c5c5f8f server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:05.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.605+0000 7f75567fc640 1 -- 172.21.15.105:0/891547249 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7550001370 con 0x7f7540004d40 2024-04-15T15:20:05.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.605+0000 7f75567fc640 1 -- 172.21.15.105:0/891547249 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75500052d0 con 0x7f7540004d40 2024-04-15T15:20:05.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.605+0000 7f75567fc640 1 -- 172.21.15.105:0/891547249 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7550003830 con 0x7f7540004d40 2024-04-15T15:20:05.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.606+0000 7f755dffa640 1 -- 172.21.15.105:0/891547249 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7540004d40 msgr2=0x7f75400051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:05.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.606+0000 7f755dffa640 1 --2- 172.21.15.105:0/891547249 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7540004d40 0x7f75400051c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7550001750 tx=0x7f7540005830 comp rx=0 tx=0).stop 2024-04-15T15:20:05.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.606+0000 7f7557fff640 1 -- 172.21.15.105:0/891547249 reap_dead start 2024-04-15T15:20:05.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.606+0000 7f755dffa640 1 -- 172.21.15.105:0/891547249 shutdown_connections 2024-04-15T15:20:05.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.606+0000 7f755dffa640 1 -- 172.21.15.105:0/891547249 >> 172.21.15.105:0/891547249 conn(0x7f75580d1670 msgr2=0x7f75580d1a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:05.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.606+0000 7f755dffa640 1 -- 172.21.15.105:0/891547249 shutdown_connections 2024-04-15T15:20:05.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.606+0000 7f755dffa640 1 -- 172.21.15.105:0/891547249 wait complete. 2024-04-15T15:20:05.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.607+0000 7f755dffa640 1 Processor -- start 2024-04-15T15:20:05.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.607+0000 7f755dffa640 1 -- start start 2024-04-15T15:20:05.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.608+0000 7f755dffa640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7540004d40 0x7f75580acd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:05.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.608+0000 7f755dffa640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75580b4860 0x7f7558156b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:05.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.608+0000 7f755dffa640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75580b8750 0x7f75580b8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:05.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.608+0000 7f755dffa640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f75580b85d0 con 0x7f75580b8750 2024-04-15T15:20:05.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.608+0000 7f755dffa640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f75580d3610 con 0x7f75580b4860 2024-04-15T15:20:05.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.608+0000 7f755dffa640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f75580d4240 con 0x7f7540004d40 2024-04-15T15:20:05.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.608+0000 7f7557fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75580b8750 0x7f75580b8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:05.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.608+0000 7f7557fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75580b8750 0x7f75580b8bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37546/0 (socket says 172.21.15.105:37546) 2024-04-15T15:20:05.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.608+0000 7f7557fff640 1 -- 172.21.15.105:0/447349107 learned_addr learned my addr 172.21.15.105:0/447349107 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:05.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.608+0000 7f75577fe640 1 --2- 172.21.15.105:0/447349107 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7540004d40 0x7f75580acd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:05.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.608+0000 7f7556ffd640 1 --2- 172.21.15.105:0/447349107 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75580b4860 0x7f7558156b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:05.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.608+0000 7f7557fff640 1 -- 172.21.15.105:0/447349107 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7540004d40 msgr2=0x7f75580acd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:05.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.608+0000 7f7557fff640 1 --2- 172.21.15.105:0/447349107 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7540004d40 0x7f75580acd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:05.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.608+0000 7f7557fff640 1 -- 172.21.15.105:0/447349107 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75580b4860 msgr2=0x7f7558156b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:05.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.608+0000 7f7557fff640 1 --2- 172.21.15.105:0/447349107 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75580b4860 0x7f7558156b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:05.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.608+0000 7f7557fff640 1 -- 172.21.15.105:0/447349107 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75580cf100 con 0x7f75580b8750 2024-04-15T15:20:05.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.608+0000 7f75577fe640 1 --2- 172.21.15.105:0/447349107 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7540004d40 0x7f75580acd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:05.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.608+0000 7f7556ffd640 1 --2- 172.21.15.105:0/447349107 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75580b4860 0x7f7558156b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:05.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.608+0000 7f7557fff640 1 --2- 172.21.15.105:0/447349107 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75580b8750 0x7f75580b8bd0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7550001870 tx=0x7f7550001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:05.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.609+0000 7f753ffff640 1 -- 172.21.15.105:0/447349107 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7550001370 con 0x7f75580b8750 2024-04-15T15:20:05.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.609+0000 7f753ffff640 1 -- 172.21.15.105:0/447349107 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7550044070 con 0x7f75580b8750 2024-04-15T15:20:05.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.609+0000 7f755dffa640 1 -- 172.21.15.105:0/447349107 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75580ac240 con 0x7f75580b8750 2024-04-15T15:20:05.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.609+0000 7f753ffff640 1 -- 172.21.15.105:0/447349107 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7550007a30 con 0x7f75580b8750 2024-04-15T15:20:05.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.609+0000 7f755dffa640 1 -- 172.21.15.105:0/447349107 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7558160d80 con 0x7f75580b8750 2024-04-15T15:20:05.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.610+0000 7f755dffa640 1 -- 172.21.15.105:0/447349107 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7558068ab0 con 0x7f75580b8750 2024-04-15T15:20:05.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.610+0000 7f753ffff640 1 -- 172.21.15.105:0/447349107 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f755003b040 con 0x7f75580b8750 2024-04-15T15:20:05.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.610+0000 7f753ffff640 1 --2- 172.21.15.105:0/447349107 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f754406c920 0x7f754406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:05.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.611+0000 7f753ffff640 1 -- 172.21.15.105:0/447349107 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f75500acef0 con 0x7f75580b8750 2024-04-15T15:20:05.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.611+0000 7f75577fe640 1 --2- 172.21.15.105:0/447349107 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f754406c920 0x7f754406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:05.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.611+0000 7f75577fe640 1 --2- 172.21.15.105:0/447349107 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f754406c920 0x7f754406ede0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7548007440 tx=0x7f754800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:05.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.615+0000 7f753ffff640 1 -- 172.21.15.105:0/447349107 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f755007b440 con 0x7f75580b8750 2024-04-15T15:20:05.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.791+0000 7f755dffa640 1 -- 172.21.15.105:0/447349107 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f75580d2a20 con 0x7f75580b8750 2024-04-15T15:20:05.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.791+0000 7f753ffff640 1 -- 172.21.15.105:0/447349107 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f755007ad40 con 0x7f75580b8750 2024-04-15T15:20:05.793 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-04-15T15:20:05.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.793+0000 7f755dffa640 1 -- 172.21.15.105:0/447349107 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f754406c920 msgr2=0x7f754406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:05.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.793+0000 7f755dffa640 1 --2- 172.21.15.105:0/447349107 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f754406c920 0x7f754406ede0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7548007440 tx=0x7f754800b040 comp rx=0 tx=0).stop 2024-04-15T15:20:05.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.793+0000 7f755dffa640 1 -- 172.21.15.105:0/447349107 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75580b8750 msgr2=0x7f75580b8bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:05.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.793+0000 7f755dffa640 1 --2- 172.21.15.105:0/447349107 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75580b8750 0x7f75580b8bd0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7550001870 tx=0x7f7550001120 comp rx=0 tx=0).stop 2024-04-15T15:20:05.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.793+0000 7f755dffa640 1 -- 172.21.15.105:0/447349107 shutdown_connections 2024-04-15T15:20:05.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.793+0000 7f755dffa640 1 --2- 172.21.15.105:0/447349107 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f754406c920 0x7f754406ede0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:05.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.793+0000 7f755dffa640 1 --2- 172.21.15.105:0/447349107 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75580b8750 0x7f75580b8bd0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:05.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.793+0000 7f755dffa640 1 --2- 172.21.15.105:0/447349107 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75580b4860 0x7f7558156b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:05.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.793+0000 7f755dffa640 1 --2- 172.21.15.105:0/447349107 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7540004d40 0x7f75580acd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:05.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.793+0000 7f755dffa640 1 -- 172.21.15.105:0/447349107 >> 172.21.15.105:0/447349107 conn(0x7f75580d1670 msgr2=0x7f75580b5af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:05.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.793+0000 7f755dffa640 1 -- 172.21.15.105:0/447349107 shutdown_connections 2024-04-15T15:20:05.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.793+0000 7f755dffa640 1 -- 172.21.15.105:0/447349107 wait complete. 2024-04-15T15:20:05.806 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-15T15:20:05.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.910+0000 7ff39495f640 1 Processor -- start 2024-04-15T15:20:05.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.910+0000 7ff39495f640 1 -- start start 2024-04-15T15:20:05.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.911+0000 7ff39495f640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff3900ce9c0 0x7ff3900ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:05.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.911+0000 7ff39495f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff39009b150 0x7ff3900b1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:05.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.911+0000 7ff39495f640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3900d1730 con 0x7ff39015ff10 2024-04-15T15:20:05.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.911+0000 7ff39495f640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7ff3900cd1e0 con 0x7ff3900ce9c0 2024-04-15T15:20:05.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.911+0000 7ff39495f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7ff3900cd480 con 0x7ff39009b150 2024-04-15T15:20:05.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.911+0000 7ff38dd9b640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff3900ce9c0 0x7ff3900ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:05.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.911+0000 7ff38ed9d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff39009b150 0x7ff3900b1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:05.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.911+0000 7ff38e59c640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7ff39015ff10 0x7ff39009aa40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.105:49196/0 (socket says 172.21.15.105:49196) 2024-04-15T15:20:05.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.911+0000 7ff38dd9b640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff3900ce9c0 0x7ff3900ce240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.105:58444/0 (socket says 172.21.15.105:58444) 2024-04-15T15:20:05.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.911+0000 7ff38e59c640 1 -- 172.21.15.105:0/194558293 learned_addr learned my addr 172.21.15.105:0/194558293 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:20:05.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.912+0000 7ff38dd9b640 1 -- 172.21.15.105:0/194558293 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff39009b150 msgr2=0x7ff3900b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:05.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.912+0000 7ff38dd9b640 1 --2- 172.21.15.105:0/194558293 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff39009b150 0x7ff3900b1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:05.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.912+0000 7ff38dd9b640 1 -- 172.21.15.105:0/194558293 >> v1:172.21.15.154:6789/0 conn(0x7ff39015ff10 legacy=0x7ff39009aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:05.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.912+0000 7ff38d59a640 1 -- 172.21.15.105:0/194558293 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4137587840 0 0) 0x7ff380001490 con 0x7ff39015ff10 2024-04-15T15:20:05.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.912+0000 7ff38dd9b640 1 -- 172.21.15.105:0/194558293 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3900cd760 con 0x7ff3900ce9c0 2024-04-15T15:20:05.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.912+0000 7ff38ed9d640 1 --2- 172.21.15.105:0/194558293 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff39009b150 0x7ff3900b1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:20:05.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.913+0000 7ff38dd9b640 1 --2- 172.21.15.105:0/194558293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3900ce9c0 0x7ff3900ce240 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff3840097a0 tx=0x7ff38402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84a1ade592bb086e server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:05.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.913+0000 7ff38d59a640 1 -- 172.21.15.105:0/194558293 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff384035020 con 0x7ff3900ce9c0 2024-04-15T15:20:05.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.913+0000 7ff38d59a640 1 -- 172.21.15.105:0/194558293 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff384031830 con 0x7ff3900ce9c0 2024-04-15T15:20:05.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.913+0000 7ff38d59a640 1 -- 172.21.15.105:0/194558293 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff384031c00 con 0x7ff3900ce9c0 2024-04-15T15:20:05.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.913+0000 7ff39495f640 1 -- 172.21.15.105:0/194558293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3900ce9c0 msgr2=0x7ff3900ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:05.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.913+0000 7ff39495f640 1 --2- 172.21.15.105:0/194558293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3900ce9c0 0x7ff3900ce240 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff3840097a0 tx=0x7ff38402b8d0 comp rx=0 tx=0).stop 2024-04-15T15:20:05.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.913+0000 7ff39495f640 1 -- 172.21.15.105:0/194558293 shutdown_connections 2024-04-15T15:20:05.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.913+0000 7ff39495f640 1 --2- 172.21.15.105:0/194558293 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff39009b150 0x7ff3900b1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:05.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.913+0000 7ff39495f640 1 --2- 172.21.15.105:0/194558293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3900ce9c0 0x7ff3900ce240 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:05.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.913+0000 7ff39495f640 1 -- 172.21.15.105:0/194558293 >> 172.21.15.105:0/194558293 conn(0x7ff3900b4170 msgr2=0x7ff3900b4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:05.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.913+0000 7ff39495f640 1 -- 172.21.15.105:0/194558293 shutdown_connections 2024-04-15T15:20:05.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.913+0000 7ff39495f640 1 -- 172.21.15.105:0/194558293 wait complete. 2024-04-15T15:20:05.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.915+0000 7ff39495f640 1 Processor -- start 2024-04-15T15:20:05.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.915+0000 7ff39495f640 1 -- start start 2024-04-15T15:20:05.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.916+0000 7ff39495f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff39009b150 0x7ff3900cb940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:05.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.916+0000 7ff39495f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3900ce9c0 0x7ff3900c9f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:05.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.916+0000 7ff39495f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff39015ff10 0x7ff3901f42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:05.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.916+0000 7ff39495f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff3900d0580 con 0x7ff39009b150 2024-04-15T15:20:05.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.916+0000 7ff39495f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff3900cdd20 con 0x7ff3900ce9c0 2024-04-15T15:20:05.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.916+0000 7ff39495f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7ff3900d0700 con 0x7ff39015ff10 2024-04-15T15:20:05.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.916+0000 7ff38e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff39009b150 0x7ff3900cb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:05.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.916+0000 7ff38e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff39009b150 0x7ff3900cb940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37548/0 (socket says 172.21.15.105:37548) 2024-04-15T15:20:05.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.916+0000 7ff38e59c640 1 -- 172.21.15.105:0/336661092 learned_addr learned my addr 172.21.15.105:0/336661092 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:05.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.916+0000 7ff38dd9b640 1 --2- 172.21.15.105:0/336661092 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3900ce9c0 0x7ff3900c9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:05.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.916+0000 7ff38ed9d640 1 --2- 172.21.15.105:0/336661092 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff39015ff10 0x7ff3901f42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:05.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.916+0000 7ff38e59c640 1 -- 172.21.15.105:0/336661092 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff39015ff10 msgr2=0x7ff3901f42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:05.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.916+0000 7ff38e59c640 1 --2- 172.21.15.105:0/336661092 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff39015ff10 0x7ff3901f42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:05.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.916+0000 7ff38e59c640 1 -- 172.21.15.105:0/336661092 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3900ce9c0 msgr2=0x7ff3900c9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:05.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.916+0000 7ff38e59c640 1 --2- 172.21.15.105:0/336661092 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3900ce9c0 0x7ff3900c9f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:05.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.916+0000 7ff38e59c640 1 -- 172.21.15.105:0/336661092 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3901f54c0 con 0x7ff39009b150 2024-04-15T15:20:05.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.916+0000 7ff38dd9b640 1 --2- 172.21.15.105:0/336661092 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3900ce9c0 0x7ff3900c9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:20:05.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.916+0000 7ff38e59c640 1 --2- 172.21.15.105:0/336661092 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff39009b150 0x7ff3900cb940 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff38000c880 tx=0x7ff38000cd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:05.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.916+0000 7ff37effd640 1 -- 172.21.15.105:0/336661092 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff380018070 con 0x7ff39009b150 2024-04-15T15:20:05.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.916+0000 7ff37effd640 1 -- 172.21.15.105:0/336661092 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff38000a640 con 0x7ff39009b150 2024-04-15T15:20:05.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.916+0000 7ff39495f640 1 -- 172.21.15.105:0/336661092 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3901f4bc0 con 0x7ff39009b150 2024-04-15T15:20:05.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.916+0000 7ff37effd640 1 -- 172.21.15.105:0/336661092 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38000aa30 con 0x7ff39009b150 2024-04-15T15:20:05.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.916+0000 7ff39495f640 1 -- 172.21.15.105:0/336661092 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3901f5180 con 0x7ff39009b150 2024-04-15T15:20:05.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.917+0000 7ff39495f640 1 -- 172.21.15.105:0/336661092 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff390068a40 con 0x7ff39009b150 2024-04-15T15:20:05.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.918+0000 7ff37effd640 1 -- 172.21.15.105:0/336661092 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff38001b070 con 0x7ff39009b150 2024-04-15T15:20:05.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.918+0000 7ff37effd640 1 --2- 172.21.15.105:0/336661092 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff37006c8f0 0x7ff37006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:05.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.918+0000 7ff37effd640 1 -- 172.21.15.105:0/336661092 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff38008b1e0 con 0x7ff39009b150 2024-04-15T15:20:05.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.918+0000 7ff38dd9b640 1 --2- 172.21.15.105:0/336661092 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff37006c8f0 0x7ff37006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:05.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.919+0000 7ff38dd9b640 1 --2- 172.21.15.105:0/336661092 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff37006c8f0 0x7ff37006edb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff384036000 tx=0x7ff384008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:05.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:05.923+0000 7ff37effd640 1 -- 172.21.15.105:0/336661092 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff3900d1730 con 0x7ff39009b150 2024-04-15T15:20:06.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.099+0000 7ff39495f640 1 -- 172.21.15.105:0/336661092 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff3900b8600 con 0x7ff39009b150 2024-04-15T15:20:06.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.099+0000 7ff37effd640 1 -- 172.21.15.105:0/336661092 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7ff380059610 con 0x7ff39009b150 2024-04-15T15:20:06.101 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-04-15T15:20:06.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.101+0000 7ff39495f640 1 -- 172.21.15.105:0/336661092 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff37006c8f0 msgr2=0x7ff37006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:06.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.101+0000 7ff39495f640 1 --2- 172.21.15.105:0/336661092 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff37006c8f0 0x7ff37006edb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff384036000 tx=0x7ff384008030 comp rx=0 tx=0).stop 2024-04-15T15:20:06.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.101+0000 7ff39495f640 1 -- 172.21.15.105:0/336661092 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff39009b150 msgr2=0x7ff3900cb940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:06.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.101+0000 7ff39495f640 1 --2- 172.21.15.105:0/336661092 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff39009b150 0x7ff3900cb940 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff38000c880 tx=0x7ff38000cd40 comp rx=0 tx=0).stop 2024-04-15T15:20:06.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.102+0000 7ff39495f640 1 -- 172.21.15.105:0/336661092 shutdown_connections 2024-04-15T15:20:06.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.102+0000 7ff39495f640 1 --2- 172.21.15.105:0/336661092 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff39015ff10 0x7ff3901f42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:06.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.102+0000 7ff39495f640 1 --2- 172.21.15.105:0/336661092 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff37006c8f0 0x7ff37006edb0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:06.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.102+0000 7ff39495f640 1 --2- 172.21.15.105:0/336661092 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3900ce9c0 0x7ff3900c9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:06.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.102+0000 7ff39495f640 1 --2- 172.21.15.105:0/336661092 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff39009b150 0x7ff3900cb940 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:06.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.102+0000 7ff39495f640 1 -- 172.21.15.105:0/336661092 >> 172.21.15.105:0/336661092 conn(0x7ff3900b4170 msgr2=0x7ff39009aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:06.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.102+0000 7ff39495f640 1 -- 172.21.15.105:0/336661092 shutdown_connections 2024-04-15T15:20:06.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.102+0000 7ff39495f640 1 -- 172.21.15.105:0/336661092 wait complete. 2024-04-15T15:20:06.115 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-15T15:20:06.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.219+0000 7f1d9f235640 1 Processor -- start 2024-04-15T15:20:06.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.219+0000 7f1d9f235640 1 -- start start 2024-04-15T15:20:06.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.219+0000 7f1d9f235640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1d9806f6a0 0x7f1d980b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:06.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.219+0000 7f1d9f235640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d980b40a0 con 0x7f1d98144c90 2024-04-15T15:20:06.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.219+0000 7f1d9f235640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d980b4380 con 0x7f1d980cd1d0 2024-04-15T15:20:06.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.219+0000 7f1d9f235640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f1d980b3e50 con 0x7f1d9806f6a0 2024-04-15T15:20:06.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.220+0000 7f1d97fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1d9806f6a0 0x7f1d980b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:06.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.220+0000 7f1d9cfd1640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f1d98144c90 0x7f1d9806ef90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:55172/0 (socket says 172.21.15.105:55172) 2024-04-15T15:20:06.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.220+0000 7f1d97fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1d9806f6a0 0x7f1d980b4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37558/0 (socket says 172.21.15.105:37558) 2024-04-15T15:20:06.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.220+0000 7f1d9cfd1640 1 -- 172.21.15.105:0/3750058047 learned_addr learned my addr 172.21.15.105:0/3750058047 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:20:06.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.220+0000 7f1d97fff640 1 -- 172.21.15.105:0/3750058047 >> v1:172.21.15.154:6789/0 conn(0x7f1d980cd1d0 legacy=0x7f1d980cd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:06.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.220+0000 7f1d97fff640 1 -- 172.21.15.105:0/3750058047 >> v1:172.21.15.105:6789/0 conn(0x7f1d98144c90 legacy=0x7f1d9806ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:06.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.220+0000 7f1d97fff640 1 -- 172.21.15.105:0/3750058047 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d980b4660 con 0x7f1d9806f6a0 2024-04-15T15:20:06.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.221+0000 7f1d97fff640 1 --2- 172.21.15.105:0/3750058047 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d9806f6a0 0x7f1d980b4e60 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1d840097a0 tx=0x7f1d8402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7f0fc30bcb96728 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:06.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.221+0000 7f1d977fe640 1 -- 172.21.15.105:0/3750058047 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d84035020 con 0x7f1d9806f6a0 2024-04-15T15:20:06.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.221+0000 7f1d977fe640 1 -- 172.21.15.105:0/3750058047 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d84031830 con 0x7f1d9806f6a0 2024-04-15T15:20:06.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.221+0000 7f1d977fe640 1 -- 172.21.15.105:0/3750058047 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d84031c00 con 0x7f1d9806f6a0 2024-04-15T15:20:06.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.221+0000 7f1d9f235640 1 -- 172.21.15.105:0/3750058047 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d9806f6a0 msgr2=0x7f1d980b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:06.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.221+0000 7f1d9f235640 1 --2- 172.21.15.105:0/3750058047 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d9806f6a0 0x7f1d980b4e60 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1d840097a0 tx=0x7f1d8402b8d0 comp rx=0 tx=0).stop 2024-04-15T15:20:06.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.221+0000 7f1d9f235640 1 -- 172.21.15.105:0/3750058047 shutdown_connections 2024-04-15T15:20:06.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.221+0000 7f1d9f235640 1 --2- 172.21.15.105:0/3750058047 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d9806f6a0 0x7f1d980b4e60 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:06.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.221+0000 7f1d9f235640 1 -- 172.21.15.105:0/3750058047 >> 172.21.15.105:0/3750058047 conn(0x7f1d98067d50 msgr2=0x7f1d9806b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:06.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.221+0000 7f1d9f235640 1 -- 172.21.15.105:0/3750058047 shutdown_connections 2024-04-15T15:20:06.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.221+0000 7f1d9f235640 1 -- 172.21.15.105:0/3750058047 wait complete. 2024-04-15T15:20:06.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.222+0000 7f1d9f235640 1 Processor -- start 2024-04-15T15:20:06.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.222+0000 7f1d9f235640 1 -- start start 2024-04-15T15:20:06.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.223+0000 7f1d9f235640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1d9806f6a0 0x7f1d98138d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:06.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.223+0000 7f1d9f235640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d980cd1d0 0x7f1d9813c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:06.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.223+0000 7f1d9f235640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1d98144c90 0x7f1d981f15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:06.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.223+0000 7f1d9f235640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1d981451f0 con 0x7f1d980cd1d0 2024-04-15T15:20:06.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.223+0000 7f1d9f235640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1d98145070 con 0x7f1d98144c90 2024-04-15T15:20:06.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.223+0000 7f1d9f235640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f1d9815bbe0 con 0x7f1d9806f6a0 2024-04-15T15:20:06.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.223+0000 7f1d97fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d980cd1d0 0x7f1d9813c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:06.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.223+0000 7f1d97fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d980cd1d0 0x7f1d9813c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37574/0 (socket says 172.21.15.105:37574) 2024-04-15T15:20:06.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.223+0000 7f1d97fff640 1 -- 172.21.15.105:0/1115472218 learned_addr learned my addr 172.21.15.105:0/1115472218 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:06.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.224+0000 7f1d9cfd1640 1 --2- 172.21.15.105:0/1115472218 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1d9806f6a0 0x7f1d98138d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:06.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.224+0000 7f1d9d7d2640 1 --2- 172.21.15.105:0/1115472218 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1d98144c90 0x7f1d981f15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:06.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.224+0000 7f1d97fff640 1 -- 172.21.15.105:0/1115472218 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1d9806f6a0 msgr2=0x7f1d98138d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:06.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.224+0000 7f1d97fff640 1 --2- 172.21.15.105:0/1115472218 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1d9806f6a0 0x7f1d98138d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:06.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.224+0000 7f1d97fff640 1 -- 172.21.15.105:0/1115472218 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1d98144c90 msgr2=0x7f1d981f15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:06.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.224+0000 7f1d97fff640 1 --2- 172.21.15.105:0/1115472218 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1d98144c90 0x7f1d981f15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:06.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.224+0000 7f1d97fff640 1 -- 172.21.15.105:0/1115472218 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d9815b2b0 con 0x7f1d980cd1d0 2024-04-15T15:20:06.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.224+0000 7f1d97fff640 1 --2- 172.21.15.105:0/1115472218 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d980cd1d0 0x7f1d9813c1e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1d8402bee0 tx=0x7f1d840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:06.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.224+0000 7f1d957fa640 1 -- 172.21.15.105:0/1115472218 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d84041070 con 0x7f1d980cd1d0 2024-04-15T15:20:06.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.224+0000 7f1d957fa640 1 -- 172.21.15.105:0/1115472218 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d8403a950 con 0x7f1d980cd1d0 2024-04-15T15:20:06.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.224+0000 7f1d9f235640 1 -- 172.21.15.105:0/1115472218 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d9815d140 con 0x7f1d980cd1d0 2024-04-15T15:20:06.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.224+0000 7f1d957fa640 1 -- 172.21.15.105:0/1115472218 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d84035040 con 0x7f1d980cd1d0 2024-04-15T15:20:06.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.224+0000 7f1d9f235640 1 -- 172.21.15.105:0/1115472218 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d9815d6c0 con 0x7f1d980cd1d0 2024-04-15T15:20:06.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.225+0000 7f1d9f235640 1 -- 172.21.15.105:0/1115472218 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d9815e150 con 0x7f1d980cd1d0 2024-04-15T15:20:06.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.226+0000 7f1d957fa640 1 -- 172.21.15.105:0/1115472218 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1d8404c060 con 0x7f1d980cd1d0 2024-04-15T15:20:06.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.226+0000 7f1d957fa640 1 --2- 172.21.15.105:0/1115472218 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f1d7c06c920 0x7f1d7c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:06.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.226+0000 7f1d957fa640 1 -- 172.21.15.105:0/1115472218 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1d840ad550 con 0x7f1d980cd1d0 2024-04-15T15:20:06.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.226+0000 7f1d9cfd1640 1 --2- 172.21.15.105:0/1115472218 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f1d7c06c920 0x7f1d7c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:06.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.227+0000 7f1d9cfd1640 1 --2- 172.21.15.105:0/1115472218 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f1d7c06c920 0x7f1d7c06ede0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c005880 tx=0x7f1d8c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:06.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.230+0000 7f1d957fa640 1 -- 172.21.15.105:0/1115472218 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1d8407b980 con 0x7f1d980cd1d0 2024-04-15T15:20:06.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.404+0000 7f1d9f235640 1 -- 172.21.15.105:0/1115472218 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1d9813f430 con 0x7f1d980cd1d0 2024-04-15T15:20:06.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.405+0000 7f1d957fa640 1 -- 172.21.15.105:0/1115472218 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f1d8407b280 con 0x7f1d980cd1d0 2024-04-15T15:20:06.407 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-04-15T15:20:06.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.407+0000 7f1d9f235640 1 -- 172.21.15.105:0/1115472218 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f1d7c06c920 msgr2=0x7f1d7c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:06.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.407+0000 7f1d9f235640 1 --2- 172.21.15.105:0/1115472218 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f1d7c06c920 0x7f1d7c06ede0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c005880 tx=0x7f1d8c009040 comp rx=0 tx=0).stop 2024-04-15T15:20:06.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.407+0000 7f1d9f235640 1 -- 172.21.15.105:0/1115472218 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d980cd1d0 msgr2=0x7f1d9813c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:06.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.407+0000 7f1d9f235640 1 --2- 172.21.15.105:0/1115472218 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d980cd1d0 0x7f1d9813c1e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1d8402bee0 tx=0x7f1d840046c0 comp rx=0 tx=0).stop 2024-04-15T15:20:06.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.407+0000 7f1d9f235640 1 -- 172.21.15.105:0/1115472218 shutdown_connections 2024-04-15T15:20:06.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.407+0000 7f1d9f235640 1 --2- 172.21.15.105:0/1115472218 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f1d7c06c920 0x7f1d7c06ede0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:06.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.407+0000 7f1d9f235640 1 --2- 172.21.15.105:0/1115472218 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1d98144c90 0x7f1d981f15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:06.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.407+0000 7f1d9f235640 1 --2- 172.21.15.105:0/1115472218 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d980cd1d0 0x7f1d9813c1e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:06.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.407+0000 7f1d9f235640 1 --2- 172.21.15.105:0/1115472218 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1d9806f6a0 0x7f1d98138d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:06.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.407+0000 7f1d9f235640 1 -- 172.21.15.105:0/1115472218 >> 172.21.15.105:0/1115472218 conn(0x7f1d98067d50 msgr2=0x7f1d980b4e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:06.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.407+0000 7f1d9f235640 1 -- 172.21.15.105:0/1115472218 shutdown_connections 2024-04-15T15:20:06.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.407+0000 7f1d9f235640 1 -- 172.21.15.105:0/1115472218 wait complete. 2024-04-15T15:20:06.420 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-15T15:20:06.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.523+0000 7fe87f596640 1 Processor -- start 2024-04-15T15:20:06.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.524+0000 7fe87f596640 1 -- start start 2024-04-15T15:20:06.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.524+0000 7fe87f596640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe8780d18e0 0x7fe878162630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:06.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.524+0000 7fe87f596640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe878162b70 0x7fe8780d3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:06.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.524+0000 7fe87f596640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe87815ef40 con 0x7fe87815eb60 2024-04-15T15:20:06.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.524+0000 7fe87f596640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fe878163540 con 0x7fe8780d18e0 2024-04-15T15:20:06.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.524+0000 7fe87f596640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fe8780d2fc0 con 0x7fe878162b70 2024-04-15T15:20:06.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.525+0000 7fe87d332640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fe87815eb60 0x7fe8780d11d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:55182/0 (socket says 172.21.15.105:55182) 2024-04-15T15:20:06.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.525+0000 7fe87d332640 1 -- 172.21.15.105:0/2033982609 learned_addr learned my addr 172.21.15.105:0/2033982609 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:20:06.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.525+0000 7fe87cb31640 1 --2- 172.21.15.105:0/2033982609 >> v2:172.21.15.154:3300/0 conn(0x7fe8780d18e0 0x7fe878162630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:06.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.525+0000 7fe87db33640 1 --2- 172.21.15.105:0/2033982609 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe878162b70 0x7fe8780d3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:06.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.525+0000 7fe877fff640 1 -- 172.21.15.105:0/2033982609 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4162812994 0 0) 0x7fe87815ef40 con 0x7fe87815eb60 2024-04-15T15:20:06.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.525+0000 7fe877fff640 1 -- 172.21.15.105:0/2033982609 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe860003520 con 0x7fe87815eb60 2024-04-15T15:20:06.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.525+0000 7fe877fff640 1 -- 172.21.15.105:0/2033982609 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 822690816 0 0) 0x7fe860003520 con 0x7fe87815eb60 2024-04-15T15:20:06.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.525+0000 7fe877fff640 1 -- 172.21.15.105:0/2033982609 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe878162b70 msgr2=0x7fe8780d3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:06.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.525+0000 7fe877fff640 1 --2- 172.21.15.105:0/2033982609 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe878162b70 0x7fe8780d3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:06.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.525+0000 7fe877fff640 1 -- 172.21.15.105:0/2033982609 >> v2:172.21.15.154:3300/0 conn(0x7fe8780d18e0 msgr2=0x7fe878162630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:06.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.525+0000 7fe877fff640 1 --2- 172.21.15.105:0/2033982609 >> v2:172.21.15.154:3300/0 conn(0x7fe8780d18e0 0x7fe878162630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:06.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.525+0000 7fe877fff640 1 -- 172.21.15.105:0/2033982609 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe878163260 con 0x7fe87815eb60 2024-04-15T15:20:06.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.525+0000 7fe877fff640 1 -- 172.21.15.105:0/2033982609 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7fe870001670 con 0x7fe87815eb60 2024-04-15T15:20:06.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.525+0000 7fe877fff640 1 -- 172.21.15.105:0/2033982609 >> v1:172.21.15.105:6789/0 conn(0x7fe87815eb60 legacy=0x7fe8780d11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:06.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.525+0000 7fe877fff640 1 --2- 172.21.15.105:0/2033982609 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8600039b0 0x7fe860003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:06.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.525+0000 7fe877fff640 1 --2- 172.21.15.105:0/2033982609 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe860004800 0x7fe860004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:06.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.525+0000 7fe877fff640 1 --2- 172.21.15.105:0/2033982609 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe878162b70 0x7fe8600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:06.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.525+0000 7fe877fff640 1 -- 172.21.15.105:0/2033982609 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe878163540 con 0x7fe8600039b0 2024-04-15T15:20:06.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.525+0000 7fe877fff640 1 -- 172.21.15.105:0/2033982609 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe8780d2fc0 con 0x7fe860004800 2024-04-15T15:20:06.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.525+0000 7fe877fff640 1 -- 172.21.15.105:0/2033982609 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fe86000b9c0 con 0x7fe878162b70 2024-04-15T15:20:06.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.525+0000 7fe87d332640 1 --2- 172.21.15.105:0/2033982609 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8600039b0 0x7fe860003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:06.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.525+0000 7fe87cb31640 1 --2- 172.21.15.105:0/2033982609 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe860004800 0x7fe860004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:06.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.525+0000 7fe87db33640 1 --2- 172.21.15.105:0/2033982609 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe878162b70 0x7fe8600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:06.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.526+0000 7fe87d332640 1 -- 172.21.15.105:0/2033982609 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe878162b70 msgr2=0x7fe8600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:06.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.526+0000 7fe87d332640 1 --2- 172.21.15.105:0/2033982609 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe878162b70 0x7fe8600051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:06.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.526+0000 7fe87d332640 1 -- 172.21.15.105:0/2033982609 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe860004800 msgr2=0x7fe860004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:06.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.526+0000 7fe87d332640 1 --2- 172.21.15.105:0/2033982609 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe860004800 0x7fe860004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:06.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.526+0000 7fe87d332640 1 -- 172.21.15.105:0/2033982609 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe860003520 con 0x7fe8600039b0 2024-04-15T15:20:06.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.526+0000 7fe87d332640 1 --2- 172.21.15.105:0/2033982609 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8600039b0 0x7fe860003dd0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe87000c210 tx=0x7fe860005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cad62868389ad72 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:06.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.526+0000 7fe877fff640 1 -- 172.21.15.105:0/2033982609 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe870007d50 con 0x7fe8600039b0 2024-04-15T15:20:06.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.527+0000 7fe877fff640 1 -- 172.21.15.105:0/2033982609 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe870030070 con 0x7fe8600039b0 2024-04-15T15:20:06.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.527+0000 7fe877fff640 1 -- 172.21.15.105:0/2033982609 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87002cda0 con 0x7fe8600039b0 2024-04-15T15:20:06.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.527+0000 7fe87f596640 1 -- 172.21.15.105:0/2033982609 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8600039b0 msgr2=0x7fe860003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:06.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.527+0000 7fe87f596640 1 --2- 172.21.15.105:0/2033982609 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8600039b0 0x7fe860003dd0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe87000c210 tx=0x7fe860005830 comp rx=0 tx=0).stop 2024-04-15T15:20:06.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.527+0000 7fe87db33640 1 -- 172.21.15.105:0/2033982609 reap_dead start 2024-04-15T15:20:06.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.527+0000 7fe87f596640 1 -- 172.21.15.105:0/2033982609 shutdown_connections 2024-04-15T15:20:06.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.527+0000 7fe87f596640 1 -- 172.21.15.105:0/2033982609 >> 172.21.15.105:0/2033982609 conn(0x7fe878067720 msgr2=0x7fe878067b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:06.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.527+0000 7fe87f596640 1 -- 172.21.15.105:0/2033982609 shutdown_connections 2024-04-15T15:20:06.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.527+0000 7fe87f596640 1 -- 172.21.15.105:0/2033982609 wait complete. 2024-04-15T15:20:06.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.528+0000 7fe87f596640 1 Processor -- start 2024-04-15T15:20:06.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.528+0000 7fe87f596640 1 -- start start 2024-04-15T15:20:06.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.529+0000 7fe87f596640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8600039b0 0x7fe8781617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:06.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.529+0000 7fe87f596640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe878160b30 0x7fe8781f48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:06.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.529+0000 7fe87f596640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8781f4e00 0x7fe8781f5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:06.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.529+0000 7fe87f596640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe878162f80 con 0x7fe8600039b0 2024-04-15T15:20:06.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.529+0000 7fe87f596640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe8780d2c50 con 0x7fe8781f4e00 2024-04-15T15:20:06.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.529+0000 7fe87f596640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fe8780b78f0 con 0x7fe878160b30 2024-04-15T15:20:06.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.529+0000 7fe87d332640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8600039b0 0x7fe8781617c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:06.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.529+0000 7fe87d332640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8600039b0 0x7fe8781617c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37582/0 (socket says 172.21.15.105:37582) 2024-04-15T15:20:06.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.529+0000 7fe87d332640 1 -- 172.21.15.105:0/169452269 learned_addr learned my addr 172.21.15.105:0/169452269 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:06.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.529+0000 7fe87db33640 1 --2- 172.21.15.105:0/169452269 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8781f4e00 0x7fe8781f5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:06.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.529+0000 7fe87cb31640 1 --2- 172.21.15.105:0/169452269 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe878160b30 0x7fe8781f48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:06.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.529+0000 7fe87d332640 1 -- 172.21.15.105:0/169452269 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe878160b30 msgr2=0x7fe8781f48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:06.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.529+0000 7fe87d332640 1 --2- 172.21.15.105:0/169452269 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe878160b30 0x7fe8781f48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:06.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.529+0000 7fe87d332640 1 -- 172.21.15.105:0/169452269 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8781f4e00 msgr2=0x7fe8781f5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:06.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.529+0000 7fe87d332640 1 --2- 172.21.15.105:0/169452269 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8781f4e00 0x7fe8781f5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:06.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.529+0000 7fe87d332640 1 -- 172.21.15.105:0/169452269 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe878161f30 con 0x7fe8600039b0 2024-04-15T15:20:06.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.530+0000 7fe87d332640 1 --2- 172.21.15.105:0/169452269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8600039b0 0x7fe8781617c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe87002c590 tx=0x7fe870001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:06.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.530+0000 7fe875ffb640 1 -- 172.21.15.105:0/169452269 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe870045070 con 0x7fe8600039b0 2024-04-15T15:20:06.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.530+0000 7fe875ffb640 1 -- 172.21.15.105:0/169452269 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe870004bf0 con 0x7fe8600039b0 2024-04-15T15:20:06.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.530+0000 7fe87f596640 1 -- 172.21.15.105:0/169452269 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8780cd390 con 0x7fe8600039b0 2024-04-15T15:20:06.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.530+0000 7fe875ffb640 1 -- 172.21.15.105:0/169452269 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe870001c90 con 0x7fe8600039b0 2024-04-15T15:20:06.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.530+0000 7fe87f596640 1 -- 172.21.15.105:0/169452269 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8780cd950 con 0x7fe8600039b0 2024-04-15T15:20:06.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.531+0000 7fe87f596640 1 -- 172.21.15.105:0/169452269 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8780d02b0 con 0x7fe8600039b0 2024-04-15T15:20:06.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.531+0000 7fe875ffb640 1 -- 172.21.15.105:0/169452269 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe870007070 con 0x7fe8600039b0 2024-04-15T15:20:06.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.532+0000 7fe875ffb640 1 --2- 172.21.15.105:0/169452269 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fe86406c8f0 0x7fe86406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:06.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.532+0000 7fe875ffb640 1 -- 172.21.15.105:0/169452269 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe8700acbd0 con 0x7fe8600039b0 2024-04-15T15:20:06.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.532+0000 7fe87cb31640 1 --2- 172.21.15.105:0/169452269 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fe86406c8f0 0x7fe86406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:06.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.532+0000 7fe87cb31640 1 --2- 172.21.15.105:0/169452269 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fe86406c8f0 0x7fe86406edb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe8781f7e00 tx=0x7fe86c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:06.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.536+0000 7fe875ffb640 1 -- 172.21.15.105:0/169452269 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe8780d02b0 con 0x7fe8600039b0 2024-04-15T15:20:06.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.733+0000 7fe87f596640 1 -- 172.21.15.105:0/169452269 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe878068a40 con 0x7fe8600039b0 2024-04-15T15:20:06.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.734+0000 7fe875ffb640 1 -- 172.21.15.105:0/169452269 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7fe87007aa90 con 0x7fe8600039b0 2024-04-15T15:20:06.736 INFO:teuthology.orchestra.run.smithi105.stderr:listed 0 entries 2024-04-15T15:20:06.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.736+0000 7fe87f596640 1 -- 172.21.15.105:0/169452269 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fe86406c8f0 msgr2=0x7fe86406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:06.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.736+0000 7fe87f596640 1 --2- 172.21.15.105:0/169452269 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fe86406c8f0 0x7fe86406edb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe8781f7e00 tx=0x7fe86c008040 comp rx=0 tx=0).stop 2024-04-15T15:20:06.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.736+0000 7fe87f596640 1 -- 172.21.15.105:0/169452269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8600039b0 msgr2=0x7fe8781617c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:06.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.736+0000 7fe87f596640 1 --2- 172.21.15.105:0/169452269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8600039b0 0x7fe8781617c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe87002c590 tx=0x7fe870001a40 comp rx=0 tx=0).stop 2024-04-15T15:20:06.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.736+0000 7fe87f596640 1 -- 172.21.15.105:0/169452269 shutdown_connections 2024-04-15T15:20:06.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.736+0000 7fe87f596640 1 --2- 172.21.15.105:0/169452269 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe878160b30 0x7fe8781f48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:06.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.736+0000 7fe87f596640 1 --2- 172.21.15.105:0/169452269 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fe86406c8f0 0x7fe86406edb0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:06.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.736+0000 7fe87f596640 1 --2- 172.21.15.105:0/169452269 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8781f4e00 0x7fe8781f5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:06.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.736+0000 7fe87f596640 1 --2- 172.21.15.105:0/169452269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8600039b0 0x7fe8781617c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:06.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.736+0000 7fe87f596640 1 -- 172.21.15.105:0/169452269 >> 172.21.15.105:0/169452269 conn(0x7fe878067720 msgr2=0x7fe8780b9730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:06.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.736+0000 7fe87f596640 1 -- 172.21.15.105:0/169452269 shutdown_connections 2024-04-15T15:20:06.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.736+0000 7fe87f596640 1 -- 172.21.15.105:0/169452269 wait complete. 2024-04-15T15:20:06.748 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-15T15:20:06.748 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:20:06.768 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:20:06.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.870+0000 7f5dca575640 1 Processor -- start 2024-04-15T15:20:06.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.870+0000 7f5dca575640 1 -- start start 2024-04-15T15:20:06.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.871+0000 7f5dca575640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5dc406c0d0 0x7f5dc4166570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:06.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.871+0000 7f5dca575640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5dc40b3c80 0x7f5dc4168ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:06.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.871+0000 7f5dca575640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5dc40c8270 con 0x7f5dc40d0540 2024-04-15T15:20:06.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.871+0000 7f5dca575640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f5dc40d0920 con 0x7f5dc40b3c80 2024-04-15T15:20:06.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.871+0000 7f5dca575640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f5dc40c7cb0 con 0x7f5dc406c0d0 2024-04-15T15:20:06.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.871+0000 7f5dc8b12640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5dc40b3c80 0x7f5dc4168ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:06.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.871+0000 7f5dc8b12640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5dc40b3c80 0x7f5dc4168ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37594/0 (socket says 172.21.15.105:37594) 2024-04-15T15:20:06.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.871+0000 7f5dc8b12640 1 -- 172.21.15.105:0/1057601816 learned_addr learned my addr 172.21.15.105:0/1057601816 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:06.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.871+0000 7f5dc37fe640 1 --2- 172.21.15.105:0/1057601816 >> v2:172.21.15.154:3300/0 conn(0x7f5dc406c0d0 0x7f5dc4166570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:06.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.871+0000 7f5dc8b12640 1 -- 172.21.15.105:0/1057601816 >> v2:172.21.15.154:3300/0 conn(0x7f5dc406c0d0 msgr2=0x7f5dc4166570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:06.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.871+0000 7f5dc8b12640 1 --2- 172.21.15.105:0/1057601816 >> v2:172.21.15.154:3300/0 conn(0x7f5dc406c0d0 0x7f5dc4166570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:06.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.871+0000 7f5dc8b12640 1 -- 172.21.15.105:0/1057601816 >> v1:172.21.15.154:6789/0 conn(0x7f5dc40d0540 legacy=0x7f5dc406b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:06.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.871+0000 7f5dc8b12640 1 -- 172.21.15.105:0/1057601816 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dc40c7f90 con 0x7f5dc40b3c80 2024-04-15T15:20:06.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.872+0000 7f5dc8b12640 1 --2- 172.21.15.105:0/1057601816 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dc40b3c80 0x7f5dc4168ac0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc009e50 tx=0x7f5dbc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b7b837485faf7cb server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:06.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.872+0000 7f5dc2ffd640 1 -- 172.21.15.105:0/1057601816 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc02ca80 con 0x7f5dc40b3c80 2024-04-15T15:20:06.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.872+0000 7f5dc2ffd640 1 -- 172.21.15.105:0/1057601816 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dbc02cc20 con 0x7f5dc40b3c80 2024-04-15T15:20:06.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.872+0000 7f5dc2ffd640 1 -- 172.21.15.105:0/1057601816 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc0343f0 con 0x7f5dc40b3c80 2024-04-15T15:20:06.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.872+0000 7f5dca575640 1 -- 172.21.15.105:0/1057601816 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dc40b3c80 msgr2=0x7f5dc4168ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:06.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.872+0000 7f5dca575640 1 --2- 172.21.15.105:0/1057601816 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dc40b3c80 0x7f5dc4168ac0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc009e50 tx=0x7f5dbc02bee0 comp rx=0 tx=0).stop 2024-04-15T15:20:06.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.872+0000 7f5dca575640 1 -- 172.21.15.105:0/1057601816 shutdown_connections 2024-04-15T15:20:06.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.872+0000 7f5dca575640 1 --2- 172.21.15.105:0/1057601816 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dc40b3c80 0x7f5dc4168ac0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:06.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.872+0000 7f5dca575640 1 --2- 172.21.15.105:0/1057601816 >> v2:172.21.15.154:3300/0 conn(0x7f5dc406c0d0 0x7f5dc4166570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:06.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.872+0000 7f5dca575640 1 -- 172.21.15.105:0/1057601816 >> 172.21.15.105:0/1057601816 conn(0x7f5dc409aa40 msgr2=0x7f5dc40b30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:06.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.873+0000 7f5dca575640 1 -- 172.21.15.105:0/1057601816 shutdown_connections 2024-04-15T15:20:06.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.873+0000 7f5dca575640 1 -- 172.21.15.105:0/1057601816 wait complete. 2024-04-15T15:20:06.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.874+0000 7f5dca575640 1 Processor -- start 2024-04-15T15:20:06.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.874+0000 7f5dca575640 1 -- start start 2024-04-15T15:20:06.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.875+0000 7f5dca575640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5dc406c0d0 0x7f5dc4159690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:06.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.875+0000 7f5dca575640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5dc40b3c80 0x7f5dc4160a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:06.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.875+0000 7f5dca575640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dc40d0540 0x7f5dc414b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:06.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.875+0000 7f5dca575640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc4070e40 con 0x7f5dc40d0540 2024-04-15T15:20:06.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.875+0000 7f5dca575640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc406f180 con 0x7f5dc406c0d0 2024-04-15T15:20:06.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.875+0000 7f5dca575640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f5dc4070fc0 con 0x7f5dc40b3c80 2024-04-15T15:20:06.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.875+0000 7f5dc8b12640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dc40d0540 0x7f5dc414b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:06.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.875+0000 7f5dc8b12640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dc40d0540 0x7f5dc414b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37600/0 (socket says 172.21.15.105:37600) 2024-04-15T15:20:06.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.875+0000 7f5dc8b12640 1 -- 172.21.15.105:0/2282866709 learned_addr learned my addr 172.21.15.105:0/2282866709 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:06.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.875+0000 7f5dc3fff640 1 --2- 172.21.15.105:0/2282866709 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5dc406c0d0 0x7f5dc4159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:06.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.875+0000 7f5dc37fe640 1 --2- 172.21.15.105:0/2282866709 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5dc40b3c80 0x7f5dc4160a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:06.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.875+0000 7f5dc8b12640 1 -- 172.21.15.105:0/2282866709 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5dc40b3c80 msgr2=0x7f5dc4160a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:06.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.875+0000 7f5dc8b12640 1 --2- 172.21.15.105:0/2282866709 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5dc40b3c80 0x7f5dc4160a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:06.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.875+0000 7f5dc8b12640 1 -- 172.21.15.105:0/2282866709 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5dc406c0d0 msgr2=0x7f5dc4159690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:06.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.875+0000 7f5dc8b12640 1 --2- 172.21.15.105:0/2282866709 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5dc406c0d0 0x7f5dc4159690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:06.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.875+0000 7f5dc8b12640 1 -- 172.21.15.105:0/2282866709 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dc4149080 con 0x7f5dc40d0540 2024-04-15T15:20:06.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.875+0000 7f5dc3fff640 1 --2- 172.21.15.105:0/2282866709 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5dc406c0d0 0x7f5dc4159690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:06.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.875+0000 7f5dc8b12640 1 --2- 172.21.15.105:0/2282866709 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dc40d0540 0x7f5dc414b3f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc009e50 tx=0x7f5dbc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:06.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.875+0000 7f5dc0ff9640 1 -- 172.21.15.105:0/2282866709 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc043020 con 0x7f5dc40d0540 2024-04-15T15:20:06.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.875+0000 7f5dc0ff9640 1 -- 172.21.15.105:0/2282866709 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dbc03e450 con 0x7f5dc40d0540 2024-04-15T15:20:06.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.875+0000 7f5dca575640 1 -- 172.21.15.105:0/2282866709 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dc4162230 con 0x7f5dc40d0540 2024-04-15T15:20:06.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.875+0000 7f5dc0ff9640 1 -- 172.21.15.105:0/2282866709 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc009510 con 0x7f5dc40d0540 2024-04-15T15:20:06.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.875+0000 7f5dca575640 1 -- 172.21.15.105:0/2282866709 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dc4161470 con 0x7f5dc40d0540 2024-04-15T15:20:06.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.876+0000 7f5dca575640 1 -- 172.21.15.105:0/2282866709 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dc4068a40 con 0x7f5dc40d0540 2024-04-15T15:20:06.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.877+0000 7f5dc0ff9640 1 -- 172.21.15.105:0/2282866709 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5dbc005d10 con 0x7f5dc40d0540 2024-04-15T15:20:06.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.877+0000 7f5dc0ff9640 1 --2- 172.21.15.105:0/2282866709 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f5db006c8f0 0x7f5db006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:06.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.877+0000 7f5dc0ff9640 1 -- 172.21.15.105:0/2282866709 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5825+0+0 (secure 0 0 0) 0x7f5dbc0ae670 con 0x7f5dc40d0540 2024-04-15T15:20:06.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.877+0000 7f5dc3fff640 1 --2- 172.21.15.105:0/2282866709 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f5db006c8f0 0x7f5db006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:06.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.878+0000 7f5dc3fff640 1 --2- 172.21.15.105:0/2282866709 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f5db006c8f0 0x7f5db006edb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5db4003eb0 tx=0x7f5db4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:06.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:06.882+0000 7f5dc0ff9640 1 -- 172.21.15.105:0/2282866709 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5dc4149080 con 0x7f5dc40d0540 2024-04-15T15:20:07.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.059+0000 7f5dca575640 1 -- 172.21.15.105:0/2282866709 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5dc406fae0 con 0x7f5dc40d0540 2024-04-15T15:20:07.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.060+0000 7f5dc0ff9640 1 -- 172.21.15.105:0/2282866709 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16901 (secure 0 0 0) 0x7f5dbc07cab0 con 0x7f5dc40d0540 2024-04-15T15:20:07.062 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:20:07.062 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":21,"fsid":"09e4d2a4-a324-4793-af77-644b25e520e1","created":"2024-04-15T15:19:01.524543+0000","modified":"2024-04-15T15:20:06.550842+0000","last_up_change":"2024-04-15T15:19:12.825815+0000","last_in_change":"2024-04-15T15:19:03.788126+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:19:13.748585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:19:23.789287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:19:24.722250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6f6292b-2d7e-4aa0-9d1c-6ddd0f97287c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6801","nonce":2873040054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6803","nonce":2873040054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6807","nonce":2873040054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6805","nonce":2873040054}]},"public_addr":"172.21.15.105:6801/2873040054","cluster_addr":"172.21.15.105:6803/2873040054","heartbeat_back_addr":"172.21.15.105:6807/2873040054","heartbeat_front_addr":"172.21.15.105:6805/2873040054","state":["exists","up"]},{"osd":1,"uuid":"8900966a-b393-41e3-8f93-e5a43bd23622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6818","nonce":2494994479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6821","nonce":2494994479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6828","nonce":2494994479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6823","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6825","nonce":2494994479}]},"public_addr":"172.21.15.105:6818/2494994479","cluster_addr":"172.21.15.105:6821/2494994479","heartbeat_back_addr":"172.21.15.105:6828/2494994479","heartbeat_front_addr":"172.21.15.105:6825/2494994479","state":["exists","up"]},{"osd":2,"uuid":"81a5dc7a-9325-4ff5-941d-a2d5fba21f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6809","nonce":3798408699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6811","nonce":3798408699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6815","nonce":3798408699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6813","nonce":3798408699}]},"public_addr":"172.21.15.105:6809/3798408699","cluster_addr":"172.21.15.105:6811/3798408699","heartbeat_back_addr":"172.21.15.105:6815/3798408699","heartbeat_front_addr":"172.21.15.105:6813/3798408699","state":["exists","up"]},{"osd":3,"uuid":"df766ed2-5c71-4805-96f4-3d4970edc960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6819","nonce":1920720186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6824","nonce":1920720186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6831","nonce":1920720186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6827","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6829","nonce":1920720186}]},"public_addr":"172.21.15.105:6819/1920720186","cluster_addr":"172.21.15.105:6824/1920720186","heartbeat_back_addr":"172.21.15.105:6831/1920720186","heartbeat_front_addr":"172.21.15.105:6829/1920720186","state":["exists","up"]},{"osd":4,"uuid":"e1039e03-28b8-49df-b951-c91930b93c9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6810","nonce":3516494456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6813","nonce":3516494456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6820","nonce":3516494456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6817","nonce":3516494456}]},"public_addr":"172.21.15.154:6810/3516494456","cluster_addr":"172.21.15.154:6813/3516494456","heartbeat_back_addr":"172.21.15.154:6820/3516494456","heartbeat_front_addr":"172.21.15.154:6817/3516494456","state":["exists","up"]},{"osd":5,"uuid":"a9f5eaf1-66ff-405a-a9e4-ec3340812a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6801","nonce":540367751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6803","nonce":540367751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6807","nonce":540367751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6805","nonce":540367751}]},"public_addr":"172.21.15.154:6801/540367751","cluster_addr":"172.21.15.154:6803/540367751","heartbeat_back_addr":"172.21.15.154:6807/540367751","heartbeat_front_addr":"172.21.15.154:6805/540367751","state":["exists","up"]},{"osd":6,"uuid":"267d02ea-675f-428f-aae8-e0375e4d4feb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6825","nonce":1550829005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6827","nonce":1550829005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6831","nonce":1550829005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6829","nonce":1550829005}]},"public_addr":"172.21.15.154:6825/1550829005","cluster_addr":"172.21.15.154:6827/1550829005","heartbeat_back_addr":"172.21.15.154:6831/1550829005","heartbeat_front_addr":"172.21.15.154:6829/1550829005","state":["exists","up"]},{"osd":7,"uuid":"759ac6cc-896a-423c-885f-9fe90d8dcb14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6809","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6811","nonce":3920089903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6816","nonce":3920089903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6823","nonce":3920089903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6819","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6821","nonce":3920089903}]},"public_addr":"172.21.15.154:6811/3920089903","cluster_addr":"172.21.15.154:6816/3920089903","heartbeat_back_addr":"172.21.15.154:6823/3920089903","heartbeat_front_addr":"172.21.15.154:6821/3920089903","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.949509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.984072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.907278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.947391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.975447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.998896+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:09.028739+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.894119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":1}]},{"pgid":"3.30","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:20:07.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.062+0000 7f5dca575640 1 -- 172.21.15.105:0/2282866709 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f5db006c8f0 msgr2=0x7f5db006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:07.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.062+0000 7f5dca575640 1 --2- 172.21.15.105:0/2282866709 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f5db006c8f0 0x7f5db006edb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5db4003eb0 tx=0x7f5db4009040 comp rx=0 tx=0).stop 2024-04-15T15:20:07.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.062+0000 7f5dca575640 1 -- 172.21.15.105:0/2282866709 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dc40d0540 msgr2=0x7f5dc414b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:07.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.062+0000 7f5dca575640 1 --2- 172.21.15.105:0/2282866709 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dc40d0540 0x7f5dc414b3f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc009e50 tx=0x7f5dbc004a40 comp rx=0 tx=0).stop 2024-04-15T15:20:07.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.062+0000 7f5dca575640 1 -- 172.21.15.105:0/2282866709 shutdown_connections 2024-04-15T15:20:07.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.062+0000 7f5dca575640 1 --2- 172.21.15.105:0/2282866709 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5dc40b3c80 0x7f5dc4160a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.062+0000 7f5dca575640 1 --2- 172.21.15.105:0/2282866709 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f5db006c8f0 0x7f5db006edb0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.062+0000 7f5dca575640 1 --2- 172.21.15.105:0/2282866709 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dc40d0540 0x7f5dc414b3f0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.062+0000 7f5dca575640 1 --2- 172.21.15.105:0/2282866709 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5dc406c0d0 0x7f5dc4159690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.062+0000 7f5dca575640 1 -- 172.21.15.105:0/2282866709 >> 172.21.15.105:0/2282866709 conn(0x7f5dc409aa40 msgr2=0x7f5dc40d1710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:07.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.062+0000 7f5dca575640 1 -- 172.21.15.105:0/2282866709 shutdown_connections 2024-04-15T15:20:07.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.062+0000 7f5dca575640 1 -- 172.21.15.105:0/2282866709 wait complete. 2024-04-15T15:20:07.075 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:19:13.748585+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-15T15:19:23.789287+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-15T15:19:24.722250+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:20:07.075 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:20:07.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.180+0000 7fb58873d640 1 Processor -- start 2024-04-15T15:20:07.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.180+0000 7fb58873d640 1 -- start start 2024-04-15T15:20:07.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.181+0000 7fb58873d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb5800cd8e0 0x7fb5800b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:07.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.181+0000 7fb58873d640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5800b40a0 con 0x7fb58006f6a0 2024-04-15T15:20:07.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.181+0000 7fb58873d640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5800b4380 con 0x7fb580144c90 2024-04-15T15:20:07.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.181+0000 7fb58873d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fb5800b3e50 con 0x7fb5800cd8e0 2024-04-15T15:20:07.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.181+0000 7fb585cd8640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fb58006f6a0 0x7fb5800cd1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:55190/0 (socket says 172.21.15.105:55190) 2024-04-15T15:20:07.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.181+0000 7fb585cd8640 1 -- 172.21.15.105:0/2567809786 learned_addr learned my addr 172.21.15.105:0/2567809786 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:20:07.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.181+0000 7fb5854d7640 1 -- 172.21.15.105:0/2567809786 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1203368258 0 0) 0x7fb5800b40a0 con 0x7fb58006f6a0 2024-04-15T15:20:07.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.181+0000 7fb5854d7640 1 -- 172.21.15.105:0/2567809786 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb56c003520 con 0x7fb58006f6a0 2024-04-15T15:20:07.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.181+0000 7fb586cda640 1 --2- 172.21.15.105:0/2567809786 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb5800cd8e0 0x7fb5800b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:07.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.181+0000 7fb5854d7640 1 -- 172.21.15.105:0/2567809786 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3421472085 0 0) 0x7fb56c003520 con 0x7fb58006f6a0 2024-04-15T15:20:07.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.181+0000 7fb5854d7640 1 -- 172.21.15.105:0/2567809786 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb5800cd8e0 msgr2=0x7fb5800b4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:07.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.181+0000 7fb5854d7640 1 --2- 172.21.15.105:0/2567809786 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb5800cd8e0 0x7fb5800b4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.181+0000 7fb5854d7640 1 -- 172.21.15.105:0/2567809786 >> v1:172.21.15.154:6789/0 conn(0x7fb580144c90 legacy=0x7fb58006ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:07.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.181+0000 7fb5854d7640 1 -- 172.21.15.105:0/2567809786 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5800b4660 con 0x7fb58006f6a0 2024-04-15T15:20:07.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.181+0000 7fb5854d7640 1 -- 172.21.15.105:0/2567809786 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7fb570001670 con 0x7fb58006f6a0 2024-04-15T15:20:07.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.181+0000 7fb5854d7640 1 -- 172.21.15.105:0/2567809786 >> v1:172.21.15.105:6789/0 conn(0x7fb58006f6a0 legacy=0x7fb5800cd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:07.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.181+0000 7fb5854d7640 1 --2- 172.21.15.105:0/2567809786 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb56c0039e0 0x7fb56c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:07.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.181+0000 7fb5854d7640 1 --2- 172.21.15.105:0/2567809786 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb5800cd8e0 0x7fb56c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:07.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.181+0000 7fb5854d7640 1 --2- 172.21.15.105:0/2567809786 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb56c004d40 0x7fb56c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:07.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.181+0000 7fb5854d7640 1 -- 172.21.15.105:0/2567809786 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb5800b3e50 con 0x7fb56c004d40 2024-04-15T15:20:07.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.181+0000 7fb5854d7640 1 -- 172.21.15.105:0/2567809786 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb56c00bba0 con 0x7fb56c0039e0 2024-04-15T15:20:07.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.181+0000 7fb5854d7640 1 -- 172.21.15.105:0/2567809786 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fb56c00bda0 con 0x7fb5800cd8e0 2024-04-15T15:20:07.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.181+0000 7fb586cda640 1 --2- 172.21.15.105:0/2567809786 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb56c004d40 0x7fb56c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:07.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.182+0000 7fb585cd8640 1 --2- 172.21.15.105:0/2567809786 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb5800cd8e0 0x7fb56c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:07.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.182+0000 7fb5864d9640 1 --2- 172.21.15.105:0/2567809786 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb56c0039e0 0x7fb56c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:07.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.182+0000 7fb586cda640 1 -- 172.21.15.105:0/2567809786 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb5800cd8e0 msgr2=0x7fb56c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:07.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.182+0000 7fb586cda640 1 --2- 172.21.15.105:0/2567809786 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb5800cd8e0 0x7fb56c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.182+0000 7fb586cda640 1 -- 172.21.15.105:0/2567809786 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb56c0039e0 msgr2=0x7fb56c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:07.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.182+0000 7fb586cda640 1 --2- 172.21.15.105:0/2567809786 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb56c0039e0 0x7fb56c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.182+0000 7fb586cda640 1 -- 172.21.15.105:0/2567809786 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb56c003520 con 0x7fb56c004d40 2024-04-15T15:20:07.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.182+0000 7fb586cda640 1 --2- 172.21.15.105:0/2567809786 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb56c004d40 0x7fb56c0051c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb578007360 tx=0x7fb56c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1e2b5dd2e9c10a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:07.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.183+0000 7fb5854d7640 1 -- 172.21.15.105:0/2567809786 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb578039070 con 0x7fb56c004d40 2024-04-15T15:20:07.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.183+0000 7fb5854d7640 1 -- 172.21.15.105:0/2567809786 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb57802ccd0 con 0x7fb56c004d40 2024-04-15T15:20:07.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.183+0000 7fb5854d7640 1 -- 172.21.15.105:0/2567809786 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb578009be0 con 0x7fb56c004d40 2024-04-15T15:20:07.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.183+0000 7fb58873d640 1 -- 172.21.15.105:0/2567809786 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb56c004d40 msgr2=0x7fb56c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:07.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.183+0000 7fb58873d640 1 --2- 172.21.15.105:0/2567809786 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb56c004d40 0x7fb56c0051c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb578007360 tx=0x7fb56c005830 comp rx=0 tx=0).stop 2024-04-15T15:20:07.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.183+0000 7fb586cda640 1 -- 172.21.15.105:0/2567809786 reap_dead start 2024-04-15T15:20:07.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.183+0000 7fb58873d640 1 -- 172.21.15.105:0/2567809786 shutdown_connections 2024-04-15T15:20:07.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.183+0000 7fb58873d640 1 -- 172.21.15.105:0/2567809786 >> 172.21.15.105:0/2567809786 conn(0x7fb580067d50 msgr2=0x7fb58006b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:07.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.183+0000 7fb58873d640 1 -- 172.21.15.105:0/2567809786 shutdown_connections 2024-04-15T15:20:07.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.183+0000 7fb58873d640 1 -- 172.21.15.105:0/2567809786 wait complete. 2024-04-15T15:20:07.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.184+0000 7fb58873d640 1 Processor -- start 2024-04-15T15:20:07.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.184+0000 7fb58873d640 1 -- start start 2024-04-15T15:20:07.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.185+0000 7fb58873d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb56c004d40 0x7fb58013ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:07.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.185+0000 7fb58873d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb58013e2d0 0x7fb5800c7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:07.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.185+0000 7fb58873d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb58006de80 0x7fb58006e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:07.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.185+0000 7fb58873d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb5801451f0 con 0x7fb58006de80 2024-04-15T15:20:07.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.185+0000 7fb58873d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb580145070 con 0x7fb58013e2d0 2024-04-15T15:20:07.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.185+0000 7fb58873d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fb5800b4e60 con 0x7fb56c004d40 2024-04-15T15:20:07.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.185+0000 7fb586cda640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb58006de80 0x7fb58006e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:07.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.185+0000 7fb586cda640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb58006de80 0x7fb58006e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37618/0 (socket says 172.21.15.105:37618) 2024-04-15T15:20:07.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.185+0000 7fb586cda640 1 -- 172.21.15.105:0/279250406 learned_addr learned my addr 172.21.15.105:0/279250406 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:07.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.185+0000 7fb586cda640 1 -- 172.21.15.105:0/279250406 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb56c004d40 msgr2=0x7fb58013ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:07.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.185+0000 7fb586cda640 1 --2- 172.21.15.105:0/279250406 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb56c004d40 0x7fb58013ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.185+0000 7fb585cd8640 1 --2- 172.21.15.105:0/279250406 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb58013e2d0 0x7fb5800c7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:07.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.185+0000 7fb586cda640 1 -- 172.21.15.105:0/279250406 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb58013e2d0 msgr2=0x7fb5800c7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:07.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.185+0000 7fb586cda640 1 --2- 172.21.15.105:0/279250406 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb58013e2d0 0x7fb5800c7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.185+0000 7fb586cda640 1 -- 172.21.15.105:0/279250406 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb58013b390 con 0x7fb58006de80 2024-04-15T15:20:07.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.186+0000 7fb586cda640 1 --2- 172.21.15.105:0/279250406 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb58006de80 0x7fb58006e300 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb57802c740 tx=0x7fb578004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:07.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.186+0000 7fb576ffd640 1 -- 172.21.15.105:0/279250406 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb578039070 con 0x7fb58006de80 2024-04-15T15:20:07.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.186+0000 7fb576ffd640 1 -- 172.21.15.105:0/279250406 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb57803a040 con 0x7fb58006de80 2024-04-15T15:20:07.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.186+0000 7fb576ffd640 1 -- 172.21.15.105:0/279250406 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb578009be0 con 0x7fb58006de80 2024-04-15T15:20:07.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.186+0000 7fb58873d640 1 -- 172.21.15.105:0/279250406 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5800c8b50 con 0x7fb58006de80 2024-04-15T15:20:07.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.186+0000 7fb58873d640 1 -- 172.21.15.105:0/279250406 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5800c82b0 con 0x7fb58006de80 2024-04-15T15:20:07.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.187+0000 7fb58873d640 1 -- 172.21.15.105:0/279250406 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb58006e840 con 0x7fb58006de80 2024-04-15T15:20:07.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.187+0000 7fb576ffd640 1 -- 172.21.15.105:0/279250406 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb578046020 con 0x7fb58006de80 2024-04-15T15:20:07.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.188+0000 7fb576ffd640 1 --2- 172.21.15.105:0/279250406 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fb56406c920 0x7fb56406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:07.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.188+0000 7fb576ffd640 1 -- 172.21.15.105:0/279250406 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5825+0+0 (secure 0 0 0) 0x7fb5780ad2d0 con 0x7fb58006de80 2024-04-15T15:20:07.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.188+0000 7fb5864d9640 1 --2- 172.21.15.105:0/279250406 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fb56406c920 0x7fb56406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:07.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.188+0000 7fb5864d9640 1 --2- 172.21.15.105:0/279250406 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fb56406c920 0x7fb56406ede0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb57c001750 tx=0x7fb57c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:07.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.192+0000 7fb576ffd640 1 -- 172.21.15.105:0/279250406 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb57804d020 con 0x7fb58006de80 2024-04-15T15:20:07.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.368+0000 7fb58873d640 1 -- 172.21.15.105:0/279250406 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb5800b2f40 con 0x7fb58006de80 2024-04-15T15:20:07.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.368+0000 7fb576ffd640 1 -- 172.21.15.105:0/279250406 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7fb58013b390 con 0x7fb58006de80 2024-04-15T15:20:07.370 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-04-15T15:20:07.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.370+0000 7fb58873d640 1 -- 172.21.15.105:0/279250406 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fb56406c920 msgr2=0x7fb56406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:07.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.370+0000 7fb58873d640 1 --2- 172.21.15.105:0/279250406 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fb56406c920 0x7fb56406ede0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb57c001750 tx=0x7fb57c009040 comp rx=0 tx=0).stop 2024-04-15T15:20:07.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.370+0000 7fb58873d640 1 -- 172.21.15.105:0/279250406 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb58006de80 msgr2=0x7fb58006e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:07.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.370+0000 7fb58873d640 1 --2- 172.21.15.105:0/279250406 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb58006de80 0x7fb58006e300 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb57802c740 tx=0x7fb578004ab0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.370+0000 7fb58873d640 1 -- 172.21.15.105:0/279250406 shutdown_connections 2024-04-15T15:20:07.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.370+0000 7fb58873d640 1 --2- 172.21.15.105:0/279250406 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fb56406c920 0x7fb56406ede0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.370+0000 7fb58873d640 1 --2- 172.21.15.105:0/279250406 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb58006de80 0x7fb58006e300 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.370+0000 7fb58873d640 1 --2- 172.21.15.105:0/279250406 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb58013e2d0 0x7fb5800c7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.370+0000 7fb58873d640 1 --2- 172.21.15.105:0/279250406 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb56c004d40 0x7fb58013ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.370+0000 7fb58873d640 1 -- 172.21.15.105:0/279250406 >> 172.21.15.105:0/279250406 conn(0x7fb580067d50 msgr2=0x7fb580143910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:07.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.370+0000 7fb58873d640 1 -- 172.21.15.105:0/279250406 shutdown_connections 2024-04-15T15:20:07.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.370+0000 7fb58873d640 1 -- 172.21.15.105:0/279250406 wait complete. 2024-04-15T15:20:07.383 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-15T15:20:07.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.488+0000 7eff8898f640 1 Processor -- start 2024-04-15T15:20:07.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.488+0000 7eff8898f640 1 -- start start 2024-04-15T15:20:07.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.489+0000 7eff8898f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7eff8415e8b0 0x7eff8409aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:07.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.489+0000 7eff8898f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7eff84070780 0x7eff8406de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:07.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.489+0000 7eff8898f640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff840d0870 con 0x7eff8409af80 2024-04-15T15:20:07.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.489+0000 7eff8898f640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7eff8415f280 con 0x7eff8415e8b0 2024-04-15T15:20:07.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.489+0000 7eff8898f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7eff8409b3c0 con 0x7eff84070780 2024-04-15T15:20:07.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.489+0000 7eff8259c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7eff8415e8b0 0x7eff8409aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:07.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.489+0000 7eff8259c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7eff8415e8b0 0x7eff8409aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37630/0 (socket says 172.21.15.105:37630) 2024-04-15T15:20:07.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.489+0000 7eff8259c640 1 -- 172.21.15.105:0/1486065979 learned_addr learned my addr 172.21.15.105:0/1486065979 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:07.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.489+0000 7eff81d9b640 1 --1- 172.21.15.105:0/1486065979 >> v1:172.21.15.105:6789/0 conn(0x7eff8409af80 0x7eff84070070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:55194/0 (socket says 172.21.15.105:55194) 2024-04-15T15:20:07.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.489+0000 7eff82d9d640 1 --2- 172.21.15.105:0/1486065979 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7eff84070780 0x7eff8406de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:07.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.489+0000 7eff8259c640 1 -- 172.21.15.105:0/1486065979 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7eff84070780 msgr2=0x7eff8406de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:07.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.489+0000 7eff8259c640 1 --2- 172.21.15.105:0/1486065979 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7eff84070780 0x7eff8406de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.489+0000 7eff8259c640 1 -- 172.21.15.105:0/1486065979 >> v1:172.21.15.105:6789/0 conn(0x7eff8409af80 legacy=0x7eff84070070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:07.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.489+0000 7eff8259c640 1 -- 172.21.15.105:0/1486065979 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff840d0af0 con 0x7eff8415e8b0 2024-04-15T15:20:07.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.489+0000 7eff8159a640 1 -- 172.21.15.105:0/1486065979 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 769690395 0 0) 0x7eff78001110 con 0x7eff8409af80 2024-04-15T15:20:07.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.490+0000 7eff8259c640 1 --2- 172.21.15.105:0/1486065979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff8415e8b0 0x7eff8409aa40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7eff74009870 tx=0x7eff7402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfa11802f3dddf55 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:07.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.490+0000 7eff8159a640 1 -- 172.21.15.105:0/1486065979 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff74034070 con 0x7eff8415e8b0 2024-04-15T15:20:07.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.490+0000 7eff8159a640 1 -- 172.21.15.105:0/1486065979 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff74030840 con 0x7eff8415e8b0 2024-04-15T15:20:07.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.490+0000 7eff8159a640 1 -- 172.21.15.105:0/1486065979 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff74030c10 con 0x7eff8415e8b0 2024-04-15T15:20:07.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.491+0000 7eff8898f640 1 -- 172.21.15.105:0/1486065979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff8415e8b0 msgr2=0x7eff8409aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:07.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.491+0000 7eff8898f640 1 --2- 172.21.15.105:0/1486065979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff8415e8b0 0x7eff8409aa40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7eff74009870 tx=0x7eff7402b980 comp rx=0 tx=0).stop 2024-04-15T15:20:07.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.491+0000 7eff8898f640 1 -- 172.21.15.105:0/1486065979 shutdown_connections 2024-04-15T15:20:07.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.491+0000 7eff8898f640 1 --2- 172.21.15.105:0/1486065979 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7eff84070780 0x7eff8406de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.491+0000 7eff8898f640 1 --2- 172.21.15.105:0/1486065979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff8415e8b0 0x7eff8409aa40 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.491+0000 7eff8898f640 1 -- 172.21.15.105:0/1486065979 >> 172.21.15.105:0/1486065979 conn(0x7eff840b4090 msgr2=0x7eff840b44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:07.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.491+0000 7eff8898f640 1 -- 172.21.15.105:0/1486065979 shutdown_connections 2024-04-15T15:20:07.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.491+0000 7eff8898f640 1 -- 172.21.15.105:0/1486065979 wait complete. 2024-04-15T15:20:07.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.492+0000 7eff8898f640 1 Processor -- start 2024-04-15T15:20:07.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.492+0000 7eff8898f640 1 -- start start 2024-04-15T15:20:07.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.493+0000 7eff8898f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7eff84070780 0x7eff841593a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:07.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.493+0000 7eff8898f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff8409af80 0x7eff840c7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:07.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.493+0000 7eff8898f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff8415e8b0 0x7eff840b0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:07.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.493+0000 7eff8898f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7eff84070f30 con 0x7eff8415e8b0 2024-04-15T15:20:07.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.493+0000 7eff8898f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7eff84070c50 con 0x7eff8409af80 2024-04-15T15:20:07.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.493+0000 7eff8898f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7eff84071210 con 0x7eff84070780 2024-04-15T15:20:07.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.493+0000 7eff82d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff8415e8b0 0x7eff840b0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:07.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.493+0000 7eff82d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff8415e8b0 0x7eff840b0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37640/0 (socket says 172.21.15.105:37640) 2024-04-15T15:20:07.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.493+0000 7eff82d9d640 1 -- 172.21.15.105:0/3288425934 learned_addr learned my addr 172.21.15.105:0/3288425934 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:07.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.493+0000 7eff8259c640 1 --2- 172.21.15.105:0/3288425934 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7eff84070780 0x7eff841593a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:07.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.493+0000 7eff81d9b640 1 --2- 172.21.15.105:0/3288425934 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff8409af80 0x7eff840c7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:07.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.493+0000 7eff82d9d640 1 -- 172.21.15.105:0/3288425934 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7eff84070780 msgr2=0x7eff841593a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:07.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.493+0000 7eff82d9d640 1 --2- 172.21.15.105:0/3288425934 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7eff84070780 0x7eff841593a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.493+0000 7eff82d9d640 1 -- 172.21.15.105:0/3288425934 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff8409af80 msgr2=0x7eff840c7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:07.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.493+0000 7eff82d9d640 1 --2- 172.21.15.105:0/3288425934 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff8409af80 0x7eff840c7b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.493+0000 7eff82d9d640 1 -- 172.21.15.105:0/3288425934 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff84160a00 con 0x7eff8415e8b0 2024-04-15T15:20:07.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.494+0000 7eff82d9d640 1 --2- 172.21.15.105:0/3288425934 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff8415e8b0 0x7eff840b0900 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7eff7c00ef90 tx=0x7eff7c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:07.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.494+0000 7eff72ffd640 1 -- 172.21.15.105:0/3288425934 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7c018020 con 0x7eff8415e8b0 2024-04-15T15:20:07.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.494+0000 7eff72ffd640 1 -- 172.21.15.105:0/3288425934 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff7c014480 con 0x7eff8415e8b0 2024-04-15T15:20:07.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.494+0000 7eff8898f640 1 -- 172.21.15.105:0/3288425934 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff84160ce0 con 0x7eff8415e8b0 2024-04-15T15:20:07.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.494+0000 7eff72ffd640 1 -- 172.21.15.105:0/3288425934 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7c014870 con 0x7eff8415e8b0 2024-04-15T15:20:07.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.494+0000 7eff8898f640 1 -- 172.21.15.105:0/3288425934 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff841612a0 con 0x7eff8415e8b0 2024-04-15T15:20:07.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.495+0000 7eff8898f640 1 -- 172.21.15.105:0/3288425934 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff84068a40 con 0x7eff8415e8b0 2024-04-15T15:20:07.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.495+0000 7eff72ffd640 1 -- 172.21.15.105:0/3288425934 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7eff7c005d10 con 0x7eff8415e8b0 2024-04-15T15:20:07.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.496+0000 7eff72ffd640 1 --2- 172.21.15.105:0/3288425934 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7eff6406c920 0x7eff6406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:07.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.496+0000 7eff72ffd640 1 -- 172.21.15.105:0/3288425934 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5825+0+0 (secure 0 0 0) 0x7eff7c08cb00 con 0x7eff8415e8b0 2024-04-15T15:20:07.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.496+0000 7eff8259c640 1 --2- 172.21.15.105:0/3288425934 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7eff6406c920 0x7eff6406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:07.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.496+0000 7eff8259c640 1 --2- 172.21.15.105:0/3288425934 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7eff6406c920 0x7eff6406ede0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7eff74036000 tx=0x7eff7403b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:07.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.500+0000 7eff72ffd640 1 -- 172.21.15.105:0/3288425934 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7eff7c022030 con 0x7eff8415e8b0 2024-04-15T15:20:07.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.677+0000 7eff8898f640 1 -- 172.21.15.105:0/3288425934 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7eff8415dec0 con 0x7eff8415e8b0 2024-04-15T15:20:07.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.678+0000 7eff72ffd640 1 -- 172.21.15.105:0/3288425934 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7eff7c010040 con 0x7eff8415e8b0 2024-04-15T15:20:07.680 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-04-15T15:20:07.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.679+0000 7eff8898f640 1 -- 172.21.15.105:0/3288425934 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7eff6406c920 msgr2=0x7eff6406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:07.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.679+0000 7eff8898f640 1 --2- 172.21.15.105:0/3288425934 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7eff6406c920 0x7eff6406ede0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7eff74036000 tx=0x7eff7403b040 comp rx=0 tx=0).stop 2024-04-15T15:20:07.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.679+0000 7eff8898f640 1 -- 172.21.15.105:0/3288425934 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff8415e8b0 msgr2=0x7eff840b0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:07.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.679+0000 7eff8898f640 1 --2- 172.21.15.105:0/3288425934 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff8415e8b0 0x7eff840b0900 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7eff7c00ef90 tx=0x7eff7c00c490 comp rx=0 tx=0).stop 2024-04-15T15:20:07.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.679+0000 7eff8898f640 1 -- 172.21.15.105:0/3288425934 shutdown_connections 2024-04-15T15:20:07.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.679+0000 7eff8898f640 1 --2- 172.21.15.105:0/3288425934 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7eff6406c920 0x7eff6406ede0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.680+0000 7eff8898f640 1 --2- 172.21.15.105:0/3288425934 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff8415e8b0 0x7eff840b0900 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.680+0000 7eff8898f640 1 --2- 172.21.15.105:0/3288425934 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff8409af80 0x7eff840c7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.680+0000 7eff8898f640 1 --2- 172.21.15.105:0/3288425934 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7eff84070780 0x7eff841593a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.680+0000 7eff8898f640 1 -- 172.21.15.105:0/3288425934 >> 172.21.15.105:0/3288425934 conn(0x7eff840b4090 msgr2=0x7eff8406e180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:07.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.680+0000 7eff8898f640 1 -- 172.21.15.105:0/3288425934 shutdown_connections 2024-04-15T15:20:07.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.680+0000 7eff8898f640 1 -- 172.21.15.105:0/3288425934 wait complete. 2024-04-15T15:20:07.693 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-15T15:20:07.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.797+0000 7f73f3a7b640 1 Processor -- start 2024-04-15T15:20:07.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.797+0000 7f73f3a7b640 1 -- start start 2024-04-15T15:20:07.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.798+0000 7f73f3a7b640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f73ec144c90 0x7f73ec0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:07.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.798+0000 7f73f3a7b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f73ec06ef90 0x7f73ec06f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:07.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.798+0000 7f73f3a7b640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73ec0b40a0 con 0x7f73ec0cd1d0 2024-04-15T15:20:07.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.798+0000 7f73f3a7b640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f73ec0b3e50 con 0x7f73ec144c90 2024-04-15T15:20:07.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.798+0000 7f73f3a7b640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f73ec06f910 con 0x7f73ec06ef90 2024-04-15T15:20:07.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.798+0000 7f73f2018640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f73ec0cd1d0 0x7f73ec0cd600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:55196/0 (socket says 172.21.15.105:55196) 2024-04-15T15:20:07.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.798+0000 7f73f2018640 1 -- 172.21.15.105:0/498950090 learned_addr learned my addr 172.21.15.105:0/498950090 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:20:07.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.799+0000 7f73f0815640 1 -- 172.21.15.105:0/498950090 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 111371081 0 0) 0x7f73ec0b40a0 con 0x7f73ec0cd1d0 2024-04-15T15:20:07.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.799+0000 7f73f1817640 1 --2- 172.21.15.105:0/498950090 >> v2:172.21.15.154:3300/0 conn(0x7f73ec144c90 0x7f73ec0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:07.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.799+0000 7f73f0815640 1 -- 172.21.15.105:0/498950090 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f73d4003520 con 0x7f73ec0cd1d0 2024-04-15T15:20:07.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.799+0000 7f73f1016640 1 --2- 172.21.15.105:0/498950090 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f73ec06ef90 0x7f73ec06f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:07.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.799+0000 7f73f0815640 1 -- 172.21.15.105:0/498950090 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2288360038 0 0) 0x7f73d4003520 con 0x7f73ec0cd1d0 2024-04-15T15:20:07.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.799+0000 7f73f0815640 1 -- 172.21.15.105:0/498950090 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f73ec06ef90 msgr2=0x7f73ec06f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:07.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.799+0000 7f73f0815640 1 --2- 172.21.15.105:0/498950090 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f73ec06ef90 0x7f73ec06f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.799+0000 7f73f0815640 1 -- 172.21.15.105:0/498950090 >> v2:172.21.15.154:3300/0 conn(0x7f73ec144c90 msgr2=0x7f73ec0b4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:07.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.799+0000 7f73f0815640 1 --2- 172.21.15.105:0/498950090 >> v2:172.21.15.154:3300/0 conn(0x7f73ec144c90 0x7f73ec0b4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.799+0000 7f73f0815640 1 -- 172.21.15.105:0/498950090 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73ec0b4380 con 0x7f73ec0cd1d0 2024-04-15T15:20:07.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.799+0000 7f73f0815640 1 -- 172.21.15.105:0/498950090 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7f73dc0019f0 con 0x7f73ec0cd1d0 2024-04-15T15:20:07.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.799+0000 7f73f0815640 1 -- 172.21.15.105:0/498950090 >> v1:172.21.15.105:6789/0 conn(0x7f73ec0cd1d0 legacy=0x7f73ec0cd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:07.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.799+0000 7f73f0815640 1 --2- 172.21.15.105:0/498950090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73d40039b0 0x7f73d4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:07.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.799+0000 7f73f0815640 1 --2- 172.21.15.105:0/498950090 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f73d4004800 0x7f73d4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:07.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.799+0000 7f73f0815640 1 --2- 172.21.15.105:0/498950090 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73d40051c0 0x7f73d400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:07.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.799+0000 7f73f0815640 1 -- 172.21.15.105:0/498950090 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec0b3e50 con 0x7f73d40039b0 2024-04-15T15:20:07.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.799+0000 7f73f0815640 1 -- 172.21.15.105:0/498950090 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec06f910 con 0x7f73d40051c0 2024-04-15T15:20:07.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.799+0000 7f73f0815640 1 -- 172.21.15.105:0/498950090 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f73d400bfe0 con 0x7f73d4004800 2024-04-15T15:20:07.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.799+0000 7f73f1016640 1 --2- 172.21.15.105:0/498950090 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f73ec06ef90 0x7f73ec06f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:20:07.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.799+0000 7f73f1817640 1 --2- 172.21.15.105:0/498950090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73d40039b0 0x7f73d4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:07.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.799+0000 7f73f2018640 1 --2- 172.21.15.105:0/498950090 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73d40051c0 0x7f73d400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:07.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.799+0000 7f73f1016640 1 --2- 172.21.15.105:0/498950090 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f73d4004800 0x7f73d4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:07.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.799+0000 7f73f1817640 1 -- 172.21.15.105:0/498950090 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f73d4004800 msgr2=0x7f73d4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:07.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.799+0000 7f73f1817640 1 --2- 172.21.15.105:0/498950090 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f73d4004800 0x7f73d4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.799+0000 7f73f1817640 1 -- 172.21.15.105:0/498950090 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73d40051c0 msgr2=0x7f73d400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:07.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.799+0000 7f73f1817640 1 --2- 172.21.15.105:0/498950090 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73d40051c0 0x7f73d400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.799+0000 7f73f1817640 1 -- 172.21.15.105:0/498950090 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73d4003520 con 0x7f73d40039b0 2024-04-15T15:20:07.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.800+0000 7f73f1817640 1 --2- 172.21.15.105:0/498950090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73d40039b0 0x7f73d4003dd0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f73e4009ac0 tx=0x7f73d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61f29d2b7691647 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:07.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.800+0000 7f73f0815640 1 -- 172.21.15.105:0/498950090 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e4039070 con 0x7f73d40039b0 2024-04-15T15:20:07.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.800+0000 7f73f0815640 1 -- 172.21.15.105:0/498950090 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73e402c710 con 0x7f73d40039b0 2024-04-15T15:20:07.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.800+0000 7f73f0815640 1 -- 172.21.15.105:0/498950090 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e4009690 con 0x7f73d40039b0 2024-04-15T15:20:07.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.801+0000 7f73f3a7b640 1 -- 172.21.15.105:0/498950090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73d40039b0 msgr2=0x7f73d4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:07.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.801+0000 7f73f3a7b640 1 --2- 172.21.15.105:0/498950090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73d40039b0 0x7f73d4003dd0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f73e4009ac0 tx=0x7f73d4005830 comp rx=0 tx=0).stop 2024-04-15T15:20:07.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.801+0000 7f73f2018640 1 -- 172.21.15.105:0/498950090 reap_dead start 2024-04-15T15:20:07.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.801+0000 7f73f3a7b640 1 -- 172.21.15.105:0/498950090 shutdown_connections 2024-04-15T15:20:07.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.801+0000 7f73f3a7b640 1 -- 172.21.15.105:0/498950090 >> 172.21.15.105:0/498950090 conn(0x7f73ec067d50 msgr2=0x7f73ec06b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:07.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.801+0000 7f73f3a7b640 1 -- 172.21.15.105:0/498950090 shutdown_connections 2024-04-15T15:20:07.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.801+0000 7f73f3a7b640 1 -- 172.21.15.105:0/498950090 wait complete. 2024-04-15T15:20:07.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.802+0000 7f73f3a7b640 1 Processor -- start 2024-04-15T15:20:07.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.802+0000 7f73f3a7b640 1 -- start start 2024-04-15T15:20:07.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.803+0000 7f73f3a7b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73d40039b0 0x7f73ec142110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:07.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.803+0000 7f73f3a7b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f73ec13a4b0 0x7f73ec13bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:07.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.803+0000 7f73f3a7b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73ec13a930 0x7f73ec13c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:07.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.803+0000 7f73f3a7b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec145070 con 0x7f73d40039b0 2024-04-15T15:20:07.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.803+0000 7f73f3a7b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec0b4940 con 0x7f73ec13a930 2024-04-15T15:20:07.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.803+0000 7f73f3a7b640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f73ec1451f0 con 0x7f73ec13a4b0 2024-04-15T15:20:07.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.803+0000 7f73f1817640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73d40039b0 0x7f73ec142110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:07.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.803+0000 7f73f1817640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73d40039b0 0x7f73ec142110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37662/0 (socket says 172.21.15.105:37662) 2024-04-15T15:20:07.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.803+0000 7f73f1817640 1 -- 172.21.15.105:0/3107814061 learned_addr learned my addr 172.21.15.105:0/3107814061 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:07.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.803+0000 7f73f1817640 1 -- 172.21.15.105:0/3107814061 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f73ec13a4b0 msgr2=0x7f73ec13bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:07.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.803+0000 7f73f1817640 1 --2- 172.21.15.105:0/3107814061 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f73ec13a4b0 0x7f73ec13bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.803+0000 7f73f1817640 1 -- 172.21.15.105:0/3107814061 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73ec13a930 msgr2=0x7f73ec13c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:07.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.803+0000 7f73f1817640 1 --2- 172.21.15.105:0/3107814061 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73ec13a930 0x7f73ec13c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.803+0000 7f73f1817640 1 -- 172.21.15.105:0/3107814061 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73ec1f4f40 con 0x7f73d40039b0 2024-04-15T15:20:07.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.803+0000 7f73f1817640 1 --2- 172.21.15.105:0/3107814061 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73d40039b0 0x7f73ec142110 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f73e4037040 tx=0x7f73e4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:07.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.803+0000 7f73ea7fc640 1 -- 172.21.15.105:0/3107814061 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e4039050 con 0x7f73d40039b0 2024-04-15T15:20:07.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.803+0000 7f73ea7fc640 1 -- 172.21.15.105:0/3107814061 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73e4045070 con 0x7f73d40039b0 2024-04-15T15:20:07.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.803+0000 7f73f3a7b640 1 -- 172.21.15.105:0/3107814061 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73ec0ada50 con 0x7f73d40039b0 2024-04-15T15:20:07.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.803+0000 7f73ea7fc640 1 -- 172.21.15.105:0/3107814061 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e4003a50 con 0x7f73d40039b0 2024-04-15T15:20:07.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.804+0000 7f73f3a7b640 1 -- 172.21.15.105:0/3107814061 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73ec1f4210 con 0x7f73d40039b0 2024-04-15T15:20:07.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.804+0000 7f73f3a7b640 1 -- 172.21.15.105:0/3107814061 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73ec068ab0 con 0x7f73d40039b0 2024-04-15T15:20:07.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.805+0000 7f73ea7fc640 1 -- 172.21.15.105:0/3107814061 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f73e4003bf0 con 0x7f73d40039b0 2024-04-15T15:20:07.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.805+0000 7f73ea7fc640 1 --2- 172.21.15.105:0/3107814061 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f73d806c8f0 0x7f73d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:07.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.805+0000 7f73ea7fc640 1 -- 172.21.15.105:0/3107814061 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f73e40ad750 con 0x7f73d40039b0 2024-04-15T15:20:07.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.805+0000 7f73f1016640 1 --2- 172.21.15.105:0/3107814061 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f73d806c8f0 0x7f73d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:07.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.806+0000 7f73f1016640 1 --2- 172.21.15.105:0/3107814061 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f73d806c8f0 0x7f73d806edb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f73e0006e80 tx=0x7f73e0006dd0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:07.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.810+0000 7f73ea7fc640 1 -- 172.21.15.105:0/3107814061 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f73e404e020 con 0x7f73d40039b0 2024-04-15T15:20:07.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.985+0000 7f73f3a7b640 1 -- 172.21.15.105:0/3107814061 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f73ec069380 con 0x7f73d40039b0 2024-04-15T15:20:07.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.985+0000 7f73ea7fc640 1 -- 172.21.15.105:0/3107814061 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f73e407bb00 con 0x7f73d40039b0 2024-04-15T15:20:07.987 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-04-15T15:20:07.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.987+0000 7f73f3a7b640 1 -- 172.21.15.105:0/3107814061 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f73d806c8f0 msgr2=0x7f73d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:07.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.987+0000 7f73f3a7b640 1 --2- 172.21.15.105:0/3107814061 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f73d806c8f0 0x7f73d806edb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f73e0006e80 tx=0x7f73e0006dd0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.987+0000 7f73f3a7b640 1 -- 172.21.15.105:0/3107814061 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73d40039b0 msgr2=0x7f73ec142110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:07.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.987+0000 7f73f3a7b640 1 --2- 172.21.15.105:0/3107814061 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73d40039b0 0x7f73ec142110 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f73e4037040 tx=0x7f73e4004730 comp rx=0 tx=0).stop 2024-04-15T15:20:07.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.987+0000 7f73f3a7b640 1 -- 172.21.15.105:0/3107814061 shutdown_connections 2024-04-15T15:20:07.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.987+0000 7f73f3a7b640 1 --2- 172.21.15.105:0/3107814061 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f73ec13a4b0 0x7f73ec13bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.987+0000 7f73f3a7b640 1 --2- 172.21.15.105:0/3107814061 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f73d806c8f0 0x7f73d806edb0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.987+0000 7f73f3a7b640 1 --2- 172.21.15.105:0/3107814061 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73ec13a930 0x7f73ec13c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.987+0000 7f73f3a7b640 1 --2- 172.21.15.105:0/3107814061 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73d40039b0 0x7f73ec142110 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.987+0000 7f73f3a7b640 1 -- 172.21.15.105:0/3107814061 >> 172.21.15.105:0/3107814061 conn(0x7f73ec067d50 msgr2=0x7f73ec0cc6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:07.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.987+0000 7f73f3a7b640 1 -- 172.21.15.105:0/3107814061 shutdown_connections 2024-04-15T15:20:07.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:07.987+0000 7f73f3a7b640 1 -- 172.21.15.105:0/3107814061 wait complete. 2024-04-15T15:20:08.000 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-15T15:20:08.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.102+0000 7f8a0dc62640 1 Processor -- start 2024-04-15T15:20:08.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.102+0000 7f8a0dc62640 1 -- start start 2024-04-15T15:20:08.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.103+0000 7f8a0dc62640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8a0813ec60 0x7f8a0806ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:08.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.103+0000 7f8a0dc62640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a0815bc90 con 0x7f8a0806f4d0 2024-04-15T15:20:08.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.103+0000 7f8a0dc62640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a0815bf70 con 0x7f8a0813e680 2024-04-15T15:20:08.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.103+0000 7f8a0dc62640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f8a080d0280 con 0x7f8a0813ec60 2024-04-15T15:20:08.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.103+0000 7f8a06ffd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8a0813ec60 0x7f8a0806ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:08.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.103+0000 7f8a06ffd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8a0813ec60 0x7f8a0806ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37668/0 (socket says 172.21.15.105:37668) 2024-04-15T15:20:08.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.103+0000 7f8a06ffd640 1 -- 172.21.15.105:0/2423478199 learned_addr learned my addr 172.21.15.105:0/2423478199 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:08.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.103+0000 7f8a07fff640 1 --1- 172.21.15.105:0/2423478199 >> v1:172.21.15.105:6789/0 conn(0x7f8a0806f4d0 0x7f8a0815f1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:55204/0 (socket says 172.21.15.105:55204) 2024-04-15T15:20:08.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.103+0000 7f8a06ffd640 1 -- 172.21.15.105:0/2423478199 >> v1:172.21.15.154:6789/0 conn(0x7f8a0813e680 legacy=0x7f8a080cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:08.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.103+0000 7f8a06ffd640 1 -- 172.21.15.105:0/2423478199 >> v1:172.21.15.105:6789/0 conn(0x7f8a0806f4d0 legacy=0x7f8a0815f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:08.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.103+0000 7f8a06ffd640 1 -- 172.21.15.105:0/2423478199 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a0815c250 con 0x7f8a0813ec60 2024-04-15T15:20:08.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.103+0000 7f8a067fc640 1 -- 172.21.15.105:0/2423478199 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1503267827 0 0) 0x7f8a0815bc90 con 0x7f8a0806f4d0 2024-04-15T15:20:08.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.104+0000 7f8a06ffd640 1 --2- 172.21.15.105:0/2423478199 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a0813ec60 0x7f8a0806ef90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f89fc00bb10 tx=0x7f89fc02e170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29f516b98b7a2c75 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:08.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.104+0000 7f8a067fc640 1 -- 172.21.15.105:0/2423478199 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc03b070 con 0x7f8a0813ec60 2024-04-15T15:20:08.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.104+0000 7f8a067fc640 1 -- 172.21.15.105:0/2423478199 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89fc036440 con 0x7f8a0813ec60 2024-04-15T15:20:08.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.104+0000 7f8a067fc640 1 -- 172.21.15.105:0/2423478199 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc036810 con 0x7f8a0813ec60 2024-04-15T15:20:08.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.105+0000 7f8a0dc62640 1 -- 172.21.15.105:0/2423478199 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a0813ec60 msgr2=0x7f8a0806ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:08.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.105+0000 7f8a0dc62640 1 --2- 172.21.15.105:0/2423478199 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a0813ec60 0x7f8a0806ef90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f89fc00bb10 tx=0x7f89fc02e170 comp rx=0 tx=0).stop 2024-04-15T15:20:08.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.105+0000 7f8a0dc62640 1 -- 172.21.15.105:0/2423478199 shutdown_connections 2024-04-15T15:20:08.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.105+0000 7f8a0dc62640 1 --2- 172.21.15.105:0/2423478199 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a0813ec60 0x7f8a0806ef90 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:08.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.105+0000 7f8a0dc62640 1 -- 172.21.15.105:0/2423478199 >> 172.21.15.105:0/2423478199 conn(0x7f8a08067d50 msgr2=0x7f8a0806b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:08.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.105+0000 7f8a0dc62640 1 -- 172.21.15.105:0/2423478199 shutdown_connections 2024-04-15T15:20:08.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.105+0000 7f8a0dc62640 1 -- 172.21.15.105:0/2423478199 wait complete. 2024-04-15T15:20:08.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.106+0000 7f8a0dc62640 1 Processor -- start 2024-04-15T15:20:08.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.106+0000 7f8a0dc62640 1 -- start start 2024-04-15T15:20:08.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.107+0000 7f8a0dc62640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8a0806f4d0 0x7f8a081e5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:08.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.107+0000 7f8a0dc62640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a0813e680 0x7f8a081380b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:08.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.107+0000 7f8a0dc62640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a0813ec60 0x7f8a0815a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:08.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.107+0000 7f8a0dc62640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0815c810 con 0x7f8a0813ec60 2024-04-15T15:20:08.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.107+0000 7f8a0dc62640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0806f900 con 0x7f8a0813e680 2024-04-15T15:20:08.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.107+0000 7f8a0dc62640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f8a080cf590 con 0x7f8a0806f4d0 2024-04-15T15:20:08.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.107+0000 7f8a07fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a0813ec60 0x7f8a0815a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:08.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.107+0000 7f8a07fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a0813ec60 0x7f8a0815a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37670/0 (socket says 172.21.15.105:37670) 2024-04-15T15:20:08.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.107+0000 7f8a07fff640 1 -- 172.21.15.105:0/212164090 learned_addr learned my addr 172.21.15.105:0/212164090 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:08.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.107+0000 7f8a06ffd640 1 --2- 172.21.15.105:0/212164090 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a0813e680 0x7f8a081380b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:08.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.107+0000 7f8a077fe640 1 --2- 172.21.15.105:0/212164090 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8a0806f4d0 0x7f8a081e5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:08.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.107+0000 7f8a07fff640 1 -- 172.21.15.105:0/212164090 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8a0806f4d0 msgr2=0x7f8a081e5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:08.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.107+0000 7f8a07fff640 1 --2- 172.21.15.105:0/212164090 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8a0806f4d0 0x7f8a081e5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:08.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.107+0000 7f8a07fff640 1 -- 172.21.15.105:0/212164090 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a0813e680 msgr2=0x7f8a081380b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:08.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.107+0000 7f8a07fff640 1 --2- 172.21.15.105:0/212164090 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a0813e680 0x7f8a081380b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:08.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.107+0000 7f8a07fff640 1 -- 172.21.15.105:0/212164090 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a081e4cd0 con 0x7f8a0813ec60 2024-04-15T15:20:08.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.107+0000 7f8a077fe640 1 --2- 172.21.15.105:0/212164090 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8a0806f4d0 0x7f8a081e5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:20:08.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.107+0000 7f8a06ffd640 1 --2- 172.21.15.105:0/212164090 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a0813e680 0x7f8a081380b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:20:08.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.108+0000 7f8a07fff640 1 --2- 172.21.15.105:0/212164090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a0813ec60 0x7f8a0815a5c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8a0000c7d0 tx=0x7f8a0000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:08.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.108+0000 7f89effff640 1 -- 172.21.15.105:0/212164090 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a00018070 con 0x7f8a0813ec60 2024-04-15T15:20:08.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.108+0000 7f89effff640 1 -- 172.21.15.105:0/212164090 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a0000a640 con 0x7f8a0813ec60 2024-04-15T15:20:08.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.108+0000 7f89effff640 1 -- 172.21.15.105:0/212164090 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0000aa70 con 0x7f8a0813ec60 2024-04-15T15:20:08.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.108+0000 7f8a0dc62640 1 -- 172.21.15.105:0/212164090 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a081e4fb0 con 0x7f8a0813ec60 2024-04-15T15:20:08.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.108+0000 7f8a0dc62640 1 -- 172.21.15.105:0/212164090 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a081e5570 con 0x7f8a0813ec60 2024-04-15T15:20:08.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.109+0000 7f8a0dc62640 1 -- 172.21.15.105:0/212164090 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a08068a40 con 0x7f8a0813ec60 2024-04-15T15:20:08.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.109+0000 7f89effff640 1 -- 172.21.15.105:0/212164090 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8a0001b070 con 0x7f8a0813ec60 2024-04-15T15:20:08.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.110+0000 7f89effff640 1 --2- 172.21.15.105:0/212164090 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f89f406c920 0x7f89f406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:08.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.110+0000 7f89effff640 1 -- 172.21.15.105:0/212164090 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f8a0008b140 con 0x7f8a0813ec60 2024-04-15T15:20:08.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.110+0000 7f8a077fe640 1 --2- 172.21.15.105:0/212164090 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f89f406c920 0x7f89f406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:08.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.110+0000 7f8a077fe640 1 --2- 172.21.15.105:0/212164090 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f89f406c920 0x7f89f406ede0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8a0813dc50 tx=0x7f89f8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:08.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.114+0000 7f89effff640 1 -- 172.21.15.105:0/212164090 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8a00059580 con 0x7f8a0813ec60 2024-04-15T15:20:08.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.301+0000 7f8a0dc62640 1 -- 172.21.15.105:0/212164090 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8a080af830 con 0x7f8a0813ec60 2024-04-15T15:20:08.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.301+0000 7f89effff640 1 -- 172.21.15.105:0/212164090 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f8a00058e80 con 0x7f8a0813ec60 2024-04-15T15:20:08.303 INFO:teuthology.orchestra.run.smithi105.stderr:listed 0 entries 2024-04-15T15:20:08.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.303+0000 7f8a0dc62640 1 -- 172.21.15.105:0/212164090 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f89f406c920 msgr2=0x7f89f406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:08.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.303+0000 7f8a0dc62640 1 --2- 172.21.15.105:0/212164090 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f89f406c920 0x7f89f406ede0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8a0813dc50 tx=0x7f89f8009040 comp rx=0 tx=0).stop 2024-04-15T15:20:08.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.303+0000 7f8a0dc62640 1 -- 172.21.15.105:0/212164090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a0813ec60 msgr2=0x7f8a0815a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:08.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.303+0000 7f8a0dc62640 1 --2- 172.21.15.105:0/212164090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a0813ec60 0x7f8a0815a5c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8a0000c7d0 tx=0x7f8a0000cc90 comp rx=0 tx=0).stop 2024-04-15T15:20:08.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.303+0000 7f8a0dc62640 1 -- 172.21.15.105:0/212164090 shutdown_connections 2024-04-15T15:20:08.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.303+0000 7f8a0dc62640 1 --2- 172.21.15.105:0/212164090 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f89f406c920 0x7f89f406ede0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:08.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.303+0000 7f8a0dc62640 1 --2- 172.21.15.105:0/212164090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a0813ec60 0x7f8a0815a5c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:08.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.303+0000 7f8a0dc62640 1 --2- 172.21.15.105:0/212164090 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a0813e680 0x7f8a081380b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:08.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.303+0000 7f8a0dc62640 1 --2- 172.21.15.105:0/212164090 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8a0806f4d0 0x7f8a081e5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:08.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.303+0000 7f8a0dc62640 1 -- 172.21.15.105:0/212164090 >> 172.21.15.105:0/212164090 conn(0x7f8a08067d50 msgr2=0x7f8a080b4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:08.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.303+0000 7f8a0dc62640 1 -- 172.21.15.105:0/212164090 shutdown_connections 2024-04-15T15:20:08.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.303+0000 7f8a0dc62640 1 -- 172.21.15.105:0/212164090 wait complete. 2024-04-15T15:20:08.316 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi154.front.sepia.ceph.com... 2024-04-15T15:20:08.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:08.316 DEBUG:teuthology.orchestra.run.smithi154:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-04-15T15:20:08.351 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:20:08.352 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi154.front.sepia.ceph.com 2024-04-15T15:20:08.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:08.352 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-15T15:20:08.410 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-15T15:20:08.411 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-15T15:20:08.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.513+0000 7f99244e4640 1 Processor -- start 2024-04-15T15:20:08.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.513+0000 7f99244e4640 1 -- start start 2024-04-15T15:20:08.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.514+0000 7f99244e4640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f991c09b150 0x7f991c0d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:08.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.514+0000 7f99244e4640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f991c09b780 0x7f991c0cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:08.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.514+0000 7f99244e4640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f991c0714f0 con 0x7f991c15ec60 2024-04-15T15:20:08.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.514+0000 7f99244e4640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f991c06cad0 con 0x7f991c09b150 2024-04-15T15:20:08.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.514+0000 7f99244e4640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f991c0cd3b0 con 0x7f991c09b780 2024-04-15T15:20:08.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.514+0000 7f9922280640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f991c15ec60 0x7f991c09aa40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.105:49256/0 (socket says 172.21.15.105:49256) 2024-04-15T15:20:08.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.514+0000 7f9922280640 1 -- 172.21.15.105:0/818812375 learned_addr learned my addr 172.21.15.105:0/818812375 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:20:08.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.514+0000 7f9922a81640 1 --2- 172.21.15.105:0/818812375 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f991c09b780 0x7f991c0cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:08.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.514+0000 7f9921a7f640 1 --2- 172.21.15.105:0/818812375 >> v2:172.21.15.154:3300/0 conn(0x7f991c09b150 0x7f991c0d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:08.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.515+0000 7f992127e640 1 -- 172.21.15.105:0/818812375 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3360873008 0 0) 0x7f991c0714f0 con 0x7f991c15ec60 2024-04-15T15:20:08.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.515+0000 7f992127e640 1 -- 172.21.15.105:0/818812375 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9908003520 con 0x7f991c15ec60 2024-04-15T15:20:08.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.515+0000 7f9921a7f640 1 -- 172.21.15.105:0/818812375 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f991c09b780 msgr2=0x7f991c0cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:08.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.515+0000 7f9921a7f640 1 --2- 172.21.15.105:0/818812375 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f991c09b780 0x7f991c0cd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:08.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.515+0000 7f9921a7f640 1 -- 172.21.15.105:0/818812375 >> v1:172.21.15.154:6789/0 conn(0x7f991c15ec60 legacy=0x7f991c09aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:08.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.515+0000 7f9921a7f640 1 -- 172.21.15.105:0/818812375 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f991c0d1940 con 0x7f991c09b150 2024-04-15T15:20:08.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.515+0000 7f9922a81640 1 --2- 172.21.15.105:0/818812375 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f991c09b780 0x7f991c0cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:20:08.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.516+0000 7f9921a7f640 1 --2- 172.21.15.105:0/818812375 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f991c09b150 0x7f991c0d1210 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f990c0097a0 tx=0x7f990c02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=839d3e3082491e11 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:08.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.516+0000 7f992127e640 1 -- 172.21.15.105:0/818812375 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990c035020 con 0x7f991c09b150 2024-04-15T15:20:08.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.516+0000 7f992127e640 1 -- 172.21.15.105:0/818812375 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f990c031830 con 0x7f991c09b150 2024-04-15T15:20:08.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.516+0000 7f992127e640 1 -- 172.21.15.105:0/818812375 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990c031be0 con 0x7f991c09b150 2024-04-15T15:20:08.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.516+0000 7f99244e4640 1 -- 172.21.15.105:0/818812375 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f991c09b150 msgr2=0x7f991c0d1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:08.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.516+0000 7f99244e4640 1 --2- 172.21.15.105:0/818812375 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f991c09b150 0x7f991c0d1210 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f990c0097a0 tx=0x7f990c02b8d0 comp rx=0 tx=0).stop 2024-04-15T15:20:08.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.516+0000 7f99244e4640 1 -- 172.21.15.105:0/818812375 shutdown_connections 2024-04-15T15:20:08.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.516+0000 7f99244e4640 1 --2- 172.21.15.105:0/818812375 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f991c09b780 0x7f991c0cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:08.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.516+0000 7f99244e4640 1 --2- 172.21.15.105:0/818812375 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f991c09b150 0x7f991c0d1210 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:08.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.516+0000 7f99244e4640 1 -- 172.21.15.105:0/818812375 >> 172.21.15.105:0/818812375 conn(0x7f991c0d2570 msgr2=0x7f991c0d2980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:08.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.516+0000 7f99244e4640 1 -- 172.21.15.105:0/818812375 shutdown_connections 2024-04-15T15:20:08.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.516+0000 7f99244e4640 1 -- 172.21.15.105:0/818812375 wait complete. 2024-04-15T15:20:08.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.518+0000 7f99244e4640 1 Processor -- start 2024-04-15T15:20:08.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.518+0000 7f99244e4640 1 -- start start 2024-04-15T15:20:08.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.518+0000 7f99244e4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f991c09b150 0x7f991c0cbe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:08.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.518+0000 7f99244e4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f991c09b780 0x7f991c0ca510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:08.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.518+0000 7f99244e4640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f991c09fd70 0x7f991c0b7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:08.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.518+0000 7f99244e4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f991c0cdd80 con 0x7f991c09b780 2024-04-15T15:20:08.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.518+0000 7f99244e4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f991c0cd040 con 0x7f991c09b150 2024-04-15T15:20:08.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.518+0000 7f99244e4640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f991c0cdf00 con 0x7f991c09fd70 2024-04-15T15:20:08.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.518+0000 7f9921a7f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f991c09b780 0x7f991c0ca510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:08.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.518+0000 7f9921a7f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f991c09b780 0x7f991c0ca510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37680/0 (socket says 172.21.15.105:37680) 2024-04-15T15:20:08.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.518+0000 7f9921a7f640 1 -- 172.21.15.105:0/252322513 learned_addr learned my addr 172.21.15.105:0/252322513 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:08.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.519+0000 7f9922280640 1 --2- 172.21.15.105:0/252322513 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f991c09b150 0x7f991c0cbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:08.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.519+0000 7f9922a81640 1 --2- 172.21.15.105:0/252322513 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f991c09fd70 0x7f991c0b7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:08.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.519+0000 7f9921a7f640 1 -- 172.21.15.105:0/252322513 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f991c09fd70 msgr2=0x7f991c0b7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:08.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.519+0000 7f9921a7f640 1 --2- 172.21.15.105:0/252322513 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f991c09fd70 0x7f991c0b7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:08.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.519+0000 7f9921a7f640 1 -- 172.21.15.105:0/252322513 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f991c09b150 msgr2=0x7f991c0cbe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:08.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.519+0000 7f9921a7f640 1 --2- 172.21.15.105:0/252322513 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f991c09b150 0x7f991c0cbe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:08.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.519+0000 7f9921a7f640 1 -- 172.21.15.105:0/252322513 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f991c0a01f0 con 0x7f991c09b780 2024-04-15T15:20:08.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.519+0000 7f9921a7f640 1 --2- 172.21.15.105:0/252322513 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f991c09b780 0x7f991c0ca510 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f990c0098c0 tx=0x7f990c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:08.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.519+0000 7f9912ffd640 1 -- 172.21.15.105:0/252322513 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990c035040 con 0x7f991c09b780 2024-04-15T15:20:08.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.519+0000 7f9912ffd640 1 -- 172.21.15.105:0/252322513 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f990c03b840 con 0x7f991c09b780 2024-04-15T15:20:08.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.519+0000 7f9912ffd640 1 -- 172.21.15.105:0/252322513 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990c03bc30 con 0x7f991c09b780 2024-04-15T15:20:08.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.519+0000 7f99244e4640 1 -- 172.21.15.105:0/252322513 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f991c0c9770 con 0x7f991c09b780 2024-04-15T15:20:08.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.519+0000 7f99244e4640 1 -- 172.21.15.105:0/252322513 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f991c0c9d30 con 0x7f991c09b780 2024-04-15T15:20:08.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.520+0000 7f99244e4640 1 -- 172.21.15.105:0/252322513 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f991c068a40 con 0x7f991c09b780 2024-04-15T15:20:08.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.521+0000 7f9912ffd640 1 -- 172.21.15.105:0/252322513 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f990c038020 con 0x7f991c09b780 2024-04-15T15:20:08.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.521+0000 7f9912ffd640 1 --2- 172.21.15.105:0/252322513 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f990006c8f0 0x7f990006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:08.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.521+0000 7f9912ffd640 1 -- 172.21.15.105:0/252322513 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f990c0ada90 con 0x7f991c09b780 2024-04-15T15:20:08.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.521+0000 7f9922280640 1 --2- 172.21.15.105:0/252322513 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f990006c8f0 0x7f990006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:08.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.521+0000 7f9922280640 1 --2- 172.21.15.105:0/252322513 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f990006c8f0 0x7f990006edb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9918005880 tx=0x7f9918009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:08.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.525+0000 7f9912ffd640 1 -- 172.21.15.105:0/252322513 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f990c07bed0 con 0x7f991c09b780 2024-04-15T15:20:08.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.739+0000 7f99244e4640 1 -- 172.21.15.105:0/252322513 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f991c0d2270 con 0x7f991c09b780 2024-04-15T15:20:08.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.740+0000 7f9912ffd640 1 -- 172.21.15.105:0/252322513 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5187 (secure 0 0 0) 0x7f990c07b7d0 con 0x7f991c09b780 2024-04-15T15:20:08.742 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:20:08.742 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4259,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/3319657830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":3319657830},{"type":"v1","addr":"172.21.15.154:6835","nonce":3319657830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2677199890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2677199890},{"type":"v1","addr":"172.21.15.105:6835","nonce":2677199890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2867739743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2867739743},{"type":"v1","addr":"172.21.15.105:6837","nonce":2867739743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-15T15:19:25.760362+0000","modified":"2024-04-15T15:19:27.857599+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.154:6834/1529675504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1529675504},{"type":"v1","addr":"172.21.15.154:6834","nonce":1529675504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4291,"qdb_cluster":[4291]},"id":1}]} 2024-04-15T15:20:08.742 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 6 2024-04-15T15:20:08.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.742+0000 7f99244e4640 1 -- 172.21.15.105:0/252322513 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f990006c8f0 msgr2=0x7f990006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:08.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.742+0000 7f99244e4640 1 --2- 172.21.15.105:0/252322513 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f990006c8f0 0x7f990006edb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9918005880 tx=0x7f9918009040 comp rx=0 tx=0).stop 2024-04-15T15:20:08.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.742+0000 7f99244e4640 1 -- 172.21.15.105:0/252322513 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f991c09b780 msgr2=0x7f991c0ca510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:08.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.742+0000 7f99244e4640 1 --2- 172.21.15.105:0/252322513 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f991c09b780 0x7f991c0ca510 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f990c0098c0 tx=0x7f990c0046c0 comp rx=0 tx=0).stop 2024-04-15T15:20:08.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.742+0000 7f99244e4640 1 -- 172.21.15.105:0/252322513 shutdown_connections 2024-04-15T15:20:08.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.742+0000 7f99244e4640 1 --2- 172.21.15.105:0/252322513 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f991c09fd70 0x7f991c0b7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:08.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.742+0000 7f99244e4640 1 --2- 172.21.15.105:0/252322513 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f990006c8f0 0x7f990006edb0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:08.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.742+0000 7f99244e4640 1 --2- 172.21.15.105:0/252322513 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f991c09b780 0x7f991c0ca510 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:08.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.742+0000 7f99244e4640 1 --2- 172.21.15.105:0/252322513 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f991c09b150 0x7f991c0cbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:08.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.742+0000 7f99244e4640 1 -- 172.21.15.105:0/252322513 >> 172.21.15.105:0/252322513 conn(0x7f991c0d2570 msgr2=0x7f991c06cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:08.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.742+0000 7f99244e4640 1 -- 172.21.15.105:0/252322513 shutdown_connections 2024-04-15T15:20:08.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.742+0000 7f99244e4640 1 -- 172.21.15.105:0/252322513 wait complete. 2024-04-15T15:20:08.755 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:20:08.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.856+0000 7f00a90d3640 1 Processor -- start 2024-04-15T15:20:08.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.856+0000 7f00a90d3640 1 -- start start 2024-04-15T15:20:08.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.857+0000 7f00a90d3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00a4145e00 0x7f00a40ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:08.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.857+0000 7f00a90d3640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f00a40ced30 0x7f00a4071e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:08.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.857+0000 7f00a90d3640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00a40ccfb0 con 0x7f00a40723a0 2024-04-15T15:20:08.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.857+0000 7f00a90d3640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f00a4072810 con 0x7f00a40ced30 2024-04-15T15:20:08.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.857+0000 7f00a90d3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f00a40cdbc0 con 0x7f00a4145e00 2024-04-15T15:20:08.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.857+0000 7f009bfff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f00a40ced30 0x7f00a4071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:08.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.857+0000 7f00a2d9d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00a4145e00 0x7f00a40ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:08.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.857+0000 7f00a359e640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f00a40723a0 0x7f00a4160230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.105:49268/0 (socket says 172.21.15.105:49268) 2024-04-15T15:20:08.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.857+0000 7f00a359e640 1 -- 172.21.15.105:0/649271292 learned_addr learned my addr 172.21.15.105:0/649271292 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:20:08.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.858+0000 7f00a2d9d640 1 -- 172.21.15.105:0/649271292 >> v2:172.21.15.154:3300/0 conn(0x7f00a40ced30 msgr2=0x7f00a4071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:08.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.858+0000 7f00a2d9d640 1 --2- 172.21.15.105:0/649271292 >> v2:172.21.15.154:3300/0 conn(0x7f00a40ced30 0x7f00a4071e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:08.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.858+0000 7f00a2d9d640 1 -- 172.21.15.105:0/649271292 >> v1:172.21.15.154:6789/0 conn(0x7f00a40723a0 legacy=0x7f00a4160230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:08.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.858+0000 7f00a2d9d640 1 -- 172.21.15.105:0/649271292 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00a40cd290 con 0x7f00a4145e00 2024-04-15T15:20:08.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.858+0000 7f009bfff640 1 --2- 172.21.15.105:0/649271292 >> v2:172.21.15.154:3300/0 conn(0x7f00a40ced30 0x7f00a4071e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:20:08.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.859+0000 7f00a2d9d640 1 --2- 172.21.15.105:0/649271292 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00a4145e00 0x7f00a40ce7f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f008c0097a0 tx=0x7f008c02b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f0c8819a345697d server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:08.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.859+0000 7f00a259c640 1 -- 172.21.15.105:0/649271292 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f008c035020 con 0x7f00a4145e00 2024-04-15T15:20:08.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.859+0000 7f00a259c640 1 -- 172.21.15.105:0/649271292 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f008c031830 con 0x7f00a4145e00 2024-04-15T15:20:08.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.859+0000 7f00a259c640 1 -- 172.21.15.105:0/649271292 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f008c031c00 con 0x7f00a4145e00 2024-04-15T15:20:08.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.859+0000 7f00a90d3640 1 -- 172.21.15.105:0/649271292 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00a4145e00 msgr2=0x7f00a40ce7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:08.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.859+0000 7f00a90d3640 1 --2- 172.21.15.105:0/649271292 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00a4145e00 0x7f00a40ce7f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f008c0097a0 tx=0x7f008c02b8d0 comp rx=0 tx=0).stop 2024-04-15T15:20:08.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.859+0000 7f00a90d3640 1 -- 172.21.15.105:0/649271292 shutdown_connections 2024-04-15T15:20:08.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.859+0000 7f00a90d3640 1 --2- 172.21.15.105:0/649271292 >> v2:172.21.15.154:3300/0 conn(0x7f00a40ced30 0x7f00a4071e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:08.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.859+0000 7f00a90d3640 1 --2- 172.21.15.105:0/649271292 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00a4145e00 0x7f00a40ce7f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:08.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.859+0000 7f00a90d3640 1 -- 172.21.15.105:0/649271292 >> 172.21.15.105:0/649271292 conn(0x7f00a4067d50 msgr2=0x7f00a406b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:08.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.859+0000 7f00a90d3640 1 -- 172.21.15.105:0/649271292 shutdown_connections 2024-04-15T15:20:08.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.859+0000 7f00a90d3640 1 -- 172.21.15.105:0/649271292 wait complete. 2024-04-15T15:20:08.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.861+0000 7f00a90d3640 1 Processor -- start 2024-04-15T15:20:08.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.861+0000 7f00a90d3640 1 -- start start 2024-04-15T15:20:08.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.861+0000 7f00a90d3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00a40723a0 0x7f00a4144910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:08.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.861+0000 7f00a90d3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00a40ced30 0x7f00a4157c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:08.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.861+0000 7f00a90d3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00a4145e00 0x7f00a41f6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:08.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.861+0000 7f00a90d3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f00a40ce370 con 0x7f00a40723a0 2024-04-15T15:20:08.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.861+0000 7f00a90d3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f00a40cd850 con 0x7f00a40ced30 2024-04-15T15:20:08.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.861+0000 7f00a90d3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f00a41461e0 con 0x7f00a4145e00 2024-04-15T15:20:08.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.862+0000 7f00a2d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00a40723a0 0x7f00a4144910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:08.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.862+0000 7f00a2d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00a40723a0 0x7f00a4144910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37694/0 (socket says 172.21.15.105:37694) 2024-04-15T15:20:08.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.862+0000 7f00a2d9d640 1 -- 172.21.15.105:0/3811707704 learned_addr learned my addr 172.21.15.105:0/3811707704 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:08.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.862+0000 7f009bfff640 1 --2- 172.21.15.105:0/3811707704 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00a40ced30 0x7f00a4157c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:08.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.862+0000 7f00a359e640 1 --2- 172.21.15.105:0/3811707704 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00a4145e00 0x7f00a41f6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:08.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.862+0000 7f00a2d9d640 1 -- 172.21.15.105:0/3811707704 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00a4145e00 msgr2=0x7f00a41f6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:08.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.862+0000 7f00a2d9d640 1 --2- 172.21.15.105:0/3811707704 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00a4145e00 0x7f00a41f6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:08.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.862+0000 7f00a2d9d640 1 -- 172.21.15.105:0/3811707704 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00a40ced30 msgr2=0x7f00a4157c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:08.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.862+0000 7f00a2d9d640 1 --2- 172.21.15.105:0/3811707704 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00a40ced30 0x7f00a4157c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:08.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.862+0000 7f00a2d9d640 1 -- 172.21.15.105:0/3811707704 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00a4158490 con 0x7f00a40723a0 2024-04-15T15:20:08.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.862+0000 7f00a2d9d640 1 --2- 172.21.15.105:0/3811707704 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00a40723a0 0x7f00a4144910 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f008c034040 tx=0x7f008c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:08.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.862+0000 7f009b7fe640 1 -- 172.21.15.105:0/3811707704 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f008c040070 con 0x7f00a40723a0 2024-04-15T15:20:08.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.862+0000 7f009b7fe640 1 -- 172.21.15.105:0/3811707704 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f008c03a910 con 0x7f00a40723a0 2024-04-15T15:20:08.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.862+0000 7f00a90d3640 1 -- 172.21.15.105:0/3811707704 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00a4158770 con 0x7f00a40723a0 2024-04-15T15:20:08.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.862+0000 7f009b7fe640 1 -- 172.21.15.105:0/3811707704 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f008c035040 con 0x7f00a40723a0 2024-04-15T15:20:08.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.862+0000 7f00a90d3640 1 -- 172.21.15.105:0/3811707704 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00a415a170 con 0x7f00a40723a0 2024-04-15T15:20:08.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.863+0000 7f00a90d3640 1 -- 172.21.15.105:0/3811707704 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00a4068a40 con 0x7f00a40723a0 2024-04-15T15:20:08.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.864+0000 7f009b7fe640 1 -- 172.21.15.105:0/3811707704 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f008c038020 con 0x7f00a40723a0 2024-04-15T15:20:08.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.864+0000 7f009b7fe640 1 --2- 172.21.15.105:0/3811707704 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f008406c8f0 0x7f008406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:08.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.864+0000 7f009b7fe640 1 -- 172.21.15.105:0/3811707704 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f008c0ad4f0 con 0x7f00a40723a0 2024-04-15T15:20:08.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.864+0000 7f009bfff640 1 --2- 172.21.15.105:0/3811707704 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f008406c8f0 0x7f008406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:08.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.865+0000 7f009bfff640 1 --2- 172.21.15.105:0/3811707704 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f008406c8f0 0x7f008406edb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0094009560 tx=0x7f0094009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:08.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:08.869+0000 7f009b7fe640 1 -- 172.21.15.105:0/3811707704 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f008c031800 con 0x7f00a40723a0 2024-04-15T15:20:09.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.043+0000 7f00a90d3640 1 -- 172.21.15.105:0/3811707704 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f00a40af9e0 con 0x7f00a40723a0 2024-04-15T15:20:09.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.044+0000 7f009b7fe640 1 -- 172.21.15.105:0/3811707704 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16901 (secure 0 0 0) 0x7f008c07b260 con 0x7f00a40723a0 2024-04-15T15:20:09.046 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:20:09.047 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":22,"fsid":"09e4d2a4-a324-4793-af77-644b25e520e1","created":"2024-04-15T15:19:01.524543+0000","modified":"2024-04-15T15:20:07.555905+0000","last_up_change":"2024-04-15T15:19:12.825815+0000","last_in_change":"2024-04-15T15:19:03.788126+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:19:13.748585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:19:23.789287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:19:24.722250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6f6292b-2d7e-4aa0-9d1c-6ddd0f97287c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6801","nonce":2873040054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6803","nonce":2873040054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6807","nonce":2873040054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6805","nonce":2873040054}]},"public_addr":"172.21.15.105:6801/2873040054","cluster_addr":"172.21.15.105:6803/2873040054","heartbeat_back_addr":"172.21.15.105:6807/2873040054","heartbeat_front_addr":"172.21.15.105:6805/2873040054","state":["exists","up"]},{"osd":1,"uuid":"8900966a-b393-41e3-8f93-e5a43bd23622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6818","nonce":2494994479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6821","nonce":2494994479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6828","nonce":2494994479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6823","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6825","nonce":2494994479}]},"public_addr":"172.21.15.105:6818/2494994479","cluster_addr":"172.21.15.105:6821/2494994479","heartbeat_back_addr":"172.21.15.105:6828/2494994479","heartbeat_front_addr":"172.21.15.105:6825/2494994479","state":["exists","up"]},{"osd":2,"uuid":"81a5dc7a-9325-4ff5-941d-a2d5fba21f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6809","nonce":3798408699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6811","nonce":3798408699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6815","nonce":3798408699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6813","nonce":3798408699}]},"public_addr":"172.21.15.105:6809/3798408699","cluster_addr":"172.21.15.105:6811/3798408699","heartbeat_back_addr":"172.21.15.105:6815/3798408699","heartbeat_front_addr":"172.21.15.105:6813/3798408699","state":["exists","up"]},{"osd":3,"uuid":"df766ed2-5c71-4805-96f4-3d4970edc960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6819","nonce":1920720186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6824","nonce":1920720186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6831","nonce":1920720186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6827","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6829","nonce":1920720186}]},"public_addr":"172.21.15.105:6819/1920720186","cluster_addr":"172.21.15.105:6824/1920720186","heartbeat_back_addr":"172.21.15.105:6831/1920720186","heartbeat_front_addr":"172.21.15.105:6829/1920720186","state":["exists","up"]},{"osd":4,"uuid":"e1039e03-28b8-49df-b951-c91930b93c9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6810","nonce":3516494456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6813","nonce":3516494456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6820","nonce":3516494456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6817","nonce":3516494456}]},"public_addr":"172.21.15.154:6810/3516494456","cluster_addr":"172.21.15.154:6813/3516494456","heartbeat_back_addr":"172.21.15.154:6820/3516494456","heartbeat_front_addr":"172.21.15.154:6817/3516494456","state":["exists","up"]},{"osd":5,"uuid":"a9f5eaf1-66ff-405a-a9e4-ec3340812a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6801","nonce":540367751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6803","nonce":540367751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6807","nonce":540367751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6805","nonce":540367751}]},"public_addr":"172.21.15.154:6801/540367751","cluster_addr":"172.21.15.154:6803/540367751","heartbeat_back_addr":"172.21.15.154:6807/540367751","heartbeat_front_addr":"172.21.15.154:6805/540367751","state":["exists","up"]},{"osd":6,"uuid":"267d02ea-675f-428f-aae8-e0375e4d4feb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6825","nonce":1550829005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6827","nonce":1550829005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6831","nonce":1550829005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6829","nonce":1550829005}]},"public_addr":"172.21.15.154:6825/1550829005","cluster_addr":"172.21.15.154:6827/1550829005","heartbeat_back_addr":"172.21.15.154:6831/1550829005","heartbeat_front_addr":"172.21.15.154:6829/1550829005","state":["exists","up"]},{"osd":7,"uuid":"759ac6cc-896a-423c-885f-9fe90d8dcb14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6809","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6811","nonce":3920089903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6816","nonce":3920089903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6823","nonce":3920089903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6819","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6821","nonce":3920089903}]},"public_addr":"172.21.15.154:6811/3920089903","cluster_addr":"172.21.15.154:6816/3920089903","heartbeat_back_addr":"172.21.15.154:6823/3920089903","heartbeat_front_addr":"172.21.15.154:6821/3920089903","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.949509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.984072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.907278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.947391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.975447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.998896+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:09.028739+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.894119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":1}]},{"pgid":"3.30","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:20:09.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.046+0000 7f00a90d3640 1 -- 172.21.15.105:0/3811707704 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f008406c8f0 msgr2=0x7f008406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:09.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.046+0000 7f00a90d3640 1 --2- 172.21.15.105:0/3811707704 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f008406c8f0 0x7f008406edb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0094009560 tx=0x7f0094009190 comp rx=0 tx=0).stop 2024-04-15T15:20:09.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.046+0000 7f00a90d3640 1 -- 172.21.15.105:0/3811707704 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00a40723a0 msgr2=0x7f00a4144910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:09.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.046+0000 7f00a90d3640 1 --2- 172.21.15.105:0/3811707704 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00a40723a0 0x7f00a4144910 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f008c034040 tx=0x7f008c0046c0 comp rx=0 tx=0).stop 2024-04-15T15:20:09.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.046+0000 7f00a90d3640 1 -- 172.21.15.105:0/3811707704 shutdown_connections 2024-04-15T15:20:09.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.046+0000 7f00a90d3640 1 --2- 172.21.15.105:0/3811707704 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00a4145e00 0x7f00a41f6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:09.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.046+0000 7f00a90d3640 1 --2- 172.21.15.105:0/3811707704 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f008406c8f0 0x7f008406edb0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:09.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.046+0000 7f00a90d3640 1 --2- 172.21.15.105:0/3811707704 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00a40ced30 0x7f00a4157c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:09.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.046+0000 7f00a90d3640 1 --2- 172.21.15.105:0/3811707704 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00a40723a0 0x7f00a4144910 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:09.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.046+0000 7f00a90d3640 1 -- 172.21.15.105:0/3811707704 >> 172.21.15.105:0/3811707704 conn(0x7f00a4067d50 msgr2=0x7f00a415f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:09.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.046+0000 7f00a90d3640 1 -- 172.21.15.105:0/3811707704 shutdown_connections 2024-04-15T15:20:09.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.046+0000 7f00a90d3640 1 -- 172.21.15.105:0/3811707704 wait complete. 2024-04-15T15:20:09.060 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:19:13.748585+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-15T15:19:23.789287+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-15T15:19:24.722250+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:20:09.060 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:20:09.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.164+0000 7fbd6922c640 1 Processor -- start 2024-04-15T15:20:09.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.164+0000 7fbd6922c640 1 -- start start 2024-04-15T15:20:09.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.165+0000 7fbd6922c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbd64145f00 0x7fbd64071e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:09.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.165+0000 7fbd6922c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbd640723a0 0x7fbd6406de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:09.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.165+0000 7fbd6922c640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd6406ebb0 con 0x7fbd6406e3c0 2024-04-15T15:20:09.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.165+0000 7fbd6922c640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fbd64072800 con 0x7fbd640723a0 2024-04-15T15:20:09.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.165+0000 7fbd6922c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fbd641462e0 con 0x7fbd64145f00 2024-04-15T15:20:09.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.165+0000 7fbd6259c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbd640723a0 0x7fbd6406de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:09.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.165+0000 7fbd6359e640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fbd6406e3c0 0x7fbd640ccec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:55210/0 (socket says 172.21.15.105:55210) 2024-04-15T15:20:09.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.165+0000 7fbd6359e640 1 -- 172.21.15.105:0/1806350490 learned_addr learned my addr 172.21.15.105:0/1806350490 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:20:09.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.165+0000 7fbd62d9d640 1 --2- 172.21.15.105:0/1806350490 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbd64145f00 0x7fbd64071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:09.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.165+0000 7fbd6259c640 1 -- 172.21.15.105:0/1806350490 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbd64145f00 msgr2=0x7fbd64071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:09.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.166+0000 7fbd6259c640 1 --2- 172.21.15.105:0/1806350490 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbd64145f00 0x7fbd64071e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:09.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.166+0000 7fbd6259c640 1 -- 172.21.15.105:0/1806350490 >> v1:172.21.15.105:6789/0 conn(0x7fbd6406e3c0 legacy=0x7fbd640ccec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:09.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.166+0000 7fbd6259c640 1 -- 172.21.15.105:0/1806350490 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd6406ee90 con 0x7fbd640723a0 2024-04-15T15:20:09.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.166+0000 7fbd61d9b640 1 -- 172.21.15.105:0/1806350490 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1439975434 0 0) 0x7fbd6406ebb0 con 0x7fbd6406e3c0 2024-04-15T15:20:09.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.166+0000 7fbd62d9d640 1 --2- 172.21.15.105:0/1806350490 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbd64145f00 0x7fbd64071e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:09.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.166+0000 7fbd6259c640 1 --2- 172.21.15.105:0/1806350490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd640723a0 0x7fbd6406de80 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fbd580097a0 tx=0x7fbd5802b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83f2fd457bda197b server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:09.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.166+0000 7fbd61d9b640 1 -- 172.21.15.105:0/1806350490 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd58038070 con 0x7fbd640723a0 2024-04-15T15:20:09.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.166+0000 7fbd61d9b640 1 -- 172.21.15.105:0/1806350490 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd58031700 con 0x7fbd640723a0 2024-04-15T15:20:09.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.166+0000 7fbd61d9b640 1 -- 172.21.15.105:0/1806350490 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd58031ad0 con 0x7fbd640723a0 2024-04-15T15:20:09.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.167+0000 7fbd6922c640 1 -- 172.21.15.105:0/1806350490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd640723a0 msgr2=0x7fbd6406de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:09.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.167+0000 7fbd6922c640 1 --2- 172.21.15.105:0/1806350490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd640723a0 0x7fbd6406de80 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fbd580097a0 tx=0x7fbd5802b7a0 comp rx=0 tx=0).stop 2024-04-15T15:20:09.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.167+0000 7fbd6922c640 1 -- 172.21.15.105:0/1806350490 shutdown_connections 2024-04-15T15:20:09.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.167+0000 7fbd6922c640 1 --2- 172.21.15.105:0/1806350490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd640723a0 0x7fbd6406de80 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:09.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.167+0000 7fbd6922c640 1 --2- 172.21.15.105:0/1806350490 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbd64145f00 0x7fbd64071e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:09.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.167+0000 7fbd6922c640 1 -- 172.21.15.105:0/1806350490 >> 172.21.15.105:0/1806350490 conn(0x7fbd64067d50 msgr2=0x7fbd6406b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:09.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.167+0000 7fbd6922c640 1 -- 172.21.15.105:0/1806350490 shutdown_connections 2024-04-15T15:20:09.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.167+0000 7fbd6922c640 1 -- 172.21.15.105:0/1806350490 wait complete. 2024-04-15T15:20:09.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.168+0000 7fbd6922c640 1 Processor -- start 2024-04-15T15:20:09.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.168+0000 7fbd6922c640 1 -- start start 2024-04-15T15:20:09.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.169+0000 7fbd6922c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbd6406e3c0 0x7fbd640c8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:09.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.169+0000 7fbd6922c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd640723a0 0x7fbd641400a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:09.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.169+0000 7fbd6922c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd64145f00 0x7fbd641405e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:09.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.169+0000 7fbd6922c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbd6406e7f0 con 0x7fbd64145f00 2024-04-15T15:20:09.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.169+0000 7fbd6922c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbd6415d0c0 con 0x7fbd640723a0 2024-04-15T15:20:09.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.169+0000 7fbd6922c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fbd6406e970 con 0x7fbd6406e3c0 2024-04-15T15:20:09.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.169+0000 7fbd6359e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd64145f00 0x7fbd641405e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:09.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.169+0000 7fbd6359e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd64145f00 0x7fbd641405e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37708/0 (socket says 172.21.15.105:37708) 2024-04-15T15:20:09.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.169+0000 7fbd6359e640 1 -- 172.21.15.105:0/3611954985 learned_addr learned my addr 172.21.15.105:0/3611954985 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:09.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.170+0000 7fbd6359e640 1 -- 172.21.15.105:0/3611954985 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbd6406e3c0 msgr2=0x7fbd640c8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:09.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.170+0000 7fbd6259c640 1 --2- 172.21.15.105:0/3611954985 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd640723a0 0x7fbd641400a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:09.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.170+0000 7fbd62d9d640 1 --2- 172.21.15.105:0/3611954985 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbd6406e3c0 0x7fbd640c8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:09.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.170+0000 7fbd6359e640 1 --2- 172.21.15.105:0/3611954985 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbd6406e3c0 0x7fbd640c8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:09.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.170+0000 7fbd6359e640 1 -- 172.21.15.105:0/3611954985 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd640723a0 msgr2=0x7fbd641400a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:09.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.170+0000 7fbd6359e640 1 --2- 172.21.15.105:0/3611954985 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd640723a0 0x7fbd641400a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:09.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.170+0000 7fbd6359e640 1 -- 172.21.15.105:0/3611954985 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd640c63c0 con 0x7fbd64145f00 2024-04-15T15:20:09.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.170+0000 7fbd6359e640 1 --2- 172.21.15.105:0/3611954985 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd64145f00 0x7fbd641405e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c00c450 tx=0x7fbd5c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:09.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.170+0000 7fbd4b7fe640 1 -- 172.21.15.105:0/3611954985 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd5c018070 con 0x7fbd64145f00 2024-04-15T15:20:09.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.170+0000 7fbd4b7fe640 1 -- 172.21.15.105:0/3611954985 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd5c007500 con 0x7fbd64145f00 2024-04-15T15:20:09.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.170+0000 7fbd6922c640 1 -- 172.21.15.105:0/3611954985 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd640c66a0 con 0x7fbd64145f00 2024-04-15T15:20:09.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.170+0000 7fbd4b7fe640 1 -- 172.21.15.105:0/3611954985 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd5c004a20 con 0x7fbd64145f00 2024-04-15T15:20:09.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.170+0000 7fbd6922c640 1 -- 172.21.15.105:0/3611954985 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd640c6c60 con 0x7fbd64145f00 2024-04-15T15:20:09.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.171+0000 7fbd6922c640 1 -- 172.21.15.105:0/3611954985 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd64068ab0 con 0x7fbd64145f00 2024-04-15T15:20:09.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.171+0000 7fbd4b7fe640 1 -- 172.21.15.105:0/3611954985 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbd5c01b070 con 0x7fbd64145f00 2024-04-15T15:20:09.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.172+0000 7fbd4b7fe640 1 --2- 172.21.15.105:0/3611954985 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fbd5006c8f0 0x7fbd5006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:09.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.172+0000 7fbd4b7fe640 1 -- 172.21.15.105:0/3611954985 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fbd5c08aea0 con 0x7fbd64145f00 2024-04-15T15:20:09.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.172+0000 7fbd62d9d640 1 --2- 172.21.15.105:0/3611954985 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fbd5006c8f0 0x7fbd5006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:09.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.172+0000 7fbd62d9d640 1 --2- 172.21.15.105:0/3611954985 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fbd5006c8f0 0x7fbd5006edb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbd54008ee0 tx=0x7fbd54008430 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:09.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.176+0000 7fbd4b7fe640 1 -- 172.21.15.105:0/3611954985 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbd5c059250 con 0x7fbd64145f00 2024-04-15T15:20:09.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.349+0000 7fbd6922c640 1 -- 172.21.15.105:0/3611954985 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbd640cdfb0 con 0x7fbd64145f00 2024-04-15T15:20:09.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.349+0000 7fbd4b7fe640 1 -- 172.21.15.105:0/3611954985 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7fbd5c058b50 con 0x7fbd64145f00 2024-04-15T15:20:09.351 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-04-15T15:20:09.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.351+0000 7fbd6922c640 1 -- 172.21.15.105:0/3611954985 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fbd5006c8f0 msgr2=0x7fbd5006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:09.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.351+0000 7fbd6922c640 1 --2- 172.21.15.105:0/3611954985 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fbd5006c8f0 0x7fbd5006edb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbd54008ee0 tx=0x7fbd54008430 comp rx=0 tx=0).stop 2024-04-15T15:20:09.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.351+0000 7fbd6922c640 1 -- 172.21.15.105:0/3611954985 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd64145f00 msgr2=0x7fbd641405e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:09.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.351+0000 7fbd6922c640 1 --2- 172.21.15.105:0/3611954985 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd64145f00 0x7fbd641405e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c00c450 tx=0x7fbd5c00c910 comp rx=0 tx=0).stop 2024-04-15T15:20:09.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.351+0000 7fbd6922c640 1 -- 172.21.15.105:0/3611954985 shutdown_connections 2024-04-15T15:20:09.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.351+0000 7fbd6922c640 1 --2- 172.21.15.105:0/3611954985 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fbd5006c8f0 0x7fbd5006edb0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:09.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.351+0000 7fbd6922c640 1 --2- 172.21.15.105:0/3611954985 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd64145f00 0x7fbd641405e0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:09.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.351+0000 7fbd6922c640 1 --2- 172.21.15.105:0/3611954985 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd640723a0 0x7fbd641400a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:09.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.351+0000 7fbd6922c640 1 --2- 172.21.15.105:0/3611954985 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbd6406e3c0 0x7fbd640c8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:09.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.351+0000 7fbd6922c640 1 -- 172.21.15.105:0/3611954985 >> 172.21.15.105:0/3611954985 conn(0x7fbd64067d50 msgr2=0x7fbd6406b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:09.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.351+0000 7fbd6922c640 1 -- 172.21.15.105:0/3611954985 shutdown_connections 2024-04-15T15:20:09.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.351+0000 7fbd6922c640 1 -- 172.21.15.105:0/3611954985 wait complete. 2024-04-15T15:20:09.364 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-15T15:20:09.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.467+0000 7fb044835640 1 Processor -- start 2024-04-15T15:20:09.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.467+0000 7fb044835640 1 -- start start 2024-04-15T15:20:09.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.468+0000 7fb044835640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb0401620c0 0x7fb0400b7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:09.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.468+0000 7fb044835640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb0400d2b40 0x7fb0400ba190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:09.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.468+0000 7fb044835640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb0401625c0 0x7fb040162a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:09.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.468+0000 7fb044835640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fb0400b8c00 con 0x7fb0401625c0 2024-04-15T15:20:09.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.468+0000 7fb044835640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fb0400d2fc0 con 0x7fb0401620c0 2024-04-15T15:20:09.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.468+0000 7fb044835640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fb0400b7ff0 con 0x7fb0400d2b40 2024-04-15T15:20:09.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.469+0000 7fb03ed9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb0401625c0 0x7fb040162a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:09.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.469+0000 7fb03ed9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb0401625c0 0x7fb040162a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37716/0 (socket says 172.21.15.105:37716) 2024-04-15T15:20:09.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.469+0000 7fb03ed9d640 1 -- 172.21.15.105:0/384633392 learned_addr learned my addr 172.21.15.105:0/384633392 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:09.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.469+0000 7fb03dd9b640 1 --2- 172.21.15.105:0/384633392 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb0400d2b40 0x7fb0400ba190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:09.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.469+0000 7fb03e59c640 1 --2- 172.21.15.105:0/384633392 >> v2:172.21.15.154:3300/0 conn(0x7fb0401620c0 0x7fb0400b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:09.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.469+0000 7fb03ed9d640 1 -- 172.21.15.105:0/384633392 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb0400d2b40 msgr2=0x7fb0400ba190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:09.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.469+0000 7fb03ed9d640 1 --2- 172.21.15.105:0/384633392 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb0400d2b40 0x7fb0400ba190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:09.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.469+0000 7fb03ed9d640 1 -- 172.21.15.105:0/384633392 >> v2:172.21.15.154:3300/0 conn(0x7fb0401620c0 msgr2=0x7fb0400b7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:09.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.469+0000 7fb03ed9d640 1 --2- 172.21.15.105:0/384633392 >> v2:172.21.15.154:3300/0 conn(0x7fb0401620c0 0x7fb0400b7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:09.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.469+0000 7fb03ed9d640 1 -- 172.21.15.105:0/384633392 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0400b82d0 con 0x7fb0401625c0 2024-04-15T15:20:09.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.470+0000 7fb03ed9d640 1 --2- 172.21.15.105:0/384633392 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0401625c0 0x7fb040162a50 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb04006d9d0 tx=0x7fb03802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28baa3294771d1ed server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:09.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.470+0000 7fb03d59a640 1 -- 172.21.15.105:0/384633392 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb038035020 con 0x7fb0401625c0 2024-04-15T15:20:09.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.470+0000 7fb03d59a640 1 -- 172.21.15.105:0/384633392 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb038031870 con 0x7fb0401625c0 2024-04-15T15:20:09.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.470+0000 7fb03d59a640 1 -- 172.21.15.105:0/384633392 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb038031c40 con 0x7fb0401625c0 2024-04-15T15:20:09.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.470+0000 7fb044835640 1 -- 172.21.15.105:0/384633392 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0401625c0 msgr2=0x7fb040162a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:09.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.470+0000 7fb044835640 1 --2- 172.21.15.105:0/384633392 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0401625c0 0x7fb040162a50 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb04006d9d0 tx=0x7fb03802b910 comp rx=0 tx=0).stop 2024-04-15T15:20:09.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.470+0000 7fb044835640 1 -- 172.21.15.105:0/384633392 shutdown_connections 2024-04-15T15:20:09.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.470+0000 7fb044835640 1 --2- 172.21.15.105:0/384633392 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0401625c0 0x7fb040162a50 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:09.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.470+0000 7fb044835640 1 --2- 172.21.15.105:0/384633392 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb0400d2b40 0x7fb0400ba190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:09.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.470+0000 7fb044835640 1 --2- 172.21.15.105:0/384633392 >> v2:172.21.15.154:3300/0 conn(0x7fb0401620c0 0x7fb0400b7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:09.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.470+0000 7fb044835640 1 -- 172.21.15.105:0/384633392 >> 172.21.15.105:0/384633392 conn(0x7fb04006ccb0 msgr2=0x7fb04006d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:09.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.470+0000 7fb044835640 1 -- 172.21.15.105:0/384633392 shutdown_connections 2024-04-15T15:20:09.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.470+0000 7fb044835640 1 -- 172.21.15.105:0/384633392 wait complete. 2024-04-15T15:20:09.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.472+0000 7fb044835640 1 Processor -- start 2024-04-15T15:20:09.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.472+0000 7fb044835640 1 -- start start 2024-04-15T15:20:09.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.472+0000 7fb044835640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0400d2b40 0x7fb040156450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:09.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.472+0000 7fb044835640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb0401620c0 0x7fb0400cd230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:09.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.472+0000 7fb044835640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb0401625c0 0x7fb0401e91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:09.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.472+0000 7fb044835640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb0400ba6d0 con 0x7fb0400d2b40 2024-04-15T15:20:09.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.472+0000 7fb044835640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb0400b8890 con 0x7fb0401620c0 2024-04-15T15:20:09.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.472+0000 7fb044835640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fb0400b9090 con 0x7fb0401625c0 2024-04-15T15:20:09.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.473+0000 7fb03e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0400d2b40 0x7fb040156450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:09.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.473+0000 7fb03e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0400d2b40 0x7fb040156450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37726/0 (socket says 172.21.15.105:37726) 2024-04-15T15:20:09.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.473+0000 7fb03e59c640 1 -- 172.21.15.105:0/3913386410 learned_addr learned my addr 172.21.15.105:0/3913386410 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:09.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.473+0000 7fb03dd9b640 1 --2- 172.21.15.105:0/3913386410 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb0401620c0 0x7fb0400cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:09.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.473+0000 7fb03ed9d640 1 --2- 172.21.15.105:0/3913386410 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb0401625c0 0x7fb0401e91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:09.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.473+0000 7fb03e59c640 1 -- 172.21.15.105:0/3913386410 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb0401625c0 msgr2=0x7fb0401e91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:09.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.473+0000 7fb03e59c640 1 --2- 172.21.15.105:0/3913386410 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb0401625c0 0x7fb0401e91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:09.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.473+0000 7fb03e59c640 1 -- 172.21.15.105:0/3913386410 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb0401620c0 msgr2=0x7fb0400cd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:09.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.473+0000 7fb03e59c640 1 --2- 172.21.15.105:0/3913386410 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb0401620c0 0x7fb0400cd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:09.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.473+0000 7fb03e59c640 1 -- 172.21.15.105:0/3913386410 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb040156990 con 0x7fb0400d2b40 2024-04-15T15:20:09.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.473+0000 7fb03e59c640 1 --2- 172.21.15.105:0/3913386410 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0400d2b40 0x7fb040156450 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb03000efc0 tx=0x7fb03000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:09.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.473+0000 7fb036ffd640 1 -- 172.21.15.105:0/3913386410 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb030018020 con 0x7fb0400d2b40 2024-04-15T15:20:09.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.473+0000 7fb036ffd640 1 -- 172.21.15.105:0/3913386410 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb030014480 con 0x7fb0400d2b40 2024-04-15T15:20:09.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.473+0000 7fb044835640 1 -- 172.21.15.105:0/3913386410 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0401553b0 con 0x7fb0400d2b40 2024-04-15T15:20:09.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.473+0000 7fb036ffd640 1 -- 172.21.15.105:0/3913386410 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb030014850 con 0x7fb0400d2b40 2024-04-15T15:20:09.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.473+0000 7fb044835640 1 -- 172.21.15.105:0/3913386410 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb040155970 con 0x7fb0400d2b40 2024-04-15T15:20:09.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.474+0000 7fb044835640 1 -- 172.21.15.105:0/3913386410 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb040156080 con 0x7fb0400d2b40 2024-04-15T15:20:09.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.475+0000 7fb036ffd640 1 -- 172.21.15.105:0/3913386410 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb030005d10 con 0x7fb0400d2b40 2024-04-15T15:20:09.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.475+0000 7fb036ffd640 1 --2- 172.21.15.105:0/3913386410 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fb02006c920 0x7fb02006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:09.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.475+0000 7fb036ffd640 1 -- 172.21.15.105:0/3913386410 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fb03008c580 con 0x7fb0400d2b40 2024-04-15T15:20:09.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.475+0000 7fb03dd9b640 1 --2- 172.21.15.105:0/3913386410 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fb02006c920 0x7fb02006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:09.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.476+0000 7fb03dd9b640 1 --2- 172.21.15.105:0/3913386410 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fb02006c920 0x7fb02006ede0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb0400cf230 tx=0x7fb028009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:09.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.479+0000 7fb036ffd640 1 -- 172.21.15.105:0/3913386410 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb030022070 con 0x7fb0400d2b40 2024-04-15T15:20:09.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.654+0000 7fb044835640 1 -- 172.21.15.105:0/3913386410 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb0400d2410 con 0x7fb0400d2b40 2024-04-15T15:20:09.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.655+0000 7fb036ffd640 1 -- 172.21.15.105:0/3913386410 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7fb030010040 con 0x7fb0400d2b40 2024-04-15T15:20:09.657 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-04-15T15:20:09.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.657+0000 7fb044835640 1 -- 172.21.15.105:0/3913386410 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fb02006c920 msgr2=0x7fb02006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:09.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.657+0000 7fb044835640 1 --2- 172.21.15.105:0/3913386410 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fb02006c920 0x7fb02006ede0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb0400cf230 tx=0x7fb028009190 comp rx=0 tx=0).stop 2024-04-15T15:20:09.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.657+0000 7fb044835640 1 -- 172.21.15.105:0/3913386410 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0400d2b40 msgr2=0x7fb040156450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:09.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.657+0000 7fb044835640 1 --2- 172.21.15.105:0/3913386410 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0400d2b40 0x7fb040156450 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb03000efc0 tx=0x7fb03000c490 comp rx=0 tx=0).stop 2024-04-15T15:20:09.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.657+0000 7fb044835640 1 -- 172.21.15.105:0/3913386410 shutdown_connections 2024-04-15T15:20:09.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.657+0000 7fb044835640 1 --2- 172.21.15.105:0/3913386410 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb0401625c0 0x7fb0401e91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:09.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.657+0000 7fb044835640 1 --2- 172.21.15.105:0/3913386410 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fb02006c920 0x7fb02006ede0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:09.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.657+0000 7fb044835640 1 --2- 172.21.15.105:0/3913386410 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb0401620c0 0x7fb0400cd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:09.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.657+0000 7fb044835640 1 --2- 172.21.15.105:0/3913386410 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0400d2b40 0x7fb040156450 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:09.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.657+0000 7fb044835640 1 -- 172.21.15.105:0/3913386410 >> 172.21.15.105:0/3913386410 conn(0x7fb04006ccb0 msgr2=0x7fb04015cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:09.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.657+0000 7fb044835640 1 -- 172.21.15.105:0/3913386410 shutdown_connections 2024-04-15T15:20:09.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.657+0000 7fb044835640 1 -- 172.21.15.105:0/3913386410 wait complete. 2024-04-15T15:20:09.670 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-15T15:20:09.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.775+0000 7ffa85c37640 1 Processor -- start 2024-04-15T15:20:09.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.775+0000 7ffa85c37640 1 -- start start 2024-04-15T15:20:09.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.776+0000 7ffa85c37640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ffa8015eaf0 0x7ffa800b6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:09.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.776+0000 7ffa85c37640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa800cfe60 con 0x7ffa800d3f00 2024-04-15T15:20:09.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.776+0000 7ffa85c37640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa80162a10 con 0x7ffa800d1560 2024-04-15T15:20:09.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.776+0000 7ffa85c37640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7ffa800b71b0 con 0x7ffa8015eaf0 2024-04-15T15:20:09.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.776+0000 7ffa7f7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ffa8015eaf0 0x7ffa800b6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:09.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.776+0000 7ffa7ffff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7ffa800d3f00 0x7ffa800cf750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:55224/0 (socket says 172.21.15.105:55224) 2024-04-15T15:20:09.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.776+0000 7ffa7f7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ffa8015eaf0 0x7ffa800b6c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37732/0 (socket says 172.21.15.105:37732) 2024-04-15T15:20:09.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.776+0000 7ffa7ffff640 1 -- 172.21.15.105:0/2200076882 learned_addr learned my addr 172.21.15.105:0/2200076882 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:20:09.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.776+0000 7ffa7f7fe640 1 -- 172.21.15.105:0/2200076882 >> v1:172.21.15.154:6789/0 conn(0x7ffa800d1560 legacy=0x7ffa800cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:09.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.776+0000 7ffa7f7fe640 1 -- 172.21.15.105:0/2200076882 >> v1:172.21.15.105:6789/0 conn(0x7ffa800d3f00 legacy=0x7ffa800cf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:09.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.776+0000 7ffa7f7fe640 1 -- 172.21.15.105:0/2200076882 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa80162cf0 con 0x7ffa8015eaf0 2024-04-15T15:20:09.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.776+0000 7ffa7effd640 1 -- 172.21.15.105:0/2200076882 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 961734342 0 0) 0x7ffa800cfe60 con 0x7ffa800d3f00 2024-04-15T15:20:09.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.777+0000 7ffa7f7fe640 1 --2- 172.21.15.105:0/2200076882 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa8015eaf0 0x7ffa800b6c70 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ffa700097a0 tx=0x7ffa7002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4147618065b30d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:09.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.777+0000 7ffa7effd640 1 -- 172.21.15.105:0/2200076882 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa70035020 con 0x7ffa8015eaf0 2024-04-15T15:20:09.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.777+0000 7ffa7effd640 1 -- 172.21.15.105:0/2200076882 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa70031830 con 0x7ffa8015eaf0 2024-04-15T15:20:09.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.777+0000 7ffa7effd640 1 -- 172.21.15.105:0/2200076882 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa70031c00 con 0x7ffa8015eaf0 2024-04-15T15:20:09.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.778+0000 7ffa85c37640 1 -- 172.21.15.105:0/2200076882 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa8015eaf0 msgr2=0x7ffa800b6c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:09.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.778+0000 7ffa85c37640 1 --2- 172.21.15.105:0/2200076882 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa8015eaf0 0x7ffa800b6c70 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ffa700097a0 tx=0x7ffa7002b8d0 comp rx=0 tx=0).stop 2024-04-15T15:20:09.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.778+0000 7ffa85c37640 1 -- 172.21.15.105:0/2200076882 shutdown_connections 2024-04-15T15:20:09.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.778+0000 7ffa85c37640 1 --2- 172.21.15.105:0/2200076882 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa8015eaf0 0x7ffa800b6c70 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:09.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.778+0000 7ffa85c37640 1 -- 172.21.15.105:0/2200076882 >> 172.21.15.105:0/2200076882 conn(0x7ffa8006cf90 msgr2=0x7ffa8006d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:09.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.778+0000 7ffa85c37640 1 -- 172.21.15.105:0/2200076882 shutdown_connections 2024-04-15T15:20:09.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.778+0000 7ffa85c37640 1 -- 172.21.15.105:0/2200076882 wait complete. 2024-04-15T15:20:09.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.779+0000 7ffa85c37640 1 Processor -- start 2024-04-15T15:20:09.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.779+0000 7ffa85c37640 1 -- start start 2024-04-15T15:20:09.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.779+0000 7ffa85c37640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffa800d1560 0x7ffa80157d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:09.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.779+0000 7ffa85c37640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffa800d3f00 0x7ffa801f4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:09.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.779+0000 7ffa85c37640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa8015eaf0 0x7ffa8015afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:09.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.779+0000 7ffa85c37640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8015eed0 con 0x7ffa8015eaf0 2024-04-15T15:20:09.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.780+0000 7ffa85c37640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ffa801632b0 con 0x7ffa800d3f00 2024-04-15T15:20:09.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.780+0000 7ffa85c37640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7ffa800b3d60 con 0x7ffa800d1560 2024-04-15T15:20:09.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.780+0000 7ffa7ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa8015eaf0 0x7ffa8015afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:09.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.780+0000 7ffa7ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa8015eaf0 0x7ffa8015afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37746/0 (socket says 172.21.15.105:37746) 2024-04-15T15:20:09.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.780+0000 7ffa7ffff640 1 -- 172.21.15.105:0/2122179524 learned_addr learned my addr 172.21.15.105:0/2122179524 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:09.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.780+0000 7ffa77fff640 1 --2- 172.21.15.105:0/2122179524 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffa800d3f00 0x7ffa801f4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:09.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.780+0000 7ffa7f7fe640 1 --2- 172.21.15.105:0/2122179524 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffa800d1560 0x7ffa80157d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:09.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.780+0000 7ffa7ffff640 1 -- 172.21.15.105:0/2122179524 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffa800d1560 msgr2=0x7ffa80157d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:09.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.780+0000 7ffa7ffff640 1 --2- 172.21.15.105:0/2122179524 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffa800d1560 0x7ffa80157d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:09.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.780+0000 7ffa7ffff640 1 -- 172.21.15.105:0/2122179524 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffa800d3f00 msgr2=0x7ffa801f4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:09.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.780+0000 7ffa7ffff640 1 --2- 172.21.15.105:0/2122179524 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffa800d3f00 0x7ffa801f4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:09.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.780+0000 7ffa7ffff640 1 -- 172.21.15.105:0/2122179524 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa801e9ba0 con 0x7ffa8015eaf0 2024-04-15T15:20:09.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.780+0000 7ffa7ffff640 1 --2- 172.21.15.105:0/2122179524 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa8015eaf0 0x7ffa8015afb0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ffa7800c7d0 tx=0x7ffa7800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:09.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.780+0000 7ffa7cff9640 1 -- 172.21.15.105:0/2122179524 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa78018070 con 0x7ffa8015eaf0 2024-04-15T15:20:09.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.780+0000 7ffa7cff9640 1 -- 172.21.15.105:0/2122179524 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa7800a5e0 con 0x7ffa8015eaf0 2024-04-15T15:20:09.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.780+0000 7ffa85c37640 1 -- 172.21.15.105:0/2122179524 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa8015a1f0 con 0x7ffa8015eaf0 2024-04-15T15:20:09.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.780+0000 7ffa7cff9640 1 -- 172.21.15.105:0/2122179524 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7800aa10 con 0x7ffa8015eaf0 2024-04-15T15:20:09.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.780+0000 7ffa85c37640 1 -- 172.21.15.105:0/2122179524 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa8015a7b0 con 0x7ffa8015eaf0 2024-04-15T15:20:09.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.781+0000 7ffa85c37640 1 -- 172.21.15.105:0/2122179524 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa80068ab0 con 0x7ffa8015eaf0 2024-04-15T15:20:09.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.782+0000 7ffa7cff9640 1 -- 172.21.15.105:0/2122179524 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffa7801b070 con 0x7ffa8015eaf0 2024-04-15T15:20:09.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.782+0000 7ffa7cff9640 1 --2- 172.21.15.105:0/2122179524 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ffa6006c8f0 0x7ffa6006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:09.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.782+0000 7ffa7cff9640 1 -- 172.21.15.105:0/2122179524 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7ffa7808b060 con 0x7ffa8015eaf0 2024-04-15T15:20:09.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.782+0000 7ffa7f7fe640 1 --2- 172.21.15.105:0/2122179524 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ffa6006c8f0 0x7ffa6006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:09.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.783+0000 7ffa7f7fe640 1 --2- 172.21.15.105:0/2122179524 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ffa6006c8f0 0x7ffa6006edb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ffa70036000 tx=0x7ffa70008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:09.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.787+0000 7ffa7cff9640 1 -- 172.21.15.105:0/2122179524 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ffa78058e50 con 0x7ffa8015eaf0 2024-04-15T15:20:09.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.959+0000 7ffa85c37640 1 -- 172.21.15.105:0/2122179524 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ffa80161e60 con 0x7ffa8015eaf0 2024-04-15T15:20:09.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.960+0000 7ffa7cff9640 1 -- 172.21.15.105:0/2122179524 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7ffa7805d150 con 0x7ffa8015eaf0 2024-04-15T15:20:09.962 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-04-15T15:20:09.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.962+0000 7ffa85c37640 1 -- 172.21.15.105:0/2122179524 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ffa6006c8f0 msgr2=0x7ffa6006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:09.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.962+0000 7ffa85c37640 1 --2- 172.21.15.105:0/2122179524 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ffa6006c8f0 0x7ffa6006edb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ffa70036000 tx=0x7ffa70008030 comp rx=0 tx=0).stop 2024-04-15T15:20:09.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.962+0000 7ffa85c37640 1 -- 172.21.15.105:0/2122179524 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa8015eaf0 msgr2=0x7ffa8015afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:09.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.962+0000 7ffa85c37640 1 --2- 172.21.15.105:0/2122179524 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa8015eaf0 0x7ffa8015afb0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ffa7800c7d0 tx=0x7ffa7800cc90 comp rx=0 tx=0).stop 2024-04-15T15:20:09.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.962+0000 7ffa85c37640 1 -- 172.21.15.105:0/2122179524 shutdown_connections 2024-04-15T15:20:09.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.962+0000 7ffa85c37640 1 --2- 172.21.15.105:0/2122179524 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ffa6006c8f0 0x7ffa6006edb0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:09.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.962+0000 7ffa85c37640 1 --2- 172.21.15.105:0/2122179524 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa8015eaf0 0x7ffa8015afb0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:09.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.962+0000 7ffa85c37640 1 --2- 172.21.15.105:0/2122179524 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffa800d3f00 0x7ffa801f4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:09.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.962+0000 7ffa85c37640 1 --2- 172.21.15.105:0/2122179524 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffa800d1560 0x7ffa80157d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:09.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.962+0000 7ffa85c37640 1 -- 172.21.15.105:0/2122179524 >> 172.21.15.105:0/2122179524 conn(0x7ffa8006cf90 msgr2=0x7ffa800d4c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:09.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.962+0000 7ffa85c37640 1 -- 172.21.15.105:0/2122179524 shutdown_connections 2024-04-15T15:20:09.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:09.962+0000 7ffa85c37640 1 -- 172.21.15.105:0/2122179524 wait complete. 2024-04-15T15:20:09.975 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-15T15:20:10.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.076+0000 7f2919fe9640 1 Processor -- start 2024-04-15T15:20:10.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.076+0000 7f2919fe9640 1 -- start start 2024-04-15T15:20:10.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.077+0000 7f2919fe9640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f29140b2430 0x7f29140b1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:10.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.077+0000 7f2919fe9640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f291409afe0 con 0x7f291415e940 2024-04-15T15:20:10.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.077+0000 7f2919fe9640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f291409b2c0 con 0x7f29140cf650 2024-04-15T15:20:10.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.077+0000 7f2919fe9640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f29140b2120 con 0x7f29140b2430 2024-04-15T15:20:10.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.077+0000 7f29137fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f291415e940 0x7f29140cef40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:55226/0 (socket says 172.21.15.105:55226) 2024-04-15T15:20:10.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.077+0000 7f29137fe640 1 -- 172.21.15.105:0/1200763442 learned_addr learned my addr 172.21.15.105:0/1200763442 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:20:10.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.077+0000 7f2913fff640 1 --2- 172.21.15.105:0/1200763442 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f29140b2430 0x7f29140b1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:10.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.077+0000 7f2912ffd640 1 -- 172.21.15.105:0/1200763442 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2061397438 0 0) 0x7f291409afe0 con 0x7f291415e940 2024-04-15T15:20:10.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.077+0000 7f2912ffd640 1 -- 172.21.15.105:0/1200763442 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2900003520 con 0x7f291415e940 2024-04-15T15:20:10.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.077+0000 7f2912ffd640 1 -- 172.21.15.105:0/1200763442 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 652603218 0 0) 0x7f291409b2c0 con 0x7f29140cf650 2024-04-15T15:20:10.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.077+0000 7f2912ffd640 1 -- 172.21.15.105:0/1200763442 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f291409afe0 con 0x7f29140cf650 2024-04-15T15:20:10.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.077+0000 7f2913fff640 1 -- 172.21.15.105:0/1200763442 >> v1:172.21.15.154:6789/0 conn(0x7f29140cf650 legacy=0x7f29140d0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:10.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.077+0000 7f2913fff640 1 -- 172.21.15.105:0/1200763442 >> v1:172.21.15.105:6789/0 conn(0x7f291415e940 legacy=0x7f29140cef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:10.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.077+0000 7f2913fff640 1 -- 172.21.15.105:0/1200763442 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f291409b5a0 con 0x7f29140b2430 2024-04-15T15:20:10.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.078+0000 7f2913fff640 1 --2- 172.21.15.105:0/1200763442 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f29140b2430 0x7f29140b1be0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f290c009dc0 tx=0x7f290c02be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c16b2e6ac1bf412c server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:10.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.078+0000 7f2912ffd640 1 -- 172.21.15.105:0/1200763442 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290c039070 con 0x7f29140b2430 2024-04-15T15:20:10.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.078+0000 7f2912ffd640 1 -- 172.21.15.105:0/1200763442 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f290c02cc40 con 0x7f29140b2430 2024-04-15T15:20:10.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.078+0000 7f2912ffd640 1 -- 172.21.15.105:0/1200763442 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290c034440 con 0x7f29140b2430 2024-04-15T15:20:10.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.079+0000 7f2919fe9640 1 -- 172.21.15.105:0/1200763442 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f29140b2430 msgr2=0x7f29140b1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:10.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.079+0000 7f2919fe9640 1 --2- 172.21.15.105:0/1200763442 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f29140b2430 0x7f29140b1be0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f290c009dc0 tx=0x7f290c02be90 comp rx=0 tx=0).stop 2024-04-15T15:20:10.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.079+0000 7f2919fe9640 1 -- 172.21.15.105:0/1200763442 shutdown_connections 2024-04-15T15:20:10.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.079+0000 7f2919fe9640 1 --2- 172.21.15.105:0/1200763442 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f29140b2430 0x7f29140b1be0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:10.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.079+0000 7f2919fe9640 1 -- 172.21.15.105:0/1200763442 >> 172.21.15.105:0/1200763442 conn(0x7f29140d25a0 msgr2=0x7f29140d29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:10.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.079+0000 7f2919fe9640 1 -- 172.21.15.105:0/1200763442 shutdown_connections 2024-04-15T15:20:10.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.079+0000 7f2919fe9640 1 -- 172.21.15.105:0/1200763442 wait complete. 2024-04-15T15:20:10.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.080+0000 7f2919fe9640 1 Processor -- start 2024-04-15T15:20:10.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.080+0000 7f2919fe9640 1 -- start start 2024-04-15T15:20:10.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.081+0000 7f2919fe9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f29140b2430 0x7f2914151900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:10.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.081+0000 7f2919fe9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29140cf650 0x7f29141614d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:10.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.081+0000 7f2919fe9640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f291415e940 0x7f2914161a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:10.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.081+0000 7f2919fe9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f29140d0730 con 0x7f29140cf650 2024-04-15T15:20:10.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.081+0000 7f2919fe9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f291415ed20 con 0x7f29140b2430 2024-04-15T15:20:10.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.081+0000 7f2919fe9640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f291409aa40 con 0x7f291415e940 2024-04-15T15:20:10.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.081+0000 7f290bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29140cf650 0x7f29141614d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:10.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.081+0000 7f290bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29140cf650 0x7f29141614d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37760/0 (socket says 172.21.15.105:37760) 2024-04-15T15:20:10.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.081+0000 7f290bfff640 1 -- 172.21.15.105:0/4115381562 learned_addr learned my addr 172.21.15.105:0/4115381562 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:10.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.081+0000 7f2913fff640 1 --2- 172.21.15.105:0/4115381562 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f291415e940 0x7f2914161a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:10.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.081+0000 7f29137fe640 1 --2- 172.21.15.105:0/4115381562 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f29140b2430 0x7f2914151900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:10.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.081+0000 7f290bfff640 1 -- 172.21.15.105:0/4115381562 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f291415e940 msgr2=0x7f2914161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:10.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.081+0000 7f290bfff640 1 --2- 172.21.15.105:0/4115381562 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f291415e940 0x7f2914161a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:10.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.081+0000 7f290bfff640 1 -- 172.21.15.105:0/4115381562 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f29140b2430 msgr2=0x7f2914151900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:10.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.081+0000 7f290bfff640 1 --2- 172.21.15.105:0/4115381562 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f29140b2430 0x7f2914151900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:10.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.081+0000 7f290bfff640 1 -- 172.21.15.105:0/4115381562 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29140b72b0 con 0x7f29140cf650 2024-04-15T15:20:10.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.081+0000 7f2913fff640 1 --2- 172.21.15.105:0/4115381562 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f291415e940 0x7f2914161a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:20:10.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.081+0000 7f290bfff640 1 --2- 172.21.15.105:0/4115381562 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29140cf650 0x7f29141614d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f28fc00c500 tx=0x7f28fc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:10.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.082+0000 7f2910ff9640 1 -- 172.21.15.105:0/4115381562 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28fc0052d0 con 0x7f29140cf650 2024-04-15T15:20:10.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.082+0000 7f2910ff9640 1 -- 172.21.15.105:0/4115381562 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28fc005490 con 0x7f29140cf650 2024-04-15T15:20:10.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.082+0000 7f2919fe9640 1 -- 172.21.15.105:0/4115381562 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29140b7550 con 0x7f29140cf650 2024-04-15T15:20:10.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.082+0000 7f2910ff9640 1 -- 172.21.15.105:0/4115381562 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28fc012870 con 0x7f29140cf650 2024-04-15T15:20:10.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.082+0000 7f2919fe9640 1 -- 172.21.15.105:0/4115381562 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29140b7b10 con 0x7f29140cf650 2024-04-15T15:20:10.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.083+0000 7f2919fe9640 1 -- 172.21.15.105:0/4115381562 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2914069150 con 0x7f29140cf650 2024-04-15T15:20:10.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.083+0000 7f2910ff9640 1 -- 172.21.15.105:0/4115381562 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f28fc012a10 con 0x7f29140cf650 2024-04-15T15:20:10.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.084+0000 7f2910ff9640 1 --2- 172.21.15.105:0/4115381562 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f28f406c8f0 0x7f28f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:10.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.084+0000 7f29137fe640 1 --2- 172.21.15.105:0/4115381562 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f28f406c8f0 0x7f28f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:10.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.084+0000 7f2910ff9640 1 -- 172.21.15.105:0/4115381562 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f28fc08b990 con 0x7f29140cf650 2024-04-15T15:20:10.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.084+0000 7f29137fe640 1 --2- 172.21.15.105:0/4115381562 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f28f406c8f0 0x7f28f406edb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f29040011e0 tx=0x7f2904009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:10.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.088+0000 7f2910ff9640 1 -- 172.21.15.105:0/4115381562 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f28fc059d50 con 0x7f29140cf650 2024-04-15T15:20:10.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.297+0000 7f2919fe9640 1 -- 172.21.15.105:0/4115381562 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f29140b8520 con 0x7f29140cf650 2024-04-15T15:20:10.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.297+0000 7f2910ff9640 1 -- 172.21.15.105:0/4115381562 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5187 (secure 0 0 0) 0x7f28fc059650 con 0x7f29140cf650 2024-04-15T15:20:10.299 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:20:10.299 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4259,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/3319657830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":3319657830},{"type":"v1","addr":"172.21.15.154:6835","nonce":3319657830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2677199890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2677199890},{"type":"v1","addr":"172.21.15.105:6835","nonce":2677199890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2867739743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2867739743},{"type":"v1","addr":"172.21.15.105:6837","nonce":2867739743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-15T15:19:25.760362+0000","modified":"2024-04-15T15:19:27.857599+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.154:6834/1529675504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1529675504},{"type":"v1","addr":"172.21.15.154:6834","nonce":1529675504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4291,"qdb_cluster":[4291]},"id":1}]} 2024-04-15T15:20:10.300 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 6 2024-04-15T15:20:10.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.299+0000 7f2919fe9640 1 -- 172.21.15.105:0/4115381562 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f28f406c8f0 msgr2=0x7f28f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:10.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.299+0000 7f2919fe9640 1 --2- 172.21.15.105:0/4115381562 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f28f406c8f0 0x7f28f406edb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f29040011e0 tx=0x7f2904009040 comp rx=0 tx=0).stop 2024-04-15T15:20:10.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.299+0000 7f2919fe9640 1 -- 172.21.15.105:0/4115381562 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29140cf650 msgr2=0x7f29141614d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:10.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.299+0000 7f2919fe9640 1 --2- 172.21.15.105:0/4115381562 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29140cf650 0x7f29141614d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f28fc00c500 tx=0x7f28fc00c9c0 comp rx=0 tx=0).stop 2024-04-15T15:20:10.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.299+0000 7f2919fe9640 1 -- 172.21.15.105:0/4115381562 shutdown_connections 2024-04-15T15:20:10.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.299+0000 7f2919fe9640 1 --2- 172.21.15.105:0/4115381562 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f291415e940 0x7f2914161a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:10.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.299+0000 7f2919fe9640 1 --2- 172.21.15.105:0/4115381562 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f28f406c8f0 0x7f28f406edb0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:10.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.299+0000 7f2919fe9640 1 --2- 172.21.15.105:0/4115381562 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29140cf650 0x7f29141614d0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:10.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.299+0000 7f2919fe9640 1 --2- 172.21.15.105:0/4115381562 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f29140b2430 0x7f2914151900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:10.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.299+0000 7f2919fe9640 1 -- 172.21.15.105:0/4115381562 >> 172.21.15.105:0/4115381562 conn(0x7f29140d25a0 msgr2=0x7f29140cef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:10.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.299+0000 7f2919fe9640 1 -- 172.21.15.105:0/4115381562 shutdown_connections 2024-04-15T15:20:10.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.299+0000 7f2919fe9640 1 -- 172.21.15.105:0/4115381562 wait complete. 2024-04-15T15:20:10.312 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:20:10.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.413+0000 7ff88bfc8640 1 Processor -- start 2024-04-15T15:20:10.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.413+0000 7ff88bfc8640 1 -- start start 2024-04-15T15:20:10.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.414+0000 7ff88bfc8640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff88415ecf0 0x7ff88406e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:10.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.414+0000 7ff88bfc8640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff88406d2a0 con 0x7ff8840cd360 2024-04-15T15:20:10.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.414+0000 7ff88bfc8640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff88406d580 con 0x7ff8840d0f10 2024-04-15T15:20:10.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.414+0000 7ff88bfc8640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7ff88406d860 con 0x7ff88415ecf0 2024-04-15T15:20:10.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.414+0000 7ff889d64640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff88415ecf0 0x7ff88406e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:10.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.414+0000 7ff889d64640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff88415ecf0 0x7ff88406e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37772/0 (socket says 172.21.15.105:37772) 2024-04-15T15:20:10.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.414+0000 7ff889d64640 1 -- 172.21.15.105:0/2462896016 learned_addr learned my addr 172.21.15.105:0/2462896016 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:10.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.414+0000 7ff88a565640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7ff8840cd360 0x7ff884162960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:55228/0 (socket says 172.21.15.105:55228) 2024-04-15T15:20:10.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.414+0000 7ff889d64640 1 -- 172.21.15.105:0/2462896016 >> v1:172.21.15.154:6789/0 conn(0x7ff8840d0f10 legacy=0x7ff8840ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:10.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.414+0000 7ff889d64640 1 -- 172.21.15.105:0/2462896016 >> v1:172.21.15.105:6789/0 conn(0x7ff8840cd360 legacy=0x7ff884162960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:10.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.414+0000 7ff889d64640 1 -- 172.21.15.105:0/2462896016 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff88406db40 con 0x7ff88415ecf0 2024-04-15T15:20:10.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.415+0000 7ff889d64640 1 --2- 172.21.15.105:0/2462896016 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff88415ecf0 0x7ff88406e060 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff880009870 tx=0x7ff88002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1afd5f817552943 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:10.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.415+0000 7ff888d62640 1 -- 172.21.15.105:0/2462896016 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff880034070 con 0x7ff88415ecf0 2024-04-15T15:20:10.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.415+0000 7ff888d62640 1 -- 172.21.15.105:0/2462896016 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff880030860 con 0x7ff88415ecf0 2024-04-15T15:20:10.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.415+0000 7ff888d62640 1 -- 172.21.15.105:0/2462896016 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff880030c30 con 0x7ff88415ecf0 2024-04-15T15:20:10.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.416+0000 7ff88bfc8640 1 -- 172.21.15.105:0/2462896016 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff88415ecf0 msgr2=0x7ff88406e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:10.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.416+0000 7ff88bfc8640 1 --2- 172.21.15.105:0/2462896016 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff88415ecf0 0x7ff88406e060 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff880009870 tx=0x7ff88002b9a0 comp rx=0 tx=0).stop 2024-04-15T15:20:10.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.416+0000 7ff88bfc8640 1 -- 172.21.15.105:0/2462896016 shutdown_connections 2024-04-15T15:20:10.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.416+0000 7ff88bfc8640 1 --2- 172.21.15.105:0/2462896016 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff88415ecf0 0x7ff88406e060 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:10.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.416+0000 7ff88bfc8640 1 -- 172.21.15.105:0/2462896016 >> 172.21.15.105:0/2462896016 conn(0x7ff8840d2600 msgr2=0x7ff8840d2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:10.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.416+0000 7ff88bfc8640 1 -- 172.21.15.105:0/2462896016 shutdown_connections 2024-04-15T15:20:10.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.416+0000 7ff88bfc8640 1 -- 172.21.15.105:0/2462896016 wait complete. 2024-04-15T15:20:10.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.417+0000 7ff88bfc8640 1 Processor -- start 2024-04-15T15:20:10.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.417+0000 7ff88bfc8640 1 -- start start 2024-04-15T15:20:10.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.418+0000 7ff88bfc8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8840cd360 0x7ff88414b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:10.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.418+0000 7ff88bfc8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff8840d0f10 0x7ff8841f48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:10.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.418+0000 7ff88bfc8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff88415ecf0 0x7ff8840c96d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:10.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.418+0000 7ff88bfc8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff88415f0d0 con 0x7ff8840cd360 2024-04-15T15:20:10.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.418+0000 7ff88bfc8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff8840d1370 con 0x7ff88415ecf0 2024-04-15T15:20:10.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.418+0000 7ff88bfc8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7ff88415f250 con 0x7ff8840d0f10 2024-04-15T15:20:10.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.418+0000 7ff889d64640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8840cd360 0x7ff88414b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:10.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.418+0000 7ff889d64640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8840cd360 0x7ff88414b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37780/0 (socket says 172.21.15.105:37780) 2024-04-15T15:20:10.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.418+0000 7ff889d64640 1 -- 172.21.15.105:0/362735289 learned_addr learned my addr 172.21.15.105:0/362735289 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:10.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.418+0000 7ff889563640 1 --2- 172.21.15.105:0/362735289 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff8840d0f10 0x7ff8841f48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:10.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.418+0000 7ff88a565640 1 --2- 172.21.15.105:0/362735289 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff88415ecf0 0x7ff8840c96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:10.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.418+0000 7ff889d64640 1 -- 172.21.15.105:0/362735289 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff8840d0f10 msgr2=0x7ff8841f48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:10.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.418+0000 7ff889d64640 1 --2- 172.21.15.105:0/362735289 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff8840d0f10 0x7ff8841f48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:10.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.418+0000 7ff889d64640 1 -- 172.21.15.105:0/362735289 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff88415ecf0 msgr2=0x7ff8840c96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:10.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.418+0000 7ff889d64640 1 --2- 172.21.15.105:0/362735289 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff88415ecf0 0x7ff8840c96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:10.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.418+0000 7ff889d64640 1 -- 172.21.15.105:0/362735289 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8840b5f60 con 0x7ff8840cd360 2024-04-15T15:20:10.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.418+0000 7ff889563640 1 --2- 172.21.15.105:0/362735289 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff8840d0f10 0x7ff8841f48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:20:10.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.418+0000 7ff889d64640 1 --2- 172.21.15.105:0/362735289 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8840cd360 0x7ff88414b9c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff880009870 tx=0x7ff880004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:10.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.418+0000 7ff87a7fc640 1 -- 172.21.15.105:0/362735289 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff880048070 con 0x7ff8840cd360 2024-04-15T15:20:10.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.418+0000 7ff87a7fc640 1 -- 172.21.15.105:0/362735289 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff880039d90 con 0x7ff8840cd360 2024-04-15T15:20:10.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.418+0000 7ff88bfc8640 1 -- 172.21.15.105:0/362735289 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8840b6220 con 0x7ff8840cd360 2024-04-15T15:20:10.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.418+0000 7ff87a7fc640 1 -- 172.21.15.105:0/362735289 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff880034040 con 0x7ff8840cd360 2024-04-15T15:20:10.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.419+0000 7ff88bfc8640 1 -- 172.21.15.105:0/362735289 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8840b67e0 con 0x7ff8840cd360 2024-04-15T15:20:10.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.420+0000 7ff88bfc8640 1 -- 172.21.15.105:0/362735289 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff884068a40 con 0x7ff8840cd360 2024-04-15T15:20:10.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.420+0000 7ff87a7fc640 1 -- 172.21.15.105:0/362735289 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff88004b070 con 0x7ff8840cd360 2024-04-15T15:20:10.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.420+0000 7ff87a7fc640 1 --2- 172.21.15.105:0/362735289 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff86806c8f0 0x7ff86806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:10.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.421+0000 7ff87a7fc640 1 -- 172.21.15.105:0/362735289 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7ff8800ade00 con 0x7ff8840cd360 2024-04-15T15:20:10.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.421+0000 7ff889563640 1 --2- 172.21.15.105:0/362735289 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff86806c8f0 0x7ff86806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:10.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.421+0000 7ff889563640 1 --2- 172.21.15.105:0/362735289 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff86806c8f0 0x7ff86806edb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff8840b5e60 tx=0x7ff874009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:10.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.425+0000 7ff87a7fc640 1 -- 172.21.15.105:0/362735289 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff88007c1b0 con 0x7ff8840cd360 2024-04-15T15:20:10.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.604+0000 7ff88bfc8640 1 -- 172.21.15.105:0/362735289 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff8840b8580 con 0x7ff8840cd360 2024-04-15T15:20:10.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.605+0000 7ff87a7fc640 1 -- 172.21.15.105:0/362735289 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16901 (secure 0 0 0) 0x7ff88007bab0 con 0x7ff8840cd360 2024-04-15T15:20:10.607 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:20:10.607 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":22,"fsid":"09e4d2a4-a324-4793-af77-644b25e520e1","created":"2024-04-15T15:19:01.524543+0000","modified":"2024-04-15T15:20:07.555905+0000","last_up_change":"2024-04-15T15:19:12.825815+0000","last_in_change":"2024-04-15T15:19:03.788126+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:19:13.748585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:19:23.789287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:19:24.722250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6f6292b-2d7e-4aa0-9d1c-6ddd0f97287c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6801","nonce":2873040054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6803","nonce":2873040054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6807","nonce":2873040054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6805","nonce":2873040054}]},"public_addr":"172.21.15.105:6801/2873040054","cluster_addr":"172.21.15.105:6803/2873040054","heartbeat_back_addr":"172.21.15.105:6807/2873040054","heartbeat_front_addr":"172.21.15.105:6805/2873040054","state":["exists","up"]},{"osd":1,"uuid":"8900966a-b393-41e3-8f93-e5a43bd23622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6818","nonce":2494994479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6821","nonce":2494994479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6828","nonce":2494994479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6823","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6825","nonce":2494994479}]},"public_addr":"172.21.15.105:6818/2494994479","cluster_addr":"172.21.15.105:6821/2494994479","heartbeat_back_addr":"172.21.15.105:6828/2494994479","heartbeat_front_addr":"172.21.15.105:6825/2494994479","state":["exists","up"]},{"osd":2,"uuid":"81a5dc7a-9325-4ff5-941d-a2d5fba21f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6809","nonce":3798408699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6811","nonce":3798408699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6815","nonce":3798408699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6813","nonce":3798408699}]},"public_addr":"172.21.15.105:6809/3798408699","cluster_addr":"172.21.15.105:6811/3798408699","heartbeat_back_addr":"172.21.15.105:6815/3798408699","heartbeat_front_addr":"172.21.15.105:6813/3798408699","state":["exists","up"]},{"osd":3,"uuid":"df766ed2-5c71-4805-96f4-3d4970edc960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6819","nonce":1920720186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6824","nonce":1920720186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6831","nonce":1920720186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6827","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6829","nonce":1920720186}]},"public_addr":"172.21.15.105:6819/1920720186","cluster_addr":"172.21.15.105:6824/1920720186","heartbeat_back_addr":"172.21.15.105:6831/1920720186","heartbeat_front_addr":"172.21.15.105:6829/1920720186","state":["exists","up"]},{"osd":4,"uuid":"e1039e03-28b8-49df-b951-c91930b93c9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6810","nonce":3516494456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6813","nonce":3516494456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6820","nonce":3516494456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6817","nonce":3516494456}]},"public_addr":"172.21.15.154:6810/3516494456","cluster_addr":"172.21.15.154:6813/3516494456","heartbeat_back_addr":"172.21.15.154:6820/3516494456","heartbeat_front_addr":"172.21.15.154:6817/3516494456","state":["exists","up"]},{"osd":5,"uuid":"a9f5eaf1-66ff-405a-a9e4-ec3340812a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6801","nonce":540367751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6803","nonce":540367751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6807","nonce":540367751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6805","nonce":540367751}]},"public_addr":"172.21.15.154:6801/540367751","cluster_addr":"172.21.15.154:6803/540367751","heartbeat_back_addr":"172.21.15.154:6807/540367751","heartbeat_front_addr":"172.21.15.154:6805/540367751","state":["exists","up"]},{"osd":6,"uuid":"267d02ea-675f-428f-aae8-e0375e4d4feb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6825","nonce":1550829005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6827","nonce":1550829005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6831","nonce":1550829005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6829","nonce":1550829005}]},"public_addr":"172.21.15.154:6825/1550829005","cluster_addr":"172.21.15.154:6827/1550829005","heartbeat_back_addr":"172.21.15.154:6831/1550829005","heartbeat_front_addr":"172.21.15.154:6829/1550829005","state":["exists","up"]},{"osd":7,"uuid":"759ac6cc-896a-423c-885f-9fe90d8dcb14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6809","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6811","nonce":3920089903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6816","nonce":3920089903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6823","nonce":3920089903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6819","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6821","nonce":3920089903}]},"public_addr":"172.21.15.154:6811/3920089903","cluster_addr":"172.21.15.154:6816/3920089903","heartbeat_back_addr":"172.21.15.154:6823/3920089903","heartbeat_front_addr":"172.21.15.154:6821/3920089903","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.949509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.984072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.907278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.947391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.975447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.998896+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:09.028739+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.894119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":1}]},{"pgid":"3.30","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:20:10.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.607+0000 7ff88bfc8640 1 -- 172.21.15.105:0/362735289 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff86806c8f0 msgr2=0x7ff86806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:10.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.607+0000 7ff88bfc8640 1 --2- 172.21.15.105:0/362735289 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff86806c8f0 0x7ff86806edb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff8840b5e60 tx=0x7ff874009040 comp rx=0 tx=0).stop 2024-04-15T15:20:10.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.607+0000 7ff88bfc8640 1 -- 172.21.15.105:0/362735289 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8840cd360 msgr2=0x7ff88414b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:10.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.607+0000 7ff88bfc8640 1 --2- 172.21.15.105:0/362735289 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8840cd360 0x7ff88414b9c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff880009870 tx=0x7ff880004730 comp rx=0 tx=0).stop 2024-04-15T15:20:10.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.607+0000 7ff88bfc8640 1 -- 172.21.15.105:0/362735289 shutdown_connections 2024-04-15T15:20:10.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.607+0000 7ff88bfc8640 1 --2- 172.21.15.105:0/362735289 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff8840d0f10 0x7ff8841f48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:10.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.607+0000 7ff88bfc8640 1 --2- 172.21.15.105:0/362735289 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff86806c8f0 0x7ff86806edb0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:10.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.607+0000 7ff88bfc8640 1 --2- 172.21.15.105:0/362735289 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff88415ecf0 0x7ff8840c96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:10.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.607+0000 7ff88bfc8640 1 --2- 172.21.15.105:0/362735289 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8840cd360 0x7ff88414b9c0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:10.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.607+0000 7ff88bfc8640 1 -- 172.21.15.105:0/362735289 >> 172.21.15.105:0/362735289 conn(0x7ff8840d2600 msgr2=0x7ff8840cd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:10.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.607+0000 7ff88bfc8640 1 -- 172.21.15.105:0/362735289 shutdown_connections 2024-04-15T15:20:10.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.607+0000 7ff88bfc8640 1 -- 172.21.15.105:0/362735289 wait complete. 2024-04-15T15:20:10.623 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-15T15:20:10.623 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-15T15:20:10.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.724+0000 7fdc334a0640 1 Processor -- start 2024-04-15T15:20:10.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.724+0000 7fdc334a0640 1 -- start start 2024-04-15T15:20:10.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.725+0000 7fdc334a0640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fdc2c13ec60 0x7fdc2c06ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:10.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.725+0000 7fdc334a0640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdc2c06f4d0 0x7fdc2c15f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:10.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.725+0000 7fdc334a0640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc2c15bc90 con 0x7fdc2c13e680 2024-04-15T15:20:10.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.725+0000 7fdc334a0640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fdc2c0d0280 con 0x7fdc2c06f4d0 2024-04-15T15:20:10.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.725+0000 7fdc334a0640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fdc2c06f900 con 0x7fdc2c13ec60 2024-04-15T15:20:10.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.725+0000 7fdc31a3d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdc2c06f4d0 0x7fdc2c15f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:10.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.725+0000 7fdc31a3d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdc2c06f4d0 0x7fdc2c15f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39640/0 (socket says 172.21.15.105:39640) 2024-04-15T15:20:10.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.725+0000 7fdc31a3d640 1 -- 172.21.15.105:0/4278013582 learned_addr learned my addr 172.21.15.105:0/4278013582 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:10.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.726+0000 7fdc31a3d640 1 -- 172.21.15.105:0/4278013582 >> v2:172.21.15.154:3300/0 conn(0x7fdc2c13ec60 msgr2=0x7fdc2c06ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:10.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.726+0000 7fdc31a3d640 1 --2- 172.21.15.105:0/4278013582 >> v2:172.21.15.154:3300/0 conn(0x7fdc2c13ec60 0x7fdc2c06ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:10.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.726+0000 7fdc31a3d640 1 -- 172.21.15.105:0/4278013582 >> v1:172.21.15.154:6789/0 conn(0x7fdc2c13e680 legacy=0x7fdc2c0cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:10.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.726+0000 7fdc31a3d640 1 -- 172.21.15.105:0/4278013582 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc2c15bf70 con 0x7fdc2c06f4d0 2024-04-15T15:20:10.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.726+0000 7fdc31a3d640 1 --2- 172.21.15.105:0/4278013582 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdc2c06f4d0 0x7fdc2c15f1e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c073300 tx=0x7fdc2402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcd6082d9c11bcc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:10.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.726+0000 7fdc23fff640 1 -- 172.21.15.105:0/4278013582 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc24035020 con 0x7fdc2c06f4d0 2024-04-15T15:20:10.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.726+0000 7fdc23fff640 1 -- 172.21.15.105:0/4278013582 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc24031870 con 0x7fdc2c06f4d0 2024-04-15T15:20:10.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.726+0000 7fdc23fff640 1 -- 172.21.15.105:0/4278013582 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc24031c40 con 0x7fdc2c06f4d0 2024-04-15T15:20:10.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.727+0000 7fdc334a0640 1 -- 172.21.15.105:0/4278013582 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdc2c06f4d0 msgr2=0x7fdc2c15f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:10.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.727+0000 7fdc334a0640 1 --2- 172.21.15.105:0/4278013582 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdc2c06f4d0 0x7fdc2c15f1e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c073300 tx=0x7fdc2402b910 comp rx=0 tx=0).stop 2024-04-15T15:20:10.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.727+0000 7fdc334a0640 1 -- 172.21.15.105:0/4278013582 shutdown_connections 2024-04-15T15:20:10.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.727+0000 7fdc334a0640 1 --2- 172.21.15.105:0/4278013582 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdc2c06f4d0 0x7fdc2c15f1e0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:10.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.727+0000 7fdc334a0640 1 --2- 172.21.15.105:0/4278013582 >> v2:172.21.15.154:3300/0 conn(0x7fdc2c13ec60 0x7fdc2c06ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:10.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.727+0000 7fdc334a0640 1 -- 172.21.15.105:0/4278013582 >> 172.21.15.105:0/4278013582 conn(0x7fdc2c067d50 msgr2=0x7fdc2c06b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:10.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.727+0000 7fdc334a0640 1 -- 172.21.15.105:0/4278013582 shutdown_connections 2024-04-15T15:20:10.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.727+0000 7fdc334a0640 1 -- 172.21.15.105:0/4278013582 wait complete. 2024-04-15T15:20:10.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.728+0000 7fdc334a0640 1 Processor -- start 2024-04-15T15:20:10.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.728+0000 7fdc334a0640 1 -- start start 2024-04-15T15:20:10.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.729+0000 7fdc334a0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc2c06f4d0 0x7fdc2c152450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:10.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.729+0000 7fdc334a0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdc2c13e680 0x7fdc2c15a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:10.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.729+0000 7fdc334a0640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdc2c13ec60 0x7fdc2c15ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:10.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.729+0000 7fdc334a0640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2c15c810 con 0x7fdc2c13e680 2024-04-15T15:20:10.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.729+0000 7fdc334a0640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2c15c530 con 0x7fdc2c06f4d0 2024-04-15T15:20:10.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.729+0000 7fdc334a0640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fdc2c0cf590 con 0x7fdc2c13ec60 2024-04-15T15:20:10.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.729+0000 7fdc30a3b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdc2c13e680 0x7fdc2c15a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:10.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.729+0000 7fdc30a3b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdc2c13e680 0x7fdc2c15a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39656/0 (socket says 172.21.15.105:39656) 2024-04-15T15:20:10.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.729+0000 7fdc30a3b640 1 -- 172.21.15.105:0/1915884074 learned_addr learned my addr 172.21.15.105:0/1915884074 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:10.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.729+0000 7fdc30a3b640 1 -- 172.21.15.105:0/1915884074 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdc2c13ec60 msgr2=0x7fdc2c15ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:10.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.729+0000 7fdc31a3d640 1 --2- 172.21.15.105:0/1915884074 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdc2c13ec60 0x7fdc2c15ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:10.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.729+0000 7fdc3123c640 1 --2- 172.21.15.105:0/1915884074 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc2c06f4d0 0x7fdc2c152450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:10.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.729+0000 7fdc30a3b640 1 --2- 172.21.15.105:0/1915884074 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdc2c13ec60 0x7fdc2c15ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:10.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.729+0000 7fdc30a3b640 1 -- 172.21.15.105:0/1915884074 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc2c06f4d0 msgr2=0x7fdc2c152450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:10.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.729+0000 7fdc30a3b640 1 --2- 172.21.15.105:0/1915884074 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc2c06f4d0 0x7fdc2c152450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:10.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.730+0000 7fdc30a3b640 1 -- 172.21.15.105:0/1915884074 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc2c152990 con 0x7fdc2c13e680 2024-04-15T15:20:10.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.730+0000 7fdc3123c640 1 --2- 172.21.15.105:0/1915884074 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc2c06f4d0 0x7fdc2c152450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:10.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.730+0000 7fdc30a3b640 1 --2- 172.21.15.105:0/1915884074 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdc2c13e680 0x7fdc2c15a5c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c00b440 tx=0x7fdc1c00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:10.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.730+0000 7fdc21ffb640 1 -- 172.21.15.105:0/1915884074 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c015020 con 0x7fdc2c13e680 2024-04-15T15:20:10.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.730+0000 7fdc21ffb640 1 -- 172.21.15.105:0/1915884074 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc1c011890 con 0x7fdc2c13e680 2024-04-15T15:20:10.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.730+0000 7fdc21ffb640 1 -- 172.21.15.105:0/1915884074 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c011c80 con 0x7fdc2c13e680 2024-04-15T15:20:10.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.730+0000 7fdc334a0640 1 -- 172.21.15.105:0/1915884074 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc2c13f2c0 con 0x7fdc2c13e680 2024-04-15T15:20:10.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.730+0000 7fdc334a0640 1 -- 172.21.15.105:0/1915884074 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc2c13f880 con 0x7fdc2c13e680 2024-04-15T15:20:10.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.731+0000 7fdc334a0640 1 -- 172.21.15.105:0/1915884074 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc2c068a40 con 0x7fdc2c13e680 2024-04-15T15:20:10.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.731+0000 7fdc21ffb640 1 -- 172.21.15.105:0/1915884074 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdc1c018020 con 0x7fdc2c13e680 2024-04-15T15:20:10.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.732+0000 7fdc21ffb640 1 --2- 172.21.15.105:0/1915884074 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fdc1006c8f0 0x7fdc1006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:10.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.732+0000 7fdc21ffb640 1 -- 172.21.15.105:0/1915884074 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fdc1c08bc30 con 0x7fdc2c13e680 2024-04-15T15:20:10.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.732+0000 7fdc3123c640 1 --2- 172.21.15.105:0/1915884074 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fdc1006c8f0 0x7fdc1006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:10.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.732+0000 7fdc3123c640 1 --2- 172.21.15.105:0/1915884074 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fdc1006c8f0 0x7fdc1006edb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c135e30 tx=0x7fdc28009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:10.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.736+0000 7fdc21ffb640 1 -- 172.21.15.105:0/1915884074 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdc1c01f020 con 0x7fdc2c13e680 2024-04-15T15:20:10.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.949+0000 7fdc334a0640 1 -- 172.21.15.105:0/1915884074 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdc2c1559a0 con 0x7fdc2c13e680 2024-04-15T15:20:10.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.950+0000 7fdc21ffb640 1 -- 172.21.15.105:0/1915884074 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5187 (secure 0 0 0) 0x7fdc1c00d040 con 0x7fdc2c13e680 2024-04-15T15:20:10.952 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:20:10.952 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4259,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/3319657830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":3319657830},{"type":"v1","addr":"172.21.15.154:6835","nonce":3319657830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2677199890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2677199890},{"type":"v1","addr":"172.21.15.105:6835","nonce":2677199890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2867739743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2867739743},{"type":"v1","addr":"172.21.15.105:6837","nonce":2867739743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-15T15:19:25.760362+0000","modified":"2024-04-15T15:19:27.857599+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.154:6834/1529675504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1529675504},{"type":"v1","addr":"172.21.15.154:6834","nonce":1529675504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4291,"qdb_cluster":[4291]},"id":1}]} 2024-04-15T15:20:10.952 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 6 2024-04-15T15:20:10.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.952+0000 7fdc334a0640 1 -- 172.21.15.105:0/1915884074 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fdc1006c8f0 msgr2=0x7fdc1006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:10.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.952+0000 7fdc334a0640 1 --2- 172.21.15.105:0/1915884074 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fdc1006c8f0 0x7fdc1006edb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c135e30 tx=0x7fdc28009040 comp rx=0 tx=0).stop 2024-04-15T15:20:10.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.952+0000 7fdc334a0640 1 -- 172.21.15.105:0/1915884074 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdc2c13e680 msgr2=0x7fdc2c15a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:10.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.952+0000 7fdc334a0640 1 --2- 172.21.15.105:0/1915884074 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdc2c13e680 0x7fdc2c15a5c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c00b440 tx=0x7fdc1c00b900 comp rx=0 tx=0).stop 2024-04-15T15:20:10.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.952+0000 7fdc334a0640 1 -- 172.21.15.105:0/1915884074 shutdown_connections 2024-04-15T15:20:10.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.952+0000 7fdc334a0640 1 --2- 172.21.15.105:0/1915884074 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdc2c13ec60 0x7fdc2c15ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:10.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.952+0000 7fdc334a0640 1 --2- 172.21.15.105:0/1915884074 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fdc1006c8f0 0x7fdc1006edb0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:10.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.952+0000 7fdc334a0640 1 --2- 172.21.15.105:0/1915884074 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdc2c13e680 0x7fdc2c15a5c0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:10.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.952+0000 7fdc334a0640 1 --2- 172.21.15.105:0/1915884074 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc2c06f4d0 0x7fdc2c152450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:10.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.952+0000 7fdc334a0640 1 -- 172.21.15.105:0/1915884074 >> 172.21.15.105:0/1915884074 conn(0x7fdc2c067d50 msgr2=0x7fdc2c0b39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:10.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.952+0000 7fdc334a0640 1 -- 172.21.15.105:0/1915884074 shutdown_connections 2024-04-15T15:20:10.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:10.952+0000 7fdc334a0640 1 -- 172.21.15.105:0/1915884074 wait complete. 2024-04-15T15:20:10.965 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-15T15:20:11.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.066+0000 7f5f1d077640 1 Processor -- start 2024-04-15T15:20:11.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.066+0000 7f5f1d077640 1 -- start start 2024-04-15T15:20:11.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.067+0000 7f5f1d077640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5f1806f6a0 0x7f5f18166470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:11.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.067+0000 7f5f1d077640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f1815bc90 con 0x7f5f1813e680 2024-04-15T15:20:11.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.067+0000 7f5f1d077640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f1815bf70 con 0x7f5f1813ec60 2024-04-15T15:20:11.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.067+0000 7f5f1d077640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f5f180d0280 con 0x7f5f1806f6a0 2024-04-15T15:20:11.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.067+0000 7f5f16d9d640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f5f1813e680 0x7f5f180cfb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:42848/0 (socket says 172.21.15.105:42848) 2024-04-15T15:20:11.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.067+0000 7f5f16d9d640 1 -- 172.21.15.105:0/3911326659 learned_addr learned my addr 172.21.15.105:0/3911326659 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:20:11.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.067+0000 7f5f1759e640 1 --2- 172.21.15.105:0/3911326659 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5f1806f6a0 0x7f5f18166470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:11.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.067+0000 7f5f1659c640 1 -- 172.21.15.105:0/3911326659 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1166252738 0 0) 0x7f5f1815bc90 con 0x7f5f1813e680 2024-04-15T15:20:11.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.067+0000 7f5f1659c640 1 -- 172.21.15.105:0/3911326659 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5f04003520 con 0x7f5f1813e680 2024-04-15T15:20:11.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.067+0000 7f5f1659c640 1 -- 172.21.15.105:0/3911326659 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1179694322 0 0) 0x7f5f04003520 con 0x7f5f1813e680 2024-04-15T15:20:11.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.067+0000 7f5f1659c640 1 -- 172.21.15.105:0/3911326659 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5f1806f6a0 msgr2=0x7f5f18166470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:11.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.067+0000 7f5f1659c640 1 --2- 172.21.15.105:0/3911326659 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5f1806f6a0 0x7f5f18166470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:11.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.067+0000 7f5f1659c640 1 -- 172.21.15.105:0/3911326659 >> v1:172.21.15.154:6789/0 conn(0x7f5f1813ec60 legacy=0x7f5f1806ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:11.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.067+0000 7f5f1659c640 1 -- 172.21.15.105:0/3911326659 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f1815c250 con 0x7f5f1813e680 2024-04-15T15:20:11.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.067+0000 7f5f1659c640 1 -- 172.21.15.105:0/3911326659 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7f5f08001670 con 0x7f5f1813e680 2024-04-15T15:20:11.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.067+0000 7f5f1659c640 1 -- 172.21.15.105:0/3911326659 >> v1:172.21.15.105:6789/0 conn(0x7f5f1813e680 legacy=0x7f5f180cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:11.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.067+0000 7f5f1659c640 1 --2- 172.21.15.105:0/3911326659 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5f1806f6a0 0x7f5f040039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:11.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.067+0000 7f5f1659c640 1 --2- 172.21.15.105:0/3911326659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f04003f20 0x7f5f04004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:11.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.067+0000 7f5f1659c640 1 --2- 172.21.15.105:0/3911326659 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f04004d40 0x7f5f040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:11.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.067+0000 7f5f1659c640 1 -- 172.21.15.105:0/3911326659 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5f180d0280 con 0x7f5f04004d40 2024-04-15T15:20:11.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.067+0000 7f5f1659c640 1 -- 172.21.15.105:0/3911326659 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5f0400ba20 con 0x7f5f04003f20 2024-04-15T15:20:11.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.067+0000 7f5f1659c640 1 -- 172.21.15.105:0/3911326659 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f5f0400bc20 con 0x7f5f1806f6a0 2024-04-15T15:20:11.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.067+0000 7f5f1759e640 1 --2- 172.21.15.105:0/3911326659 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f04004d40 0x7f5f040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:11.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.068+0000 7f5f16d9d640 1 --2- 172.21.15.105:0/3911326659 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5f1806f6a0 0x7f5f040039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:11.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.068+0000 7f5f0ffff640 1 --2- 172.21.15.105:0/3911326659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f04003f20 0x7f5f04004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:11.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.068+0000 7f5f1759e640 1 -- 172.21.15.105:0/3911326659 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5f1806f6a0 msgr2=0x7f5f040039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:11.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.068+0000 7f5f1759e640 1 --2- 172.21.15.105:0/3911326659 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5f1806f6a0 0x7f5f040039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:11.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.068+0000 7f5f1759e640 1 -- 172.21.15.105:0/3911326659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f04003f20 msgr2=0x7f5f04004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:11.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.068+0000 7f5f1759e640 1 --2- 172.21.15.105:0/3911326659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f04003f20 0x7f5f04004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:11.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.068+0000 7f5f1759e640 1 -- 172.21.15.105:0/3911326659 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f04003520 con 0x7f5f04004d40 2024-04-15T15:20:11.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.068+0000 7f5f0ffff640 1 --2- 172.21.15.105:0/3911326659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f04003f20 0x7f5f04004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:20:11.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.068+0000 7f5f1759e640 1 --2- 172.21.15.105:0/3911326659 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f04004d40 0x7f5f040051c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f5f18071cd0 tx=0x7f5f04005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=445963f900009daf server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:11.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.069+0000 7f5f1659c640 1 -- 172.21.15.105:0/3911326659 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f10039070 con 0x7f5f04004d40 2024-04-15T15:20:11.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.069+0000 7f5f1659c640 1 -- 172.21.15.105:0/3911326659 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f1002cb90 con 0x7f5f04004d40 2024-04-15T15:20:11.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.069+0000 7f5f1659c640 1 -- 172.21.15.105:0/3911326659 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f10009c10 con 0x7f5f04004d40 2024-04-15T15:20:11.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.069+0000 7f5f1d077640 1 -- 172.21.15.105:0/3911326659 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f04004d40 msgr2=0x7f5f040051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:11.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.069+0000 7f5f1d077640 1 --2- 172.21.15.105:0/3911326659 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f04004d40 0x7f5f040051c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f5f18071cd0 tx=0x7f5f04005830 comp rx=0 tx=0).stop 2024-04-15T15:20:11.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.069+0000 7f5f1759e640 1 -- 172.21.15.105:0/3911326659 reap_dead start 2024-04-15T15:20:11.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.069+0000 7f5f1d077640 1 -- 172.21.15.105:0/3911326659 shutdown_connections 2024-04-15T15:20:11.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.069+0000 7f5f1d077640 1 -- 172.21.15.105:0/3911326659 >> 172.21.15.105:0/3911326659 conn(0x7f5f18067d50 msgr2=0x7f5f1806b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:11.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.069+0000 7f5f1d077640 1 -- 172.21.15.105:0/3911326659 shutdown_connections 2024-04-15T15:20:11.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.069+0000 7f5f1d077640 1 -- 172.21.15.105:0/3911326659 wait complete. 2024-04-15T15:20:11.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.070+0000 7f5f1d077640 1 Processor -- start 2024-04-15T15:20:11.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.070+0000 7f5f1d077640 1 -- start start 2024-04-15T15:20:11.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.071+0000 7f5f1d077640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f04004d40 0x7f5f18155430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:11.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.071+0000 7f5f1d077640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5f1806e330 0x7f5f18136620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:11.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.071+0000 7f5f1d077640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f18137f10 0x7f5f1813c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:11.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.071+0000 7f5f1d077640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5f180cf590 con 0x7f5f18137f10 2024-04-15T15:20:11.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.071+0000 7f5f1d077640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5f1815c810 con 0x7f5f04004d40 2024-04-15T15:20:11.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.071+0000 7f5f1d077640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f5f180cf710 con 0x7f5f1806e330 2024-04-15T15:20:11.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.071+0000 7f5f1759e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f18137f10 0x7f5f1813c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:11.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.071+0000 7f5f1759e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f18137f10 0x7f5f1813c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39680/0 (socket says 172.21.15.105:39680) 2024-04-15T15:20:11.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.071+0000 7f5f1759e640 1 -- 172.21.15.105:0/2875212778 learned_addr learned my addr 172.21.15.105:0/2875212778 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:11.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.072+0000 7f5f1759e640 1 -- 172.21.15.105:0/2875212778 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5f1806e330 msgr2=0x7f5f18136620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:11.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.072+0000 7f5f1759e640 1 --2- 172.21.15.105:0/2875212778 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5f1806e330 0x7f5f18136620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:11.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.072+0000 7f5f1759e640 1 -- 172.21.15.105:0/2875212778 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f04004d40 msgr2=0x7f5f18155430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:11.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.072+0000 7f5f16d9d640 1 --2- 172.21.15.105:0/2875212778 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f04004d40 0x7f5f18155430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:11.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.072+0000 7f5f1759e640 1 --2- 172.21.15.105:0/2875212778 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f04004d40 0x7f5f18155430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:11.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.072+0000 7f5f1759e640 1 -- 172.21.15.105:0/2875212778 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f18136e70 con 0x7f5f18137f10 2024-04-15T15:20:11.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.072+0000 7f5f1759e640 1 --2- 172.21.15.105:0/2875212778 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f18137f10 0x7f5f1813c690 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5f10037040 tx=0x7f5f10009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:11.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.072+0000 7f5f0f7fe640 1 -- 172.21.15.105:0/2875212778 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f10039030 con 0x7f5f18137f10 2024-04-15T15:20:11.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.072+0000 7f5f0f7fe640 1 -- 172.21.15.105:0/2875212778 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f1003a040 con 0x7f5f18137f10 2024-04-15T15:20:11.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.072+0000 7f5f0f7fe640 1 -- 172.21.15.105:0/2875212778 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f10003990 con 0x7f5f18137f10 2024-04-15T15:20:11.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.072+0000 7f5f1d077640 1 -- 172.21.15.105:0/2875212778 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f18137150 con 0x7f5f18137f10 2024-04-15T15:20:11.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.072+0000 7f5f1d077640 1 -- 172.21.15.105:0/2875212778 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f18137710 con 0x7f5f18137f10 2024-04-15T15:20:11.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.073+0000 7f5f1d077640 1 -- 172.21.15.105:0/2875212778 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f18068a40 con 0x7f5f18137f10 2024-04-15T15:20:11.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.073+0000 7f5f0f7fe640 1 -- 172.21.15.105:0/2875212778 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5f10045030 con 0x7f5f18137f10 2024-04-15T15:20:11.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.074+0000 7f5f0f7fe640 1 --2- 172.21.15.105:0/2875212778 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f5ef806c920 0x7f5ef806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:11.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.074+0000 7f5f0f7fe640 1 -- 172.21.15.105:0/2875212778 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f5f100adb00 con 0x7f5f18137f10 2024-04-15T15:20:11.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.074+0000 7f5f16d9d640 1 --2- 172.21.15.105:0/2875212778 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f5ef806c920 0x7f5ef806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:11.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.074+0000 7f5f16d9d640 1 --2- 172.21.15.105:0/2875212778 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f5ef806c920 0x7f5ef806ede0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5f08000c00 tx=0x7f5f08009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:11.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.078+0000 7f5f0f7fe640 1 -- 172.21.15.105:0/2875212778 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5f1004c020 con 0x7f5f18137f10 2024-04-15T15:20:11.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.290+0000 7f5f1d077640 1 -- 172.21.15.105:0/2875212778 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5f180b5880 con 0x7f5f18137f10 2024-04-15T15:20:11.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.291+0000 7f5f0f7fe640 1 -- 172.21.15.105:0/2875212778 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5187 (secure 0 0 0) 0x7f5f1007bf40 con 0x7f5f18137f10 2024-04-15T15:20:11.293 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:20:11.293 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4259,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/3319657830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":3319657830},{"type":"v1","addr":"172.21.15.154:6835","nonce":3319657830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2677199890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2677199890},{"type":"v1","addr":"172.21.15.105:6835","nonce":2677199890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2867739743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2867739743},{"type":"v1","addr":"172.21.15.105:6837","nonce":2867739743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-15T15:19:25.760362+0000","modified":"2024-04-15T15:19:27.857599+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.154:6834/1529675504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1529675504},{"type":"v1","addr":"172.21.15.154:6834","nonce":1529675504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4291,"qdb_cluster":[4291]},"id":1}]} 2024-04-15T15:20:11.293 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 6 2024-04-15T15:20:11.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.293+0000 7f5f1d077640 1 -- 172.21.15.105:0/2875212778 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f5ef806c920 msgr2=0x7f5ef806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:11.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.293+0000 7f5f1d077640 1 --2- 172.21.15.105:0/2875212778 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f5ef806c920 0x7f5ef806ede0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5f08000c00 tx=0x7f5f08009040 comp rx=0 tx=0).stop 2024-04-15T15:20:11.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.293+0000 7f5f1d077640 1 -- 172.21.15.105:0/2875212778 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f18137f10 msgr2=0x7f5f1813c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:11.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.293+0000 7f5f1d077640 1 --2- 172.21.15.105:0/2875212778 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f18137f10 0x7f5f1813c690 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5f10037040 tx=0x7f5f10009750 comp rx=0 tx=0).stop 2024-04-15T15:20:11.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.293+0000 7f5f1d077640 1 -- 172.21.15.105:0/2875212778 shutdown_connections 2024-04-15T15:20:11.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.293+0000 7f5f1d077640 1 --2- 172.21.15.105:0/2875212778 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5f1806e330 0x7f5f18136620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:11.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.293+0000 7f5f1d077640 1 --2- 172.21.15.105:0/2875212778 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f5ef806c920 0x7f5ef806ede0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:11.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.293+0000 7f5f1d077640 1 --2- 172.21.15.105:0/2875212778 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f18137f10 0x7f5f1813c690 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:11.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.293+0000 7f5f1d077640 1 --2- 172.21.15.105:0/2875212778 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f04004d40 0x7f5f18155430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:11.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.293+0000 7f5f1d077640 1 -- 172.21.15.105:0/2875212778 >> 172.21.15.105:0/2875212778 conn(0x7f5f18067d50 msgr2=0x7f5f180cfb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:11.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.293+0000 7f5f1d077640 1 -- 172.21.15.105:0/2875212778 shutdown_connections 2024-04-15T15:20:11.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.293+0000 7f5f1d077640 1 -- 172.21.15.105:0/2875212778 wait complete. 2024-04-15T15:20:11.306 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:20:11.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.407+0000 7f7e63dbd640 1 Processor -- start 2024-04-15T15:20:11.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.407+0000 7f7e63dbd640 1 -- start start 2024-04-15T15:20:11.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.408+0000 7f7e63dbd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7e5c1554d0 0x7f7e5c0afaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:11.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.408+0000 7f7e63dbd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7e5c0b5580 0x7f7e5c0cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:11.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.408+0000 7f7e63dbd640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e5c0b5a00 con 0x7f7e5c0cfc50 2024-04-15T15:20:11.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.408+0000 7f7e63dbd640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f7e5c0cf280 con 0x7f7e5c1554d0 2024-04-15T15:20:11.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.408+0000 7f7e63dbd640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f7e5c15bf70 con 0x7f7e5c0b5580 2024-04-15T15:20:11.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.408+0000 7f7e61b59640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7e5c1554d0 0x7f7e5c0afaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:11.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.408+0000 7f7e6235a640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f7e5c0cfc50 0x7f7e5c0726f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:42858/0 (socket says 172.21.15.105:42858) 2024-04-15T15:20:11.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.408+0000 7f7e61b59640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7e5c1554d0 0x7f7e5c0afaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39694/0 (socket says 172.21.15.105:39694) 2024-04-15T15:20:11.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.408+0000 7f7e6235a640 1 -- 172.21.15.105:0/3225387738 learned_addr learned my addr 172.21.15.105:0/3225387738 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:20:11.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.408+0000 7f7e61358640 1 --2- 172.21.15.105:0/3225387738 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7e5c0b5580 0x7f7e5c0cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:11.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.408+0000 7f7e61b59640 1 -- 172.21.15.105:0/3225387738 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7e5c0b5580 msgr2=0x7f7e5c0cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:11.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.408+0000 7f7e61b59640 1 --2- 172.21.15.105:0/3225387738 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7e5c0b5580 0x7f7e5c0cf710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:11.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.408+0000 7f7e61b59640 1 -- 172.21.15.105:0/3225387738 >> v1:172.21.15.105:6789/0 conn(0x7f7e5c0cfc50 legacy=0x7f7e5c0726f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:11.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.408+0000 7f7e61b59640 1 -- 172.21.15.105:0/3225387738 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e5c0cec30 con 0x7f7e5c1554d0 2024-04-15T15:20:11.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.408+0000 7f7e60b57640 1 -- 172.21.15.105:0/3225387738 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3735758541 0 0) 0x7f7e5c0b5a00 con 0x7f7e5c0cfc50 2024-04-15T15:20:11.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.409+0000 7f7e61b59640 1 --2- 172.21.15.105:0/3225387738 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e5c1554d0 0x7f7e5c0afaa0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c00bfa0 tx=0x7f7e4c02e0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23a61ec84fc5e5ad server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:11.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.409+0000 7f7e60b57640 1 -- 172.21.15.105:0/3225387738 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c034070 con 0x7f7e5c1554d0 2024-04-15T15:20:11.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.409+0000 7f7e60b57640 1 -- 172.21.15.105:0/3225387738 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e4c02ee50 con 0x7f7e5c1554d0 2024-04-15T15:20:11.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.409+0000 7f7e60b57640 1 -- 172.21.15.105:0/3225387738 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c038620 con 0x7f7e5c1554d0 2024-04-15T15:20:11.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.410+0000 7f7e63dbd640 1 -- 172.21.15.105:0/3225387738 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e5c1554d0 msgr2=0x7f7e5c0afaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:11.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.410+0000 7f7e63dbd640 1 --2- 172.21.15.105:0/3225387738 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e5c1554d0 0x7f7e5c0afaa0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c00bfa0 tx=0x7f7e4c02e0a0 comp rx=0 tx=0).stop 2024-04-15T15:20:11.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.410+0000 7f7e63dbd640 1 -- 172.21.15.105:0/3225387738 shutdown_connections 2024-04-15T15:20:11.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.410+0000 7f7e63dbd640 1 --2- 172.21.15.105:0/3225387738 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7e5c0b5580 0x7f7e5c0cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:11.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.410+0000 7f7e63dbd640 1 --2- 172.21.15.105:0/3225387738 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e5c1554d0 0x7f7e5c0afaa0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:11.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.410+0000 7f7e63dbd640 1 -- 172.21.15.105:0/3225387738 >> 172.21.15.105:0/3225387738 conn(0x7f7e5c067d50 msgr2=0x7f7e5c06b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:11.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.410+0000 7f7e63dbd640 1 -- 172.21.15.105:0/3225387738 shutdown_connections 2024-04-15T15:20:11.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.410+0000 7f7e63dbd640 1 -- 172.21.15.105:0/3225387738 wait complete. 2024-04-15T15:20:11.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.411+0000 7f7e63dbd640 1 Processor -- start 2024-04-15T15:20:11.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.411+0000 7f7e63dbd640 1 -- start start 2024-04-15T15:20:11.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.412+0000 7f7e63dbd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7e5c0b5580 0x7f7e5c154270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:11.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.412+0000 7f7e63dbd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e5c0cfc50 0x7f7e5c155920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:11.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.412+0000 7f7e63dbd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e5c1554d0 0x7f7e5c14af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:11.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.412+0000 7f7e63dbd640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5c072e00 con 0x7f7e5c0cfc50 2024-04-15T15:20:11.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.412+0000 7f7e63dbd640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5c15c250 con 0x7f7e5c1554d0 2024-04-15T15:20:11.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.412+0000 7f7e63dbd640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f7e5c072f80 con 0x7f7e5c0b5580 2024-04-15T15:20:11.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.412+0000 7f7e61358640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e5c0cfc50 0x7f7e5c155920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:11.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.412+0000 7f7e61358640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e5c0cfc50 0x7f7e5c155920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39708/0 (socket says 172.21.15.105:39708) 2024-04-15T15:20:11.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.412+0000 7f7e61358640 1 -- 172.21.15.105:0/3144321453 learned_addr learned my addr 172.21.15.105:0/3144321453 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:11.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.412+0000 7f7e61b59640 1 --2- 172.21.15.105:0/3144321453 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7e5c0b5580 0x7f7e5c154270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:11.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.412+0000 7f7e6235a640 1 --2- 172.21.15.105:0/3144321453 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e5c1554d0 0x7f7e5c14af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:11.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.412+0000 7f7e61358640 1 -- 172.21.15.105:0/3144321453 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7e5c0b5580 msgr2=0x7f7e5c154270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:11.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.412+0000 7f7e61358640 1 --2- 172.21.15.105:0/3144321453 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7e5c0b5580 0x7f7e5c154270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:11.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.412+0000 7f7e61358640 1 -- 172.21.15.105:0/3144321453 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e5c1554d0 msgr2=0x7f7e5c14af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:11.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.412+0000 7f7e61358640 1 --2- 172.21.15.105:0/3144321453 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e5c1554d0 0x7f7e5c14af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:11.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.412+0000 7f7e61358640 1 -- 172.21.15.105:0/3144321453 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e5c155e60 con 0x7f7e5c0cfc50 2024-04-15T15:20:11.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.413+0000 7f7e61358640 1 --2- 172.21.15.105:0/3144321453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e5c0cfc50 0x7f7e5c155920 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7e5000c820 tx=0x7f7e5000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:11.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.413+0000 7f7e5a7fc640 1 -- 172.21.15.105:0/3144321453 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e50015070 con 0x7f7e5c0cfc50 2024-04-15T15:20:11.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.413+0000 7f7e5a7fc640 1 -- 172.21.15.105:0/3144321453 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e50004240 con 0x7f7e5c0cfc50 2024-04-15T15:20:11.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.413+0000 7f7e63dbd640 1 -- 172.21.15.105:0/3144321453 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e5c153060 con 0x7f7e5c0cfc50 2024-04-15T15:20:11.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.413+0000 7f7e5a7fc640 1 -- 172.21.15.105:0/3144321453 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e5000ce10 con 0x7f7e5c0cfc50 2024-04-15T15:20:11.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.413+0000 7f7e63dbd640 1 -- 172.21.15.105:0/3144321453 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e5c156540 con 0x7f7e5c0cfc50 2024-04-15T15:20:11.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.414+0000 7f7e63dbd640 1 -- 172.21.15.105:0/3144321453 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e5c068a40 con 0x7f7e5c0cfc50 2024-04-15T15:20:11.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.414+0000 7f7e5a7fc640 1 -- 172.21.15.105:0/3144321453 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7e50007830 con 0x7f7e5c0cfc50 2024-04-15T15:20:11.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.415+0000 7f7e5a7fc640 1 --2- 172.21.15.105:0/3144321453 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f7e4806c8f0 0x7f7e4806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:11.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.415+0000 7f7e5a7fc640 1 -- 172.21.15.105:0/3144321453 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f7e5008ba90 con 0x7f7e5c0cfc50 2024-04-15T15:20:11.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.415+0000 7f7e61b59640 1 --2- 172.21.15.105:0/3144321453 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f7e4806c8f0 0x7f7e4806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:11.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.415+0000 7f7e61b59640 1 --2- 172.21.15.105:0/3144321453 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f7e4806c8f0 0x7f7e4806edb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c039000 tx=0x7f7e4c034000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:11.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.419+0000 7f7e5a7fc640 1 -- 172.21.15.105:0/3144321453 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7e50059760 con 0x7f7e5c0cfc50 2024-04-15T15:20:11.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.596+0000 7f7e63dbd640 1 -- 172.21.15.105:0/3144321453 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7e5c06f690 con 0x7f7e5c0cfc50 2024-04-15T15:20:11.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.597+0000 7f7e5a7fc640 1 -- 172.21.15.105:0/3144321453 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16901 (secure 0 0 0) 0x7f7e5005da60 con 0x7f7e5c0cfc50 2024-04-15T15:20:11.599 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:20:11.599 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":22,"fsid":"09e4d2a4-a324-4793-af77-644b25e520e1","created":"2024-04-15T15:19:01.524543+0000","modified":"2024-04-15T15:20:07.555905+0000","last_up_change":"2024-04-15T15:19:12.825815+0000","last_in_change":"2024-04-15T15:19:03.788126+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:19:13.748585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:19:23.789287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:19:24.722250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6f6292b-2d7e-4aa0-9d1c-6ddd0f97287c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6801","nonce":2873040054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6803","nonce":2873040054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6807","nonce":2873040054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6805","nonce":2873040054}]},"public_addr":"172.21.15.105:6801/2873040054","cluster_addr":"172.21.15.105:6803/2873040054","heartbeat_back_addr":"172.21.15.105:6807/2873040054","heartbeat_front_addr":"172.21.15.105:6805/2873040054","state":["exists","up"]},{"osd":1,"uuid":"8900966a-b393-41e3-8f93-e5a43bd23622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6818","nonce":2494994479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6821","nonce":2494994479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6828","nonce":2494994479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6823","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6825","nonce":2494994479}]},"public_addr":"172.21.15.105:6818/2494994479","cluster_addr":"172.21.15.105:6821/2494994479","heartbeat_back_addr":"172.21.15.105:6828/2494994479","heartbeat_front_addr":"172.21.15.105:6825/2494994479","state":["exists","up"]},{"osd":2,"uuid":"81a5dc7a-9325-4ff5-941d-a2d5fba21f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6809","nonce":3798408699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6811","nonce":3798408699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6815","nonce":3798408699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6813","nonce":3798408699}]},"public_addr":"172.21.15.105:6809/3798408699","cluster_addr":"172.21.15.105:6811/3798408699","heartbeat_back_addr":"172.21.15.105:6815/3798408699","heartbeat_front_addr":"172.21.15.105:6813/3798408699","state":["exists","up"]},{"osd":3,"uuid":"df766ed2-5c71-4805-96f4-3d4970edc960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6819","nonce":1920720186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6824","nonce":1920720186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6831","nonce":1920720186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6827","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6829","nonce":1920720186}]},"public_addr":"172.21.15.105:6819/1920720186","cluster_addr":"172.21.15.105:6824/1920720186","heartbeat_back_addr":"172.21.15.105:6831/1920720186","heartbeat_front_addr":"172.21.15.105:6829/1920720186","state":["exists","up"]},{"osd":4,"uuid":"e1039e03-28b8-49df-b951-c91930b93c9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6810","nonce":3516494456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6813","nonce":3516494456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6820","nonce":3516494456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6817","nonce":3516494456}]},"public_addr":"172.21.15.154:6810/3516494456","cluster_addr":"172.21.15.154:6813/3516494456","heartbeat_back_addr":"172.21.15.154:6820/3516494456","heartbeat_front_addr":"172.21.15.154:6817/3516494456","state":["exists","up"]},{"osd":5,"uuid":"a9f5eaf1-66ff-405a-a9e4-ec3340812a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6801","nonce":540367751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6803","nonce":540367751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6807","nonce":540367751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6805","nonce":540367751}]},"public_addr":"172.21.15.154:6801/540367751","cluster_addr":"172.21.15.154:6803/540367751","heartbeat_back_addr":"172.21.15.154:6807/540367751","heartbeat_front_addr":"172.21.15.154:6805/540367751","state":["exists","up"]},{"osd":6,"uuid":"267d02ea-675f-428f-aae8-e0375e4d4feb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6825","nonce":1550829005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6827","nonce":1550829005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6831","nonce":1550829005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6829","nonce":1550829005}]},"public_addr":"172.21.15.154:6825/1550829005","cluster_addr":"172.21.15.154:6827/1550829005","heartbeat_back_addr":"172.21.15.154:6831/1550829005","heartbeat_front_addr":"172.21.15.154:6829/1550829005","state":["exists","up"]},{"osd":7,"uuid":"759ac6cc-896a-423c-885f-9fe90d8dcb14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6809","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6811","nonce":3920089903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6816","nonce":3920089903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6823","nonce":3920089903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6819","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6821","nonce":3920089903}]},"public_addr":"172.21.15.154:6811/3920089903","cluster_addr":"172.21.15.154:6816/3920089903","heartbeat_back_addr":"172.21.15.154:6823/3920089903","heartbeat_front_addr":"172.21.15.154:6821/3920089903","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.949509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.984072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.907278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.947391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.975447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.998896+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:09.028739+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.894119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":1}]},{"pgid":"3.30","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:20:11.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.599+0000 7f7e63dbd640 1 -- 172.21.15.105:0/3144321453 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f7e4806c8f0 msgr2=0x7f7e4806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:11.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.599+0000 7f7e63dbd640 1 --2- 172.21.15.105:0/3144321453 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f7e4806c8f0 0x7f7e4806edb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c039000 tx=0x7f7e4c034000 comp rx=0 tx=0).stop 2024-04-15T15:20:11.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.599+0000 7f7e63dbd640 1 -- 172.21.15.105:0/3144321453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e5c0cfc50 msgr2=0x7f7e5c155920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:11.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.599+0000 7f7e63dbd640 1 --2- 172.21.15.105:0/3144321453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e5c0cfc50 0x7f7e5c155920 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7e5000c820 tx=0x7f7e5000cce0 comp rx=0 tx=0).stop 2024-04-15T15:20:11.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.599+0000 7f7e63dbd640 1 -- 172.21.15.105:0/3144321453 shutdown_connections 2024-04-15T15:20:11.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.599+0000 7f7e63dbd640 1 --2- 172.21.15.105:0/3144321453 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f7e4806c8f0 0x7f7e4806edb0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:11.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.599+0000 7f7e63dbd640 1 --2- 172.21.15.105:0/3144321453 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e5c1554d0 0x7f7e5c14af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:11.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.599+0000 7f7e63dbd640 1 --2- 172.21.15.105:0/3144321453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e5c0cfc50 0x7f7e5c155920 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:11.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.599+0000 7f7e63dbd640 1 --2- 172.21.15.105:0/3144321453 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7e5c0b5580 0x7f7e5c154270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:11.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.599+0000 7f7e63dbd640 1 -- 172.21.15.105:0/3144321453 >> 172.21.15.105:0/3144321453 conn(0x7f7e5c067d50 msgr2=0x7f7e5c06b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:11.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.599+0000 7f7e63dbd640 1 -- 172.21.15.105:0/3144321453 shutdown_connections 2024-04-15T15:20:11.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.599+0000 7f7e63dbd640 1 -- 172.21.15.105:0/3144321453 wait complete. 2024-04-15T15:20:11.612 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-15T15:20:11.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.715+0000 7f025b89a640 1 Processor -- start 2024-04-15T15:20:11.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.715+0000 7f025b89a640 1 -- start start 2024-04-15T15:20:11.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.716+0000 7f025b89a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f02540d0540 0x7f025406b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:11.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.716+0000 7f025b89a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02540c8270 con 0x7f02540b3c80 2024-04-15T15:20:11.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.716+0000 7f025b89a640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02540d0920 con 0x7f02540ce800 2024-04-15T15:20:11.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.716+0000 7f025b89a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f02540c7cb0 con 0x7f02540d0540 2024-04-15T15:20:11.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.716+0000 7f0259636640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f02540d0540 0x7f025406b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:11.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.716+0000 7f0258e35640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f02540b3c80 0x7f02540c6310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:42870/0 (socket says 172.21.15.105:42870) 2024-04-15T15:20:11.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.716+0000 7f0258e35640 1 -- 172.21.15.105:0/640208946 learned_addr learned my addr 172.21.15.105:0/640208946 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:20:11.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.716+0000 7f0259636640 1 -- 172.21.15.105:0/640208946 >> v1:172.21.15.154:6789/0 conn(0x7f02540ce800 legacy=0x7f02540c6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:11.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.716+0000 7f0259636640 1 -- 172.21.15.105:0/640208946 >> v1:172.21.15.105:6789/0 conn(0x7f02540b3c80 legacy=0x7f02540c6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:11.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.716+0000 7f024bfff640 1 -- 172.21.15.105:0/640208946 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4109783317 0 0) 0x7f02540c8270 con 0x7f02540b3c80 2024-04-15T15:20:11.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.716+0000 7f0259636640 1 -- 172.21.15.105:0/640208946 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02540c7f90 con 0x7f02540d0540 2024-04-15T15:20:11.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.717+0000 7f0259636640 1 --2- 172.21.15.105:0/640208946 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02540d0540 0x7f025406b9c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f02500097d0 tx=0x7f025002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bdc94b90ddf5f5c server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:11.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.717+0000 7f024bfff640 1 -- 172.21.15.105:0/640208946 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0250035020 con 0x7f02540d0540 2024-04-15T15:20:11.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.717+0000 7f024bfff640 1 -- 172.21.15.105:0/640208946 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0250031830 con 0x7f02540d0540 2024-04-15T15:20:11.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.717+0000 7f024bfff640 1 -- 172.21.15.105:0/640208946 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0250031c00 con 0x7f02540d0540 2024-04-15T15:20:11.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.717+0000 7f025b89a640 1 -- 172.21.15.105:0/640208946 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02540d0540 msgr2=0x7f025406b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:11.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.717+0000 7f025b89a640 1 --2- 172.21.15.105:0/640208946 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02540d0540 0x7f025406b9c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f02500097d0 tx=0x7f025002b8d0 comp rx=0 tx=0).stop 2024-04-15T15:20:11.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.718+0000 7f025b89a640 1 -- 172.21.15.105:0/640208946 shutdown_connections 2024-04-15T15:20:11.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.718+0000 7f025b89a640 1 --2- 172.21.15.105:0/640208946 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02540d0540 0x7f025406b9c0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:11.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.718+0000 7f025b89a640 1 -- 172.21.15.105:0/640208946 >> 172.21.15.105:0/640208946 conn(0x7f025409aa40 msgr2=0x7f02540b30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:11.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.718+0000 7f025b89a640 1 -- 172.21.15.105:0/640208946 shutdown_connections 2024-04-15T15:20:11.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.718+0000 7f025b89a640 1 -- 172.21.15.105:0/640208946 wait complete. 2024-04-15T15:20:11.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.719+0000 7f025b89a640 1 Processor -- start 2024-04-15T15:20:11.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.719+0000 7f025b89a640 1 -- start start 2024-04-15T15:20:11.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.720+0000 7f025b89a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02540b3c80 0x7f0254150620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:11.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.720+0000 7f025b89a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f02540ce800 0x7f02540b5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:11.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.720+0000 7f025b89a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f02540d0540 0x7f0254156e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:11.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.720+0000 7f025b89a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f02540c7130 con 0x7f02540b3c80 2024-04-15T15:20:11.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.720+0000 7f025b89a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f025406f180 con 0x7f02540ce800 2024-04-15T15:20:11.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.720+0000 7f025b89a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f02540c72b0 con 0x7f02540d0540 2024-04-15T15:20:11.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.720+0000 7f0259636640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02540b3c80 0x7f0254150620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:11.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.720+0000 7f0259636640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02540b3c80 0x7f0254150620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39714/0 (socket says 172.21.15.105:39714) 2024-04-15T15:20:11.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.720+0000 7f0259636640 1 -- 172.21.15.105:0/2869034118 learned_addr learned my addr 172.21.15.105:0/2869034118 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:11.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.720+0000 7f0258e35640 1 --2- 172.21.15.105:0/2869034118 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f02540ce800 0x7f02540b5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:11.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.720+0000 7f0259e37640 1 --2- 172.21.15.105:0/2869034118 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f02540d0540 0x7f0254156e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:11.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.720+0000 7f0259636640 1 -- 172.21.15.105:0/2869034118 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f02540d0540 msgr2=0x7f0254156e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:11.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.720+0000 7f0259636640 1 --2- 172.21.15.105:0/2869034118 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f02540d0540 0x7f0254156e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:11.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.720+0000 7f0259636640 1 -- 172.21.15.105:0/2869034118 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f02540ce800 msgr2=0x7f02540b5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:11.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.720+0000 7f0259636640 1 --2- 172.21.15.105:0/2869034118 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f02540ce800 0x7f02540b5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:11.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.720+0000 7f0259636640 1 -- 172.21.15.105:0/2869034118 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02540b6f30 con 0x7f02540b3c80 2024-04-15T15:20:11.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.720+0000 7f0259636640 1 --2- 172.21.15.105:0/2869034118 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02540b3c80 0x7f0254150620 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0250000c00 tx=0x7f02500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:11.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.720+0000 7f0249ffb640 1 -- 172.21.15.105:0/2869034118 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0250004230 con 0x7f02540b3c80 2024-04-15T15:20:11.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.720+0000 7f0249ffb640 1 -- 172.21.15.105:0/2869034118 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02500043d0 con 0x7f02540b3c80 2024-04-15T15:20:11.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.720+0000 7f025b89a640 1 -- 172.21.15.105:0/2869034118 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02540b5e90 con 0x7f02540b3c80 2024-04-15T15:20:11.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.720+0000 7f0249ffb640 1 -- 172.21.15.105:0/2869034118 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0250003830 con 0x7f02540b3c80 2024-04-15T15:20:11.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.721+0000 7f025b89a640 1 -- 172.21.15.105:0/2869034118 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02540b6450 con 0x7f02540b3c80 2024-04-15T15:20:11.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.721+0000 7f025b89a640 1 -- 172.21.15.105:0/2869034118 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0254068a40 con 0x7f02540b3c80 2024-04-15T15:20:11.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.722+0000 7f0249ffb640 1 -- 172.21.15.105:0/2869034118 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f025004c060 con 0x7f02540b3c80 2024-04-15T15:20:11.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.722+0000 7f0249ffb640 1 --2- 172.21.15.105:0/2869034118 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f023806c8f0 0x7f023806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:11.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.722+0000 7f0249ffb640 1 -- 172.21.15.105:0/2869034118 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f02500ad720 con 0x7f02540b3c80 2024-04-15T15:20:11.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.722+0000 7f0258e35640 1 --2- 172.21.15.105:0/2869034118 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f023806c8f0 0x7f023806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:11.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.723+0000 7f0258e35640 1 --2- 172.21.15.105:0/2869034118 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f023806c8f0 0x7f023806edb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f02540aefa0 tx=0x7f0244007b50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:11.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.727+0000 7f0249ffb640 1 -- 172.21.15.105:0/2869034118 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0250040070 con 0x7f02540b3c80 2024-04-15T15:20:11.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.941+0000 7f025b89a640 1 -- 172.21.15.105:0/2869034118 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f025406dee0 con 0x7f02540b3c80 2024-04-15T15:20:11.944 INFO:tasks.ceph.mon.a.smithi105.stderr:2024-04-15T15:20:11.942+0000 7fd391944640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-15T15:20:11.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.943+0000 7f0249ffb640 1 -- 172.21.15.105:0/2869034118 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v7) ==== 161+0+0 (secure 0 0 0) 0x7f025007bae0 con 0x7f02540b3c80 2024-04-15T15:20:11.945 INFO:teuthology.orchestra.run.smithi105.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-15T15:20:11.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.946+0000 7f025b89a640 1 -- 172.21.15.105:0/2869034118 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f023806c8f0 msgr2=0x7f023806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:11.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.946+0000 7f025b89a640 1 --2- 172.21.15.105:0/2869034118 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f023806c8f0 0x7f023806edb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f02540aefa0 tx=0x7f0244007b50 comp rx=0 tx=0).stop 2024-04-15T15:20:11.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.946+0000 7f025b89a640 1 -- 172.21.15.105:0/2869034118 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02540b3c80 msgr2=0x7f0254150620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:11.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.946+0000 7f025b89a640 1 --2- 172.21.15.105:0/2869034118 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02540b3c80 0x7f0254150620 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0250000c00 tx=0x7f02500046c0 comp rx=0 tx=0).stop 2024-04-15T15:20:11.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.946+0000 7f025b89a640 1 -- 172.21.15.105:0/2869034118 shutdown_connections 2024-04-15T15:20:11.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.946+0000 7f025b89a640 1 --2- 172.21.15.105:0/2869034118 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f02540d0540 0x7f0254156e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:11.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.946+0000 7f025b89a640 1 --2- 172.21.15.105:0/2869034118 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f023806c8f0 0x7f023806edb0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:11.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.946+0000 7f025b89a640 1 --2- 172.21.15.105:0/2869034118 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f02540ce800 0x7f02540b5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:11.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.946+0000 7f025b89a640 1 --2- 172.21.15.105:0/2869034118 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02540b3c80 0x7f0254150620 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:11.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.946+0000 7f025b89a640 1 -- 172.21.15.105:0/2869034118 >> 172.21.15.105:0/2869034118 conn(0x7f025409aa40 msgr2=0x7f025415dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:11.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.946+0000 7f025b89a640 1 -- 172.21.15.105:0/2869034118 shutdown_connections 2024-04-15T15:20:11.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:11.946+0000 7f025b89a640 1 -- 172.21.15.105:0/2869034118 wait complete. 2024-04-15T15:20:11.951 INFO:tasks.ceph.mds.b.smithi154.stderr: -2994> 2024-04-15T15:19:30.347+0000 7fe2b2e07640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-15T15:20:11.960 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-15T15:20:11.982 INFO:tasks.ceph.mds.b.smithi154.stdout:starting mds.b at 2024-04-15T15:20:12.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.062+0000 7f9e0fb52640 1 Processor -- start 2024-04-15T15:20:12.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.063+0000 7f9e0fb52640 1 -- start start 2024-04-15T15:20:12.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.063+0000 7f9e0fb52640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e080d0480 0x7f9e080b8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:12.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.063+0000 7f9e0fb52640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9e080b6240 0x7f9e080b66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:12.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.063+0000 7f9e0fb52640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e080d2430 con 0x7f9e0815f370 2024-04-15T15:20:12.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.063+0000 7f9e0fb52640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f9e080d3040 con 0x7f9e080b6240 2024-04-15T15:20:12.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.063+0000 7f9e0fb52640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f9e080d2710 con 0x7f9e080d0480 2024-04-15T15:20:12.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.064+0000 7f9e0e0ef640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9e080b6240 0x7f9e080b66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:12.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.064+0000 7f9e0d0ed640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e080d0480 0x7f9e080b8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:12.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.064+0000 7f9e0d8ee640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f9e0815f370 0x7f9e080cfd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.105:33012/0 (socket says 172.21.15.105:33012) 2024-04-15T15:20:12.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.064+0000 7f9e0d8ee640 1 -- 172.21.15.105:0/3216788200 learned_addr learned my addr 172.21.15.105:0/3216788200 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:20:12.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.064+0000 7f9e0e0ef640 1 -- 172.21.15.105:0/3216788200 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e080d0480 msgr2=0x7f9e080b8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:12.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.064+0000 7f9e0e0ef640 1 --2- 172.21.15.105:0/3216788200 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e080d0480 0x7f9e080b8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:12.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.064+0000 7f9e0e0ef640 1 -- 172.21.15.105:0/3216788200 >> v1:172.21.15.154:6789/0 conn(0x7f9e0815f370 legacy=0x7f9e080cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:12.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.064+0000 7f9e0e0ef640 1 -- 172.21.15.105:0/3216788200 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e080d29f0 con 0x7f9e080b6240 2024-04-15T15:20:12.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.064+0000 7f9e0d0ed640 1 --2- 172.21.15.105:0/3216788200 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e080d0480 0x7f9e080b8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:20:12.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.065+0000 7f9e0e0ef640 1 --2- 172.21.15.105:0/3216788200 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e080b6240 0x7f9e080b66f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9df80097d0 tx=0x7f9df802b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca34433db187482d server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:12.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.065+0000 7f9e0c8ec640 1 -- 172.21.15.105:0/3216788200 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df8035020 con 0x7f9e080b6240 2024-04-15T15:20:12.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.065+0000 7f9e0c8ec640 1 -- 172.21.15.105:0/3216788200 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9df8031800 con 0x7f9e080b6240 2024-04-15T15:20:12.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.065+0000 7f9e0c8ec640 1 -- 172.21.15.105:0/3216788200 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df8031bd0 con 0x7f9e080b6240 2024-04-15T15:20:12.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.066+0000 7f9e0fb52640 1 -- 172.21.15.105:0/3216788200 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e080b6240 msgr2=0x7f9e080b66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:12.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.066+0000 7f9e0fb52640 1 --2- 172.21.15.105:0/3216788200 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e080b6240 0x7f9e080b66f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9df80097d0 tx=0x7f9df802b8a0 comp rx=0 tx=0).stop 2024-04-15T15:20:12.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.066+0000 7f9e0fb52640 1 -- 172.21.15.105:0/3216788200 shutdown_connections 2024-04-15T15:20:12.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.066+0000 7f9e0fb52640 1 --2- 172.21.15.105:0/3216788200 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e080b6240 0x7f9e080b66f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:12.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.066+0000 7f9e0fb52640 1 --2- 172.21.15.105:0/3216788200 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e080d0480 0x7f9e080b8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:12.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.066+0000 7f9e0fb52640 1 -- 172.21.15.105:0/3216788200 >> 172.21.15.105:0/3216788200 conn(0x7f9e080d1240 msgr2=0x7f9e080d1650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:12.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.066+0000 7f9e0fb52640 1 -- 172.21.15.105:0/3216788200 shutdown_connections 2024-04-15T15:20:12.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.066+0000 7f9e0fb52640 1 -- 172.21.15.105:0/3216788200 wait complete. 2024-04-15T15:20:12.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.067+0000 7f9e0fb52640 1 Processor -- start 2024-04-15T15:20:12.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.067+0000 7f9e0fb52640 1 -- start start 2024-04-15T15:20:12.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.068+0000 7f9e0fb52640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e080b6240 0x7f9e081535d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:12.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.068+0000 7f9e0fb52640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e080d0480 0x7f9e08154640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:12.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.068+0000 7f9e0fb52640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e0815f370 0x7f9e080b94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:12.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.068+0000 7f9e0fb52640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9e080d34d0 con 0x7f9e0815f370 2024-04-15T15:20:12.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.068+0000 7f9e0fb52640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9e080b6c30 con 0x7f9e080d0480 2024-04-15T15:20:12.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.068+0000 7f9e0fb52640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f9e080d3650 con 0x7f9e080b6240 2024-04-15T15:20:12.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.068+0000 7f9e0e0ef640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e0815f370 0x7f9e080b94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:12.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.068+0000 7f9e0e0ef640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e0815f370 0x7f9e080b94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39730/0 (socket says 172.21.15.105:39730) 2024-04-15T15:20:12.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.068+0000 7f9e0e0ef640 1 -- 172.21.15.105:0/2061801524 learned_addr learned my addr 172.21.15.105:0/2061801524 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:12.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.068+0000 7f9e0d0ed640 1 --2- 172.21.15.105:0/2061801524 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e080d0480 0x7f9e08154640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:12.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.068+0000 7f9e0d8ee640 1 --2- 172.21.15.105:0/2061801524 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e080b6240 0x7f9e081535d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:12.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.068+0000 7f9e0e0ef640 1 -- 172.21.15.105:0/2061801524 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e080b6240 msgr2=0x7f9e081535d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:12.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.068+0000 7f9e0e0ef640 1 --2- 172.21.15.105:0/2061801524 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e080b6240 0x7f9e081535d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:12.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.068+0000 7f9e0e0ef640 1 -- 172.21.15.105:0/2061801524 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e080d0480 msgr2=0x7f9e08154640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:12.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.068+0000 7f9e0e0ef640 1 --2- 172.21.15.105:0/2061801524 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e080d0480 0x7f9e08154640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:12.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.068+0000 7f9e0e0ef640 1 -- 172.21.15.105:0/2061801524 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e08153b10 con 0x7f9e0815f370 2024-04-15T15:20:12.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.068+0000 7f9e0e0ef640 1 --2- 172.21.15.105:0/2061801524 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e0815f370 0x7f9e080b94d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f9df802beb0 tx=0x7f9df80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:12.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.068+0000 7f9e067fc640 1 -- 172.21.15.105:0/2061801524 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df8041070 con 0x7f9e0815f370 2024-04-15T15:20:12.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.068+0000 7f9e067fc640 1 -- 172.21.15.105:0/2061801524 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9df803ac80 con 0x7f9e0815f370 2024-04-15T15:20:12.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.068+0000 7f9e067fc640 1 -- 172.21.15.105:0/2061801524 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df8035040 con 0x7f9e0815f370 2024-04-15T15:20:12.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.068+0000 7f9e0fb52640 1 -- 172.21.15.105:0/2061801524 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e08154b80 con 0x7f9e0815f370 2024-04-15T15:20:12.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.068+0000 7f9e0fb52640 1 -- 172.21.15.105:0/2061801524 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e080b4e20 con 0x7f9e0815f370 2024-04-15T15:20:12.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.069+0000 7f9e0fb52640 1 -- 172.21.15.105:0/2061801524 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e08069150 con 0x7f9e0815f370 2024-04-15T15:20:12.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.070+0000 7f9e067fc640 1 -- 172.21.15.105:0/2061801524 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9df8038020 con 0x7f9e0815f370 2024-04-15T15:20:12.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.070+0000 7f9e067fc640 1 --2- 172.21.15.105:0/2061801524 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f9df406c8f0 0x7f9df406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:12.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.071+0000 7f9e0d8ee640 1 --2- 172.21.15.105:0/2061801524 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f9df406c8f0 0x7f9df406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:12.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.071+0000 7f9e067fc640 1 -- 172.21.15.105:0/2061801524 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5911+0+0 (secure 0 0 0) 0x7f9df80ad6b0 con 0x7f9e0815f370 2024-04-15T15:20:12.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.071+0000 7f9e0d8ee640 1 --2- 172.21.15.105:0/2061801524 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f9df406c8f0 0x7f9df406edb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9e00007fd0 tx=0x7f9e00007e80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:12.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.075+0000 7f9e067fc640 1 -- 172.21.15.105:0/2061801524 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9df807b3c0 con 0x7f9e0815f370 2024-04-15T15:20:12.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.285+0000 7f9e0fb52640 1 -- 172.21.15.105:0/2061801524 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f9e08154250 con 0x7f9e0815f370 2024-04-15T15:20:12.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.945+0000 7f9e067fc640 1 -- 172.21.15.105:0/2061801524 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v8) ==== 104+0+0 (secure 0 0 0) 0x7f9df807f6c0 con 0x7f9e0815f370 2024-04-15T15:20:12.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.947+0000 7f9e0fb52640 1 -- 172.21.15.105:0/2061801524 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f9df406c8f0 msgr2=0x7f9df406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:12.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.947+0000 7f9e0fb52640 1 --2- 172.21.15.105:0/2061801524 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f9df406c8f0 0x7f9df406edb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9e00007fd0 tx=0x7f9e00007e80 comp rx=0 tx=0).stop 2024-04-15T15:20:12.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.947+0000 7f9e0fb52640 1 -- 172.21.15.105:0/2061801524 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e0815f370 msgr2=0x7f9e080b94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:12.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.947+0000 7f9e0fb52640 1 --2- 172.21.15.105:0/2061801524 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e0815f370 0x7f9e080b94d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f9df802beb0 tx=0x7f9df80046c0 comp rx=0 tx=0).stop 2024-04-15T15:20:12.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.947+0000 7f9e0fb52640 1 -- 172.21.15.105:0/2061801524 shutdown_connections 2024-04-15T15:20:12.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.947+0000 7f9e0fb52640 1 --2- 172.21.15.105:0/2061801524 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f9df406c8f0 0x7f9df406edb0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:12.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.947+0000 7f9e0fb52640 1 --2- 172.21.15.105:0/2061801524 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e0815f370 0x7f9e080b94d0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:12.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.947+0000 7f9e0fb52640 1 --2- 172.21.15.105:0/2061801524 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e080d0480 0x7f9e08154640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:12.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.947+0000 7f9e0fb52640 1 --2- 172.21.15.105:0/2061801524 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e080b6240 0x7f9e081535d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:12.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.947+0000 7f9e0fb52640 1 -- 172.21.15.105:0/2061801524 >> 172.21.15.105:0/2061801524 conn(0x7f9e080d1240 msgr2=0x7f9e080b5a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:12.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.947+0000 7f9e0fb52640 1 -- 172.21.15.105:0/2061801524 shutdown_connections 2024-04-15T15:20:12.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:12.947+0000 7f9e0fb52640 1 -- 172.21.15.105:0/2061801524 wait complete. 2024-04-15T15:20:12.961 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-15T15:20:13.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.068+0000 7f8a47e33640 1 Processor -- start 2024-04-15T15:20:13.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.068+0000 7f8a47e33640 1 -- start start 2024-04-15T15:20:13.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.069+0000 7f8a47e33640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8a4013ecd0 0x7f8a4006efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:13.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.069+0000 7f8a47e33640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8a4006f500 0x7f8a4015bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:13.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.069+0000 7f8a47e33640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a4015f280 con 0x7f8a4013e6f0 2024-04-15T15:20:13.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.069+0000 7f8a47e33640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f8a4015f560 con 0x7f8a4006f500 2024-04-15T15:20:13.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.069+0000 7f8a47e33640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f8a4015f840 con 0x7f8a4013ecd0 2024-04-15T15:20:13.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.069+0000 7f8a463d0640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8a4006f500 0x7f8a4015bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:13.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.069+0000 7f8a463d0640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8a4006f500 0x7f8a4015bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39738/0 (socket says 172.21.15.105:39738) 2024-04-15T15:20:13.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.069+0000 7f8a463d0640 1 -- 172.21.15.105:0/1372029972 learned_addr learned my addr 172.21.15.105:0/1372029972 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:13.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.069+0000 7f8a453ce640 1 --2- 172.21.15.105:0/1372029972 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8a4013ecd0 0x7f8a4006efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:13.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.069+0000 7f8a463d0640 1 -- 172.21.15.105:0/1372029972 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8a4013ecd0 msgr2=0x7f8a4006efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:13.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.069+0000 7f8a463d0640 1 --2- 172.21.15.105:0/1372029972 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8a4013ecd0 0x7f8a4006efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:13.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.069+0000 7f8a463d0640 1 -- 172.21.15.105:0/1372029972 >> v1:172.21.15.154:6789/0 conn(0x7f8a4013e6f0 legacy=0x7f8a400cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:13.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.069+0000 7f8a463d0640 1 -- 172.21.15.105:0/1372029972 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a4015fb20 con 0x7f8a4006f500 2024-04-15T15:20:13.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.070+0000 7f8a463d0640 1 --2- 172.21.15.105:0/1372029972 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a4006f500 0x7f8a4015bcd0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8a38009870 tx=0x7f8a3802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e01e8f30ec1eed8b server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:13.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.070+0000 7f8a44bcd640 1 -- 172.21.15.105:0/1372029972 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a380317c0 con 0x7f8a4006f500 2024-04-15T15:20:13.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.070+0000 7f8a44bcd640 1 -- 172.21.15.105:0/1372029972 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a38037070 con 0x7f8a4006f500 2024-04-15T15:20:13.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.070+0000 7f8a44bcd640 1 -- 172.21.15.105:0/1372029972 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a38031b80 con 0x7f8a4006f500 2024-04-15T15:20:13.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.071+0000 7f8a47e33640 1 -- 172.21.15.105:0/1372029972 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a4006f500 msgr2=0x7f8a4015bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:13.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.071+0000 7f8a47e33640 1 --2- 172.21.15.105:0/1372029972 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a4006f500 0x7f8a4015bcd0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8a38009870 tx=0x7f8a3802b860 comp rx=0 tx=0).stop 2024-04-15T15:20:13.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.071+0000 7f8a47e33640 1 -- 172.21.15.105:0/1372029972 shutdown_connections 2024-04-15T15:20:13.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.071+0000 7f8a47e33640 1 --2- 172.21.15.105:0/1372029972 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a4006f500 0x7f8a4015bcd0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:13.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.071+0000 7f8a47e33640 1 --2- 172.21.15.105:0/1372029972 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8a4013ecd0 0x7f8a4006efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:13.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.071+0000 7f8a47e33640 1 -- 172.21.15.105:0/1372029972 >> 172.21.15.105:0/1372029972 conn(0x7f8a40067e10 msgr2=0x7f8a400b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:13.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.071+0000 7f8a47e33640 1 -- 172.21.15.105:0/1372029972 shutdown_connections 2024-04-15T15:20:13.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.071+0000 7f8a47e33640 1 -- 172.21.15.105:0/1372029972 wait complete. 2024-04-15T15:20:13.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.072+0000 7f8a47e33640 1 Processor -- start 2024-04-15T15:20:13.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.072+0000 7f8a47e33640 1 -- start start 2024-04-15T15:20:13.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.073+0000 7f8a47e33640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a4006f500 0x7f8a40139490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:13.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.073+0000 7f8a47e33640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a4013e6f0 0x7f8a4013dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:13.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.073+0000 7f8a47e33640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8a4013ecd0 0x7f8a40150b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:13.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.073+0000 7f8a47e33640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8a400cf670 con 0x7f8a4013e6f0 2024-04-15T15:20:13.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.073+0000 7f8a47e33640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8a400d0230 con 0x7f8a4006f500 2024-04-15T15:20:13.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.073+0000 7f8a47e33640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f8a400cf7f0 con 0x7f8a4013ecd0 2024-04-15T15:20:13.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.073+0000 7f8a453ce640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a4013e6f0 0x7f8a4013dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:13.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.073+0000 7f8a453ce640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a4013e6f0 0x7f8a4013dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39750/0 (socket says 172.21.15.105:39750) 2024-04-15T15:20:13.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.073+0000 7f8a453ce640 1 -- 172.21.15.105:0/3265615000 learned_addr learned my addr 172.21.15.105:0/3265615000 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:13.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.073+0000 7f8a45bcf640 1 --2- 172.21.15.105:0/3265615000 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a4006f500 0x7f8a40139490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:13.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.073+0000 7f8a463d0640 1 --2- 172.21.15.105:0/3265615000 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8a4013ecd0 0x7f8a40150b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:13.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.073+0000 7f8a453ce640 1 -- 172.21.15.105:0/3265615000 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8a4013ecd0 msgr2=0x7f8a40150b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:13.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.073+0000 7f8a453ce640 1 --2- 172.21.15.105:0/3265615000 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8a4013ecd0 0x7f8a40150b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:13.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.073+0000 7f8a453ce640 1 -- 172.21.15.105:0/3265615000 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a4006f500 msgr2=0x7f8a40139490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:13.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.073+0000 7f8a453ce640 1 --2- 172.21.15.105:0/3265615000 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a4006f500 0x7f8a40139490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:13.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.073+0000 7f8a453ce640 1 -- 172.21.15.105:0/3265615000 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a40139a70 con 0x7f8a4013e6f0 2024-04-15T15:20:13.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.074+0000 7f8a453ce640 1 --2- 172.21.15.105:0/3265615000 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a4013e6f0 0x7f8a4013dcf0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8a3400c7f0 tx=0x7f8a3400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:13.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.074+0000 7f8a3e7fc640 1 -- 172.21.15.105:0/3265615000 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a34015070 con 0x7f8a4013e6f0 2024-04-15T15:20:13.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.074+0000 7f8a3e7fc640 1 -- 172.21.15.105:0/3265615000 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a34004220 con 0x7f8a4013e6f0 2024-04-15T15:20:13.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.074+0000 7f8a47e33640 1 -- 172.21.15.105:0/3265615000 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a40139d50 con 0x7f8a4013e6f0 2024-04-15T15:20:13.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.074+0000 7f8a3e7fc640 1 -- 172.21.15.105:0/3265615000 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a34007620 con 0x7f8a4013e6f0 2024-04-15T15:20:13.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.074+0000 7f8a47e33640 1 -- 172.21.15.105:0/3265615000 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a4013e230 con 0x7f8a4013e6f0 2024-04-15T15:20:13.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.075+0000 7f8a47e33640 1 -- 172.21.15.105:0/3265615000 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a400b4040 con 0x7f8a4013e6f0 2024-04-15T15:20:13.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.075+0000 7f8a3e7fc640 1 -- 172.21.15.105:0/3265615000 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8a340079d0 con 0x7f8a4013e6f0 2024-04-15T15:20:13.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.076+0000 7f8a3e7fc640 1 --2- 172.21.15.105:0/3265615000 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f8a2c06c8f0 0x7f8a2c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:13.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.076+0000 7f8a3e7fc640 1 -- 172.21.15.105:0/3265615000 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5911+0+0 (secure 0 0 0) 0x7f8a3408bd10 con 0x7f8a4013e6f0 2024-04-15T15:20:13.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.076+0000 7f8a45bcf640 1 --2- 172.21.15.105:0/3265615000 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f8a2c06c8f0 0x7f8a2c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:13.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.076+0000 7f8a45bcf640 1 --2- 172.21.15.105:0/3265615000 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f8a2c06c8f0 0x7f8a2c06edb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8a30001db0 tx=0x7f8a30009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:13.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.080+0000 7f8a3e7fc640 1 -- 172.21.15.105:0/3265615000 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8a400b4040 con 0x7f8a4013e6f0 2024-04-15T15:20:13.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.255+0000 7f8a47e33640 1 -- 172.21.15.105:0/3265615000 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8a4015b4a0 con 0x7f8a4013e6f0 2024-04-15T15:20:13.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.952+0000 7f8a3e7fc640 1 -- 172.21.15.105:0/3265615000 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) ==== 181+0+0 (secure 0 0 0) 0x7f8a4015b4a0 con 0x7f8a4013e6f0 2024-04-15T15:20:13.954 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_metadata' removed 2024-04-15T15:20:13.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.954+0000 7f8a47e33640 1 -- 172.21.15.105:0/3265615000 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f8a2c06c8f0 msgr2=0x7f8a2c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:13.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.954+0000 7f8a47e33640 1 --2- 172.21.15.105:0/3265615000 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f8a2c06c8f0 0x7f8a2c06edb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8a30001db0 tx=0x7f8a30009040 comp rx=0 tx=0).stop 2024-04-15T15:20:13.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.954+0000 7f8a47e33640 1 -- 172.21.15.105:0/3265615000 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a4013e6f0 msgr2=0x7f8a4013dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:13.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.954+0000 7f8a47e33640 1 --2- 172.21.15.105:0/3265615000 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a4013e6f0 0x7f8a4013dcf0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8a3400c7f0 tx=0x7f8a3400ccb0 comp rx=0 tx=0).stop 2024-04-15T15:20:13.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.954+0000 7f8a47e33640 1 -- 172.21.15.105:0/3265615000 shutdown_connections 2024-04-15T15:20:13.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.954+0000 7f8a47e33640 1 --2- 172.21.15.105:0/3265615000 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8a4013ecd0 0x7f8a40150b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:13.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.954+0000 7f8a47e33640 1 --2- 172.21.15.105:0/3265615000 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f8a2c06c8f0 0x7f8a2c06edb0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:13.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.954+0000 7f8a47e33640 1 --2- 172.21.15.105:0/3265615000 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a4013e6f0 0x7f8a4013dcf0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:13.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.954+0000 7f8a47e33640 1 --2- 172.21.15.105:0/3265615000 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a4006f500 0x7f8a40139490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:13.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.954+0000 7f8a47e33640 1 -- 172.21.15.105:0/3265615000 >> 172.21.15.105:0/3265615000 conn(0x7f8a40067e10 msgr2=0x7f8a400b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:13.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.954+0000 7f8a47e33640 1 -- 172.21.15.105:0/3265615000 shutdown_connections 2024-04-15T15:20:13.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:13.954+0000 7f8a47e33640 1 -- 172.21.15.105:0/3265615000 wait complete. 2024-04-15T15:20:13.971 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-15T15:20:14.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.076+0000 7fc6ba84a640 1 Processor -- start 2024-04-15T15:20:14.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.076+0000 7fc6ba84a640 1 -- start start 2024-04-15T15:20:14.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.077+0000 7fc6ba84a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc6b4160360 0x7fc6b409b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:14.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.077+0000 7fc6ba84a640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc6b40cd090 0x7fc6b4160740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:14.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.078+0000 7fc6ba84a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc6b415cb30 0x7fc6b415cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:14.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.078+0000 7fc6ba84a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fc6b409b990 con 0x7fc6b415cb30 2024-04-15T15:20:14.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.078+0000 7fc6ba84a640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fc6b4160c80 con 0x7fc6b40cd090 2024-04-15T15:20:14.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.078+0000 7fc6ba84a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fc6b40cd550 con 0x7fc6b4160360 2024-04-15T15:20:14.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.078+0000 7fc6abfff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc6b40cd090 0x7fc6b4160740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:14.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.078+0000 7fc6b3fff640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc6b4160360 0x7fc6b409b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:14.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.078+0000 7fc6b8de7640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc6b415cb30 0x7fc6b415cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:14.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.078+0000 7fc6b3fff640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc6b4160360 0x7fc6b409b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.105:57780/0 (socket says 172.21.15.105:57780) 2024-04-15T15:20:14.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.078+0000 7fc6abfff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc6b40cd090 0x7fc6b4160740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.105:57060/0 (socket says 172.21.15.105:57060) 2024-04-15T15:20:14.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.078+0000 7fc6b3fff640 1 -- 172.21.15.105:0/2952303063 learned_addr learned my addr 172.21.15.105:0/2952303063 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:14.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.078+0000 7fc6b8de7640 1 --2- 172.21.15.105:0/2952303063 >> v2:172.21.15.105:3300/0 conn(0x7fc6b415cb30 0x7fc6b415cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39752/0 (socket says 172.21.15.105:39752) 2024-04-15T15:20:14.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.078+0000 7fc6b8de7640 1 -- 172.21.15.105:0/2952303063 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc6b4160360 msgr2=0x7fc6b409b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:14.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.078+0000 7fc6b8de7640 1 --2- 172.21.15.105:0/2952303063 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc6b4160360 0x7fc6b409b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:14.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.078+0000 7fc6b8de7640 1 -- 172.21.15.105:0/2952303063 >> v2:172.21.15.154:3300/0 conn(0x7fc6b40cd090 msgr2=0x7fc6b4160740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:14.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.078+0000 7fc6b8de7640 1 --2- 172.21.15.105:0/2952303063 >> v2:172.21.15.154:3300/0 conn(0x7fc6b40cd090 0x7fc6b4160740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:14.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.078+0000 7fc6b8de7640 1 -- 172.21.15.105:0/2952303063 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6b40cd830 con 0x7fc6b415cb30 2024-04-15T15:20:14.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.079+0000 7fc6b8de7640 1 --2- 172.21.15.105:0/2952303063 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6b415cb30 0x7fc6b415cfc0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc6b40739e0 tx=0x7fc6ac02be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dca101202a34201 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:14.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.079+0000 7fc6b37fe640 1 -- 172.21.15.105:0/2952303063 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac039070 con 0x7fc6b415cb30 2024-04-15T15:20:14.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.079+0000 7fc6b37fe640 1 -- 172.21.15.105:0/2952303063 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6ac02cbd0 con 0x7fc6b415cb30 2024-04-15T15:20:14.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.079+0000 7fc6b37fe640 1 -- 172.21.15.105:0/2952303063 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac034440 con 0x7fc6b415cb30 2024-04-15T15:20:14.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.079+0000 7fc6ba84a640 1 -- 172.21.15.105:0/2952303063 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6b415cb30 msgr2=0x7fc6b415cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:14.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.080+0000 7fc6ba84a640 1 --2- 172.21.15.105:0/2952303063 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6b415cb30 0x7fc6b415cfc0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc6b40739e0 tx=0x7fc6ac02be20 comp rx=0 tx=0).stop 2024-04-15T15:20:14.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.080+0000 7fc6ba84a640 1 -- 172.21.15.105:0/2952303063 shutdown_connections 2024-04-15T15:20:14.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.080+0000 7fc6ba84a640 1 --2- 172.21.15.105:0/2952303063 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6b415cb30 0x7fc6b415cfc0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:14.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.080+0000 7fc6ba84a640 1 --2- 172.21.15.105:0/2952303063 >> v2:172.21.15.154:3300/0 conn(0x7fc6b40cd090 0x7fc6b4160740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:14.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.080+0000 7fc6ba84a640 1 --2- 172.21.15.105:0/2952303063 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc6b4160360 0x7fc6b409b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:14.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.080+0000 7fc6ba84a640 1 -- 172.21.15.105:0/2952303063 >> 172.21.15.105:0/2952303063 conn(0x7fc6b40b4070 msgr2=0x7fc6b40b4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:14.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.080+0000 7fc6ba84a640 1 -- 172.21.15.105:0/2952303063 shutdown_connections 2024-04-15T15:20:14.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.080+0000 7fc6ba84a640 1 -- 172.21.15.105:0/2952303063 wait complete. 2024-04-15T15:20:14.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.081+0000 7fc6ba84a640 1 Processor -- start 2024-04-15T15:20:14.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.081+0000 7fc6ba84a640 1 -- start start 2024-04-15T15:20:14.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.082+0000 7fc6ba84a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc6b40cd090 0x7fc6b41f19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:14.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.082+0000 7fc6ba84a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6b415cb30 0x7fc6b40d0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:14.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.082+0000 7fc6ba84a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc6b4160360 0x7fc6b4155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:14.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.082+0000 7fc6ba84a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b40ce0d0 con 0x7fc6b415cb30 2024-04-15T15:20:14.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.082+0000 7fc6ba84a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b40cddf0 con 0x7fc6b40cd090 2024-04-15T15:20:14.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.082+0000 7fc6ba84a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fc6b41610b0 con 0x7fc6b4160360 2024-04-15T15:20:14.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.082+0000 7fc6abfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6b415cb30 0x7fc6b40d0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:14.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.082+0000 7fc6abfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6b415cb30 0x7fc6b40d0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39764/0 (socket says 172.21.15.105:39764) 2024-04-15T15:20:14.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.082+0000 7fc6abfff640 1 -- 172.21.15.105:0/1769182800 learned_addr learned my addr 172.21.15.105:0/1769182800 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:14.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.082+0000 7fc6b3fff640 1 --2- 172.21.15.105:0/1769182800 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc6b40cd090 0x7fc6b41f19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:14.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.082+0000 7fc6b8de7640 1 --2- 172.21.15.105:0/1769182800 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc6b4160360 0x7fc6b4155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:14.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.082+0000 7fc6abfff640 1 -- 172.21.15.105:0/1769182800 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc6b4160360 msgr2=0x7fc6b4155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:14.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.082+0000 7fc6abfff640 1 --2- 172.21.15.105:0/1769182800 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc6b4160360 0x7fc6b4155fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:14.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.082+0000 7fc6abfff640 1 -- 172.21.15.105:0/1769182800 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc6b40cd090 msgr2=0x7fc6b41f19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:14.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.082+0000 7fc6abfff640 1 --2- 172.21.15.105:0/1769182800 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc6b40cd090 0x7fc6b41f19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:14.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.082+0000 7fc6abfff640 1 -- 172.21.15.105:0/1769182800 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6b41f1f20 con 0x7fc6b415cb30 2024-04-15T15:20:14.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.082+0000 7fc6abfff640 1 --2- 172.21.15.105:0/1769182800 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6b415cb30 0x7fc6b40d0c00 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc69c00e870 tx=0x7fc69c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:14.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.082+0000 7fc6b17fa640 1 -- 172.21.15.105:0/1769182800 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc69c017070 con 0x7fc6b415cb30 2024-04-15T15:20:14.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.082+0000 7fc6b17fa640 1 -- 172.21.15.105:0/1769182800 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc69c00c940 con 0x7fc6b415cb30 2024-04-15T15:20:14.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.082+0000 7fc6b17fa640 1 -- 172.21.15.105:0/1769182800 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc69c00cd30 con 0x7fc6b415cb30 2024-04-15T15:20:14.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.082+0000 7fc6ba84a640 1 -- 172.21.15.105:0/1769182800 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6b40cf800 con 0x7fc6b415cb30 2024-04-15T15:20:14.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.082+0000 7fc6ba84a640 1 -- 172.21.15.105:0/1769182800 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6b40cfdc0 con 0x7fc6b415cb30 2024-04-15T15:20:14.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.083+0000 7fc6ba84a640 1 -- 172.21.15.105:0/1769182800 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6b4068a40 con 0x7fc6b415cb30 2024-04-15T15:20:14.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.084+0000 7fc6b17fa640 1 -- 172.21.15.105:0/1769182800 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc69c01a070 con 0x7fc6b415cb30 2024-04-15T15:20:14.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.084+0000 7fc6b17fa640 1 --2- 172.21.15.105:0/1769182800 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fc69406c8f0 0x7fc69406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:14.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.084+0000 7fc6b17fa640 1 -- 172.21.15.105:0/1769182800 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5473+0+0 (secure 0 0 0) 0x7fc69c08b800 con 0x7fc6b415cb30 2024-04-15T15:20:14.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.084+0000 7fc6b3fff640 1 --2- 172.21.15.105:0/1769182800 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fc69406c8f0 0x7fc69406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:14.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.085+0000 7fc6b3fff640 1 --2- 172.21.15.105:0/1769182800 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fc69406c8f0 0x7fc69406edb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc6b4158460 tx=0x7fc6a4009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:14.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.089+0000 7fc6b17fa640 1 -- 172.21.15.105:0/1769182800 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc69c059750 con 0x7fc6b415cb30 2024-04-15T15:20:14.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.268+0000 7fc6ba84a640 1 -- 172.21.15.105:0/1769182800 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc6b40d16a0 con 0x7fc6b415cb30 2024-04-15T15:20:14.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.954+0000 7fc6b17fa640 1 -- 172.21.15.105:0/1769182800 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) ==== 169+0+0 (secure 0 0 0) 0x7fc69c05da50 con 0x7fc6b415cb30 2024-04-15T15:20:14.956 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_data' removed 2024-04-15T15:20:14.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.957+0000 7fc6ba84a640 1 -- 172.21.15.105:0/1769182800 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fc69406c8f0 msgr2=0x7fc69406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:14.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.957+0000 7fc6ba84a640 1 --2- 172.21.15.105:0/1769182800 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fc69406c8f0 0x7fc69406edb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc6b4158460 tx=0x7fc6a4009240 comp rx=0 tx=0).stop 2024-04-15T15:20:14.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.957+0000 7fc6ba84a640 1 -- 172.21.15.105:0/1769182800 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6b415cb30 msgr2=0x7fc6b40d0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:14.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.957+0000 7fc6ba84a640 1 --2- 172.21.15.105:0/1769182800 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6b415cb30 0x7fc6b40d0c00 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc69c00e870 tx=0x7fc69c00ed30 comp rx=0 tx=0).stop 2024-04-15T15:20:14.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.957+0000 7fc6ba84a640 1 -- 172.21.15.105:0/1769182800 shutdown_connections 2024-04-15T15:20:14.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.957+0000 7fc6ba84a640 1 --2- 172.21.15.105:0/1769182800 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc6b4160360 0x7fc6b4155fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:14.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.957+0000 7fc6ba84a640 1 --2- 172.21.15.105:0/1769182800 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fc69406c8f0 0x7fc69406edb0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:14.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.957+0000 7fc6ba84a640 1 --2- 172.21.15.105:0/1769182800 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6b415cb30 0x7fc6b40d0c00 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:14.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.957+0000 7fc6ba84a640 1 --2- 172.21.15.105:0/1769182800 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc6b40cd090 0x7fc6b41f19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:14.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.957+0000 7fc6ba84a640 1 -- 172.21.15.105:0/1769182800 >> 172.21.15.105:0/1769182800 conn(0x7fc6b40b4070 msgr2=0x7fc6b40b1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:14.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.957+0000 7fc6ba84a640 1 -- 172.21.15.105:0/1769182800 shutdown_connections 2024-04-15T15:20:14.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:14.957+0000 7fc6ba84a640 1 -- 172.21.15.105:0/1769182800 wait complete. 2024-04-15T15:20:14.974 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-15T15:20:14.974 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-15T15:20:14.974 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-15T15:20:14.975 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:20:14.975 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-15T15:20:14.975 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-15T15:20:14.975 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-15T15:20:14.975 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:20:14.975 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-15T15:20:14.975 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-15T15:20:14.976 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-15T15:20:14.976 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:20:14.976 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-15T15:20:14.976 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-15T15:20:14.976 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-15T15:20:14.976 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:20:14.985 INFO:tasks.ceph.mds.a:Stopped 2024-04-15T15:20:14.986 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-15T15:20:14.986 INFO:tasks.ceph.mds.c:Stopped 2024-04-15T15:20:14.987 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-15T15:20:14.990 INFO:tasks.ceph.mds.d:Stopped 2024-04-15T15:20:14.990 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-15T15:20:15.028 INFO:tasks.ceph.mds.a:Started 2024-04-15T15:20:15.028 DEBUG:teuthology.parallel:result is None 2024-04-15T15:20:15.029 INFO:tasks.ceph.mds.c:Started 2024-04-15T15:20:15.029 DEBUG:teuthology.parallel:result is None 2024-04-15T15:20:15.033 INFO:tasks.ceph.mds.d:Started 2024-04-15T15:20:15.033 DEBUG:teuthology.parallel:result is None 2024-04-15T15:20:15.110 INFO:tasks.ceph.mds.a.smithi105.stdout:starting mds.a at 2024-04-15T15:20:15.110 INFO:tasks.ceph.mds.c.smithi105.stdout:starting mds.c at 2024-04-15T15:20:15.113 INFO:tasks.ceph.mds.d.smithi154.stdout:starting mds.d at 2024-04-15T15:20:18.404 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-04-15T15:20:20.989 INFO:tasks.ceph.mds.b:Stopped 2024-04-15T15:20:20.990 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-15T15:20:20.993 INFO:tasks.ceph.mds.b:Started 2024-04-15T15:20:20.993 DEBUG:teuthology.parallel:result is None 2024-04-15T15:20:20.993 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-15T15:20:21.081 INFO:tasks.ceph.mds.b.smithi154.stdout:starting mds.b at 2024-04-15T15:20:21.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:21.108+0000 7f3ad0893640 1 Processor -- start 2024-04-15T15:20:21.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:21.108+0000 7f3ad0893640 1 -- start start 2024-04-15T15:20:21.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:21.109+0000 7f3ad0893640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3acc162d40 0x7f3acc169960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:21.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:21.109+0000 7f3ad0893640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3acc15ef40 con 0x7f3acc15eb60 2024-04-15T15:20:21.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:21.109+0000 7f3ad0893640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3acc163260 con 0x7f3acc0d18e0 2024-04-15T15:20:21.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:21.109+0000 7f3ad0893640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f3acc163540 con 0x7f3acc162d40 2024-04-15T15:20:21.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:21.110+0000 7f3aca59c640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f3acc15eb60 0x7f3acc0d11d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:44322/0 (socket says 172.21.15.105:44322) 2024-04-15T15:20:21.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:21.110+0000 7f3aca59c640 1 -- 172.21.15.105:0/2942106326 learned_addr learned my addr 172.21.15.105:0/2942106326 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:20:21.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:21.110+0000 7f3acad9d640 1 --2- 172.21.15.105:0/2942106326 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3acc162d40 0x7f3acc169960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:21.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:21.110+0000 7f3ac959a640 1 -- 172.21.15.105:0/2942106326 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1855589397 0 0) 0x7f3acc15ef40 con 0x7f3acc15eb60 2024-04-15T15:20:21.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:21.110+0000 7f3ac959a640 1 -- 172.21.15.105:0/2942106326 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ab4003520 con 0x7f3acc15eb60 2024-04-15T15:20:21.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:21.110+0000 7f3ac959a640 1 -- 172.21.15.105:0/2942106326 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4121003212 0 0) 0x7f3acc163260 con 0x7f3acc0d18e0 2024-04-15T15:20:21.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:21.110+0000 7f3acad9d640 1 -- 172.21.15.105:0/2942106326 >> v1:172.21.15.154:6789/0 conn(0x7f3acc0d18e0 legacy=0x7f3acc162630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:21.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:21.110+0000 7f3acad9d640 1 -- 172.21.15.105:0/2942106326 >> v1:172.21.15.105:6789/0 conn(0x7f3acc15eb60 legacy=0x7f3acc0d11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:21.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:21.110+0000 7f3acad9d640 1 -- 172.21.15.105:0/2942106326 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3acc0d2970 con 0x7f3acc162d40 2024-04-15T15:20:21.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:21.111+0000 7f3acad9d640 1 --2- 172.21.15.105:0/2942106326 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3acc162d40 0x7f3acc169960 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3ac40097d0 tx=0x7f3ac402b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=726e4d690580d8f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:21.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:21.111+0000 7f3ac959a640 1 -- 172.21.15.105:0/2942106326 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac4035020 con 0x7f3acc162d40 2024-04-15T15:20:21.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:21.111+0000 7f3ac959a640 1 -- 172.21.15.105:0/2942106326 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ac4031890 con 0x7f3acc162d40 2024-04-15T15:20:21.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:21.111+0000 7f3ac959a640 1 -- 172.21.15.105:0/2942106326 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac4031c60 con 0x7f3acc162d40 2024-04-15T15:20:21.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:21.112+0000 7f3ad0893640 1 -- 172.21.15.105:0/2942106326 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3acc162d40 msgr2=0x7f3acc169960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:21.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:21.112+0000 7f3ad0893640 1 --2- 172.21.15.105:0/2942106326 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3acc162d40 0x7f3acc169960 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3ac40097d0 tx=0x7f3ac402b930 comp rx=0 tx=0).stop 2024-04-15T15:20:21.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:21.112+0000 7f3ad0893640 1 -- 172.21.15.105:0/2942106326 shutdown_connections 2024-04-15T15:20:21.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:21.112+0000 7f3ad0893640 1 --2- 172.21.15.105:0/2942106326 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3acc162d40 0x7f3acc169960 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:21.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:21.112+0000 7f3ad0893640 1 -- 172.21.15.105:0/2942106326 >> 172.21.15.105:0/2942106326 conn(0x7f3acc067720 msgr2=0x7f3acc067b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:21.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:21.112+0000 7f3ad0893640 1 -- 172.21.15.105:0/2942106326 shutdown_connections 2024-04-15T15:20:21.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:21.112+0000 7f3ad0893640 1 -- 172.21.15.105:0/2942106326 wait complete. 2024-04-15T15:20:21.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:21.113+0000 7f3ad0893640 1 Processor -- start 2024-04-15T15:20:21.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:21.113+0000 7f3ad0893640 1 -- start start 2024-04-15T15:20:21.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:21.114+0000 7f3ad0893640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3acc0d18e0 0x7f3acc161ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:21.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:21.114+0000 7f3ad0893640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3acc15eb60 0x7f3acc152050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:21.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:21.114+0000 7f3ad0893640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3acc162d40 0x7f3acc0b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:21.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:21.114+0000 7f3ad0893640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3acc0b78f0 con 0x7f3acc15eb60 2024-04-15T15:20:21.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:21.114+0000 7f3ad0893640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3acc0d2fc0 con 0x7f3acc162d40 2024-04-15T15:20:21.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:21.114+0000 7f3ad0893640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f3acc0d4460 con 0x7f3acc0d18e0 2024-04-15T15:20:21.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:21.114+0000 7f3ac9d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3acc15eb60 0x7f3acc152050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:21.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:21.114+0000 7f3ac9d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3acc15eb60 0x7f3acc152050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54692/0 (socket says 172.21.15.105:54692) 2024-04-15T15:20:21.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:21.114+0000 7f3ac9d9b640 1 -- 172.21.15.105:0/691514640 learned_addr learned my addr 172.21.15.105:0/691514640 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:21.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:21.114+0000 7f3aca59c640 1 --2- 172.21.15.105:0/691514640 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3acc0d18e0 0x7f3acc161ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:21.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:21.114+0000 7f3acad9d640 1 --2- 172.21.15.105:0/691514640 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3acc162d40 0x7f3acc0b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:21.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:21.114+0000 7f3ac9d9b640 1 -- 172.21.15.105:0/691514640 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3acc0d18e0 msgr2=0x7f3acc161ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:21.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:21.114+0000 7f3ac9d9b640 1 --2- 172.21.15.105:0/691514640 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3acc0d18e0 0x7f3acc161ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:21.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:21.114+0000 7f3ac9d9b640 1 -- 172.21.15.105:0/691514640 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3acc162d40 msgr2=0x7f3acc0b4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:21.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:21.114+0000 7f3ac9d9b640 1 --2- 172.21.15.105:0/691514640 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3acc162d40 0x7f3acc0b4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:21.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:21.114+0000 7f3ac9d9b640 1 -- 172.21.15.105:0/691514640 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3acc0b36b0 con 0x7f3acc15eb60 2024-04-15T15:20:21.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:21.115+0000 7f3ac9d9b640 1 --2- 172.21.15.105:0/691514640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3acc15eb60 0x7f3acc152050 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3ac000c4b0 tx=0x7f3ac000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:21.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:21.115+0000 7f3abaffd640 1 -- 172.21.15.105:0/691514640 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac0017070 con 0x7f3acc15eb60 2024-04-15T15:20:21.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:21.115+0000 7f3abaffd640 1 -- 172.21.15.105:0/691514640 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ac0005480 con 0x7f3acc15eb60 2024-04-15T15:20:21.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:21.115+0000 7f3ad0893640 1 -- 172.21.15.105:0/691514640 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3acc0b3990 con 0x7f3acc15eb60 2024-04-15T15:20:21.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:21.115+0000 7f3abaffd640 1 -- 172.21.15.105:0/691514640 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac0003830 con 0x7f3acc15eb60 2024-04-15T15:20:21.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:21.115+0000 7f3ad0893640 1 -- 172.21.15.105:0/691514640 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3acc0b3f50 con 0x7f3acc15eb60 2024-04-15T15:20:21.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:21.116+0000 7f3ad0893640 1 -- 172.21.15.105:0/691514640 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3acc068a40 con 0x7f3acc15eb60 2024-04-15T15:20:21.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:21.116+0000 7f3abaffd640 1 -- 172.21.15.105:0/691514640 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3ac001a070 con 0x7f3acc15eb60 2024-04-15T15:20:21.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:21.117+0000 7f3abaffd640 1 --2- 172.21.15.105:0/691514640 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f3aac06c8f0 0x7f3aac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:21.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:21.117+0000 7f3abaffd640 1 -- 172.21.15.105:0/691514640 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4943+0+0 (secure 0 0 0) 0x7f3ac008abb0 con 0x7f3acc15eb60 2024-04-15T15:20:21.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:21.117+0000 7f3aca59c640 1 --2- 172.21.15.105:0/691514640 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f3aac06c8f0 0x7f3aac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:21.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:21.117+0000 7f3aca59c640 1 --2- 172.21.15.105:0/691514640 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f3aac06c8f0 0x7f3aac06edb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3abc005c00 tx=0x7f3abc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:21.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:21.121+0000 7f3abaffd640 1 -- 172.21.15.105:0/691514640 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3ac0058ca0 con 0x7f3acc15eb60 2024-04-15T15:20:21.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:21.307+0000 7f3ad0893640 1 -- 172.21.15.105:0/691514640 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f3acc0d23f0 con 0x7f3acc15eb60 2024-04-15T15:20:22.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:22.982+0000 7f3abaffd640 1 -- 172.21.15.105:0/691514640 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v30) ==== 97+0+0 (secure 0 0 0) 0x7f3ac005cfa0 con 0x7f3acc15eb60 2024-04-15T15:20:22.984 INFO:teuthology.orchestra.run.smithi105.stderr: removed all blocklist entries 2024-04-15T15:20:22.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:22.985+0000 7f3ad0893640 1 -- 172.21.15.105:0/691514640 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f3aac06c8f0 msgr2=0x7f3aac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:22.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:22.985+0000 7f3ad0893640 1 --2- 172.21.15.105:0/691514640 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f3aac06c8f0 0x7f3aac06edb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3abc005c00 tx=0x7f3abc009040 comp rx=0 tx=0).stop 2024-04-15T15:20:22.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:22.985+0000 7f3ad0893640 1 -- 172.21.15.105:0/691514640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3acc15eb60 msgr2=0x7f3acc152050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:22.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:22.985+0000 7f3ad0893640 1 --2- 172.21.15.105:0/691514640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3acc15eb60 0x7f3acc152050 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3ac000c4b0 tx=0x7f3ac000c970 comp rx=0 tx=0).stop 2024-04-15T15:20:22.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:22.985+0000 7f3ad0893640 1 -- 172.21.15.105:0/691514640 shutdown_connections 2024-04-15T15:20:22.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:22.985+0000 7f3ad0893640 1 --2- 172.21.15.105:0/691514640 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f3aac06c8f0 0x7f3aac06edb0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:22.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:22.985+0000 7f3ad0893640 1 --2- 172.21.15.105:0/691514640 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3acc162d40 0x7f3acc0b4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:22.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:22.985+0000 7f3ad0893640 1 --2- 172.21.15.105:0/691514640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3acc15eb60 0x7f3acc152050 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:22.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:22.985+0000 7f3ad0893640 1 --2- 172.21.15.105:0/691514640 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3acc0d18e0 0x7f3acc161ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:22.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:22.985+0000 7f3ad0893640 1 -- 172.21.15.105:0/691514640 >> 172.21.15.105:0/691514640 conn(0x7f3acc067720 msgr2=0x7f3acc0c8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:22.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:22.985+0000 7f3ad0893640 1 -- 172.21.15.105:0/691514640 shutdown_connections 2024-04-15T15:20:22.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:22.985+0000 7f3ad0893640 1 -- 172.21.15.105:0/691514640 wait complete. 2024-04-15T15:20:23.000 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-15T15:20:23.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.106+0000 7f32dc6a5640 1 Processor -- start 2024-04-15T15:20:23.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.106+0000 7f32dc6a5640 1 -- start start 2024-04-15T15:20:23.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.107+0000 7f32dc6a5640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f32d40d1420 0x7f32d40b7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:23.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.107+0000 7f32dc6a5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32d40afc70 0x7f32d40b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:23.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.107+0000 7f32dc6a5640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32d4161260 con 0x7f32d4160d10 2024-04-15T15:20:23.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.107+0000 7f32dc6a5640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f32d41462f0 con 0x7f32d40d1420 2024-04-15T15:20:23.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.107+0000 7f32dc6a5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f32d415ceb0 con 0x7f32d40afc70 2024-04-15T15:20:23.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.108+0000 7f32da441640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f32d40d1420 0x7f32d40b7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:23.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.108+0000 7f32da441640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f32d40d1420 0x7f32d40b7070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54698/0 (socket says 172.21.15.105:54698) 2024-04-15T15:20:23.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.108+0000 7f32da441640 1 -- 172.21.15.105:0/2234161872 learned_addr learned my addr 172.21.15.105:0/2234161872 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:23.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.108+0000 7f32dac42640 1 --2- 172.21.15.105:0/2234161872 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32d40afc70 0x7f32d40b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:23.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.108+0000 7f32da441640 1 -- 172.21.15.105:0/2234161872 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32d40afc70 msgr2=0x7f32d40b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:23.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.108+0000 7f32da441640 1 --2- 172.21.15.105:0/2234161872 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32d40afc70 0x7f32d40b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:23.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.108+0000 7f32da441640 1 -- 172.21.15.105:0/2234161872 >> v1:172.21.15.154:6789/0 conn(0x7f32d4160d10 legacy=0x7f32d40d1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:23.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.108+0000 7f32da441640 1 -- 172.21.15.105:0/2234161872 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32d415d150 con 0x7f32d40d1420 2024-04-15T15:20:23.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.109+0000 7f32da441640 1 --2- 172.21.15.105:0/2234161872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32d40d1420 0x7f32d40b7070 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f32c4009e50 tx=0x7f32c402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efbf0f8e923432bf server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:23.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.109+0000 7f32d943f640 1 -- 172.21.15.105:0/2234161872 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c402cb00 con 0x7f32d40d1420 2024-04-15T15:20:23.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.109+0000 7f32d943f640 1 -- 172.21.15.105:0/2234161872 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32c402cca0 con 0x7f32d40d1420 2024-04-15T15:20:23.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.109+0000 7f32d943f640 1 -- 172.21.15.105:0/2234161872 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c4034610 con 0x7f32d40d1420 2024-04-15T15:20:23.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.109+0000 7f32dc6a5640 1 -- 172.21.15.105:0/2234161872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32d40d1420 msgr2=0x7f32d40b7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:23.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.109+0000 7f32dc6a5640 1 --2- 172.21.15.105:0/2234161872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32d40d1420 0x7f32d40b7070 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f32c4009e50 tx=0x7f32c402bf60 comp rx=0 tx=0).stop 2024-04-15T15:20:23.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.109+0000 7f32dc6a5640 1 -- 172.21.15.105:0/2234161872 shutdown_connections 2024-04-15T15:20:23.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.109+0000 7f32dc6a5640 1 --2- 172.21.15.105:0/2234161872 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32d40afc70 0x7f32d40b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:23.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.109+0000 7f32dc6a5640 1 --2- 172.21.15.105:0/2234161872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32d40d1420 0x7f32d40b7070 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:23.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.109+0000 7f32dc6a5640 1 -- 172.21.15.105:0/2234161872 >> 172.21.15.105:0/2234161872 conn(0x7f32d409aa40 msgr2=0x7f32d40b6960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:23.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.109+0000 7f32dc6a5640 1 -- 172.21.15.105:0/2234161872 shutdown_connections 2024-04-15T15:20:23.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.110+0000 7f32dc6a5640 1 -- 172.21.15.105:0/2234161872 wait complete. 2024-04-15T15:20:23.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.111+0000 7f32dc6a5640 1 Processor -- start 2024-04-15T15:20:23.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.111+0000 7f32dc6a5640 1 -- start start 2024-04-15T15:20:23.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.111+0000 7f32dc6a5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32d40afc70 0x7f32d40ce310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:23.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.111+0000 7f32dc6a5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32d40d1420 0x7f32d40cd880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:23.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.111+0000 7f32dc6a5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32d4160d10 0x7f32d413bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:23.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.111+0000 7f32dc6a5640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f32d415dc30 con 0x7f32d40afc70 2024-04-15T15:20:23.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.111+0000 7f32dc6a5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f32d415d710 con 0x7f32d40d1420 2024-04-15T15:20:23.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.111+0000 7f32dc6a5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f32d40ae6c0 con 0x7f32d4160d10 2024-04-15T15:20:23.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.111+0000 7f32da441640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32d40afc70 0x7f32d40ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:23.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.111+0000 7f32da441640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32d40afc70 0x7f32d40ce310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54700/0 (socket says 172.21.15.105:54700) 2024-04-15T15:20:23.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.112+0000 7f32da441640 1 -- 172.21.15.105:0/1802128190 learned_addr learned my addr 172.21.15.105:0/1802128190 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:23.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.112+0000 7f32dac42640 1 --2- 172.21.15.105:0/1802128190 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32d4160d10 0x7f32d413bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:23.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.112+0000 7f32d9c40640 1 --2- 172.21.15.105:0/1802128190 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32d40d1420 0x7f32d40cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:23.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.112+0000 7f32da441640 1 -- 172.21.15.105:0/1802128190 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32d4160d10 msgr2=0x7f32d413bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:23.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.112+0000 7f32da441640 1 --2- 172.21.15.105:0/1802128190 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32d4160d10 0x7f32d413bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:23.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.112+0000 7f32da441640 1 -- 172.21.15.105:0/1802128190 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32d40d1420 msgr2=0x7f32d40cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:23.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.112+0000 7f32da441640 1 --2- 172.21.15.105:0/1802128190 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32d40d1420 0x7f32d40cd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:23.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.112+0000 7f32da441640 1 -- 172.21.15.105:0/1802128190 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32d40ce980 con 0x7f32d40afc70 2024-04-15T15:20:23.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.112+0000 7f32da441640 1 --2- 172.21.15.105:0/1802128190 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32d40afc70 0x7f32d40ce310 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f32c4000f80 tx=0x7f32c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:23.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.112+0000 7f32d2ffd640 1 -- 172.21.15.105:0/1802128190 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c4043020 con 0x7f32d40afc70 2024-04-15T15:20:23.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.112+0000 7f32d2ffd640 1 -- 172.21.15.105:0/1802128190 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32c402ce50 con 0x7f32d40afc70 2024-04-15T15:20:23.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.112+0000 7f32d2ffd640 1 -- 172.21.15.105:0/1802128190 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c4009510 con 0x7f32d40afc70 2024-04-15T15:20:23.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.112+0000 7f32dc6a5640 1 -- 172.21.15.105:0/1802128190 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32d40cec60 con 0x7f32d40afc70 2024-04-15T15:20:23.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.112+0000 7f32dc6a5640 1 -- 172.21.15.105:0/1802128190 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32d4149cd0 con 0x7f32d40afc70 2024-04-15T15:20:23.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.113+0000 7f32dc6a5640 1 -- 172.21.15.105:0/1802128190 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32d4069150 con 0x7f32d40afc70 2024-04-15T15:20:23.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.114+0000 7f32d2ffd640 1 -- 172.21.15.105:0/1802128190 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f32c4046020 con 0x7f32d40afc70 2024-04-15T15:20:23.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.114+0000 7f32d2ffd640 1 --2- 172.21.15.105:0/1802128190 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f32c006c920 0x7f32c006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:23.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.114+0000 7f32d2ffd640 1 -- 172.21.15.105:0/1802128190 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f32c40adc20 con 0x7f32d40afc70 2024-04-15T15:20:23.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.114+0000 7f32d9c40640 1 --2- 172.21.15.105:0/1802128190 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f32c006c920 0x7f32c006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:23.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.115+0000 7f32d9c40640 1 --2- 172.21.15.105:0/1802128190 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f32c006c920 0x7f32c006ede0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f32d41f6050 tx=0x7f32c8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:23.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.119+0000 7f32d2ffd640 1 -- 172.21.15.105:0/1802128190 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f32c404d030 con 0x7f32d40afc70 2024-04-15T15:20:23.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.360+0000 7f32dc6a5640 1 -- 172.21.15.105:0/1802128190 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f32d409af20 con 0x7f32d40afc70 2024-04-15T15:20:23.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.360+0000 7f32d2ffd640 1 -- 172.21.15.105:0/1802128190 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) ==== 80+0+6645 (secure 0 0 0) 0x7f32c407bd60 con 0x7f32d40afc70 2024-04-15T15:20:23.362 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:20:23.363 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-04-15T15:20:23.363 INFO:teuthology.orchestra.run.smithi105.stdout: "auth_dump": [ 2024-04-15T15:20:23.363 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-04-15T15:20:23.363 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mds.a", 2024-04-15T15:20:23.363 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQBDRR1myeBfJxAAvHlKDMW3De4FFrvZ0r0WRw==", 2024-04-15T15:20:23.363 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-04-15T15:20:23.363 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2024-04-15T15:20:23.363 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow *", 2024-04-15T15:20:23.363 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow *", 2024-04-15T15:20:23.363 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2024-04-15T15:20:23.363 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-04-15T15:20:23.363 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-04-15T15:20:23.363 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-04-15T15:20:23.364 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mds.b", 2024-04-15T15:20:23.364 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQBDRR1mTM5dNBAA36E7C2eeFskEfxTIBTwRJw==", 2024-04-15T15:20:23.364 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-04-15T15:20:23.364 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2024-04-15T15:20:23.364 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow *", 2024-04-15T15:20:23.364 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow *", 2024-04-15T15:20:23.364 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2024-04-15T15:20:23.364 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-04-15T15:20:23.364 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-04-15T15:20:23.364 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-04-15T15:20:23.364 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mds.c", 2024-04-15T15:20:23.365 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQBDRR1m8HsELxAAe4IxWY/OGr9Jr4n8k+JhvQ==", 2024-04-15T15:20:23.365 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-04-15T15:20:23.365 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2024-04-15T15:20:23.365 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow *", 2024-04-15T15:20:23.365 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow *", 2024-04-15T15:20:23.365 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2024-04-15T15:20:23.365 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-04-15T15:20:23.365 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-04-15T15:20:23.365 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-04-15T15:20:23.365 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mds.d", 2024-04-15T15:20:23.365 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQBERR1m2op6ABAAVnXL6UQjmdYDgaihrxNqjQ==", 2024-04-15T15:20:23.365 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-04-15T15:20:23.365 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2024-04-15T15:20:23.366 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow *", 2024-04-15T15:20:23.366 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow *", 2024-04-15T15:20:23.366 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2024-04-15T15:20:23.366 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-04-15T15:20:23.366 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-04-15T15:20:23.366 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-04-15T15:20:23.366 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.0", 2024-04-15T15:20:23.366 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQBHRR1mT7whIBAABC1sgtSsUmjcvEyhTT5oPA==", 2024-04-15T15:20:23.366 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-04-15T15:20:23.366 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2024-04-15T15:20:23.366 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2024-04-15T15:20:23.366 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2024-04-15T15:20:23.367 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-04-15T15:20:23.367 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-04-15T15:20:23.367 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-04-15T15:20:23.367 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.1", 2024-04-15T15:20:23.367 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQBKRR1mNulAHxAAMfDgB2AsfxPimbWOcJATHA==", 2024-04-15T15:20:23.367 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-04-15T15:20:23.367 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2024-04-15T15:20:23.367 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2024-04-15T15:20:23.367 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2024-04-15T15:20:23.367 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-04-15T15:20:23.367 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-04-15T15:20:23.367 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-04-15T15:20:23.367 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.2", 2024-04-15T15:20:23.367 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQBNRR1mQML+HRAAkQe3VRgWiylsdC5EZrDE5Q==", 2024-04-15T15:20:23.367 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-04-15T15:20:23.367 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2024-04-15T15:20:23.368 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2024-04-15T15:20:23.368 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2024-04-15T15:20:23.368 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-04-15T15:20:23.368 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-04-15T15:20:23.368 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-04-15T15:20:23.368 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.3", 2024-04-15T15:20:23.368 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQBQRR1mxLduHBAAaqVeV0mMeqXIrNyKDws4iA==", 2024-04-15T15:20:23.368 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-04-15T15:20:23.368 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2024-04-15T15:20:23.368 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2024-04-15T15:20:23.368 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2024-04-15T15:20:23.368 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-04-15T15:20:23.368 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-04-15T15:20:23.369 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-04-15T15:20:23.369 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.4", 2024-04-15T15:20:23.369 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQBWRR1m2gqGJRAApBMj7eVuqtTUDJBgl4nGag==", 2024-04-15T15:20:23.369 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-04-15T15:20:23.369 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2024-04-15T15:20:23.369 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2024-04-15T15:20:23.369 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2024-04-15T15:20:23.369 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-04-15T15:20:23.369 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-04-15T15:20:23.369 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-04-15T15:20:23.369 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.5", 2024-04-15T15:20:23.370 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQBZRR1mcGR9JBAArgDqBSvm3q6wF28o+DzCRg==", 2024-04-15T15:20:23.370 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-04-15T15:20:23.370 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2024-04-15T15:20:23.370 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2024-04-15T15:20:23.370 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2024-04-15T15:20:23.370 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-04-15T15:20:23.370 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-04-15T15:20:23.370 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-04-15T15:20:23.370 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.6", 2024-04-15T15:20:23.370 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQBcRR1mOYfWIhAA+CwyZLGcoScCZXdAYih9xw==", 2024-04-15T15:20:23.370 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-04-15T15:20:23.370 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2024-04-15T15:20:23.371 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2024-04-15T15:20:23.371 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2024-04-15T15:20:23.371 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-04-15T15:20:23.371 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-04-15T15:20:23.371 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-04-15T15:20:23.371 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.7", 2024-04-15T15:20:23.371 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQBfRR1mwdeIIRAAwPA5PsCibACLK3Eydl1YlA==", 2024-04-15T15:20:23.371 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-04-15T15:20:23.371 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2024-04-15T15:20:23.371 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2024-04-15T15:20:23.371 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2024-04-15T15:20:23.371 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-04-15T15:20:23.371 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-04-15T15:20:23.372 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-04-15T15:20:23.372 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.0", 2024-04-15T15:20:23.372 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQBERR1mNnrzBBAAg6vNvYozUQe3sI+Xxt0QvA==", 2024-04-15T15:20:23.372 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-04-15T15:20:23.372 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2024-04-15T15:20:23.372 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow r", 2024-04-15T15:20:23.372 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow rw", 2024-04-15T15:20:23.372 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow rwx" 2024-04-15T15:20:23.372 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-04-15T15:20:23.372 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-04-15T15:20:23.372 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-04-15T15:20:23.372 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.1", 2024-04-15T15:20:23.372 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQBERR1mvcwqDBAAPw/HCrSX2PdKn2owrVjJXQ==", 2024-04-15T15:20:23.372 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-04-15T15:20:23.373 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2024-04-15T15:20:23.373 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow r", 2024-04-15T15:20:23.373 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow rw", 2024-04-15T15:20:23.373 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow rwx" 2024-04-15T15:20:23.373 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-04-15T15:20:23.373 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-04-15T15:20:23.373 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-04-15T15:20:23.373 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.2", 2024-04-15T15:20:23.373 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQBERR1mUf2WCBAACYRnwP9EJeGOTdRVDHIjhA==", 2024-04-15T15:20:23.373 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-04-15T15:20:23.373 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2024-04-15T15:20:23.373 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow r", 2024-04-15T15:20:23.373 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow rw", 2024-04-15T15:20:23.373 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow rwx" 2024-04-15T15:20:23.374 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-04-15T15:20:23.374 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-04-15T15:20:23.374 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-04-15T15:20:23.374 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.3", 2024-04-15T15:20:23.374 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQBERR1mBV/EDxAAeGJ4NqcoKlf7sJMH9klH/w==", 2024-04-15T15:20:23.374 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-04-15T15:20:23.374 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2024-04-15T15:20:23.374 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow r", 2024-04-15T15:20:23.374 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow rw", 2024-04-15T15:20:23.374 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow rwx" 2024-04-15T15:20:23.374 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-04-15T15:20:23.375 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-04-15T15:20:23.375 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-04-15T15:20:23.375 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.admin", 2024-04-15T15:20:23.375 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQBDRR1m2OHLCRAA7Pkn6O+He5QbyMscQlUvYA==", 2024-04-15T15:20:23.375 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-04-15T15:20:23.375 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow *", 2024-04-15T15:20:23.375 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow *", 2024-04-15T15:20:23.375 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow *", 2024-04-15T15:20:23.376 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2024-04-15T15:20:23.376 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-04-15T15:20:23.376 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-04-15T15:20:23.376 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-04-15T15:20:23.376 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.bootstrap-mds", 2024-04-15T15:20:23.376 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQBlRR1mYItKHxAA3qKGovRIMUm644zRDEln5g==", 2024-04-15T15:20:23.376 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-04-15T15:20:23.377 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile bootstrap-mds" 2024-04-15T15:20:23.377 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-04-15T15:20:23.377 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-04-15T15:20:23.377 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-04-15T15:20:23.377 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.bootstrap-mgr", 2024-04-15T15:20:23.377 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQBlRR1md5NKHxAAJuLk7exAiuJr91V4X6yp3g==", 2024-04-15T15:20:23.377 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-04-15T15:20:23.377 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-15T15:20:23.378 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-04-15T15:20:23.378 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-04-15T15:20:23.378 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-04-15T15:20:23.378 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.bootstrap-osd", 2024-04-15T15:20:23.378 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQBlRR1mG5pKHxAAkYLWF5kLdIpzBRzpelP6zQ==", 2024-04-15T15:20:23.378 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-04-15T15:20:23.378 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile bootstrap-osd" 2024-04-15T15:20:23.379 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-04-15T15:20:23.379 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-04-15T15:20:23.379 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-04-15T15:20:23.379 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.bootstrap-rbd", 2024-04-15T15:20:23.379 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQBlRR1miKBKHxAAcnObwBeH55Lw8R2lYeH9UA==", 2024-04-15T15:20:23.379 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-04-15T15:20:23.379 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-15T15:20:23.380 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-04-15T15:20:23.380 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-04-15T15:20:23.380 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-04-15T15:20:23.380 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-15T15:20:23.380 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQBlRR1mO6dKHxAAK0GlpZiOpOx0wArqG/7bYg==", 2024-04-15T15:20:23.380 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-04-15T15:20:23.380 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-15T15:20:23.380 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-04-15T15:20:23.380 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-04-15T15:20:23.381 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-04-15T15:20:23.381 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.bootstrap-rgw", 2024-04-15T15:20:23.381 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQBlRR1mGa5KHxAAZNWUob0uXWfxHhWEmEAjvg==", 2024-04-15T15:20:23.381 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-04-15T15:20:23.381 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-15T15:20:23.381 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-04-15T15:20:23.381 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-04-15T15:20:23.381 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-04-15T15:20:23.382 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mgr.x", 2024-04-15T15:20:23.382 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQBDRR1mmkXDIxAAz2Dpih9eXQIHLl94qsCjvQ==", 2024-04-15T15:20:23.382 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-04-15T15:20:23.382 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow *", 2024-04-15T15:20:23.382 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile mgr", 2024-04-15T15:20:23.382 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2024-04-15T15:20:23.382 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-04-15T15:20:23.383 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-04-15T15:20:23.383 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-04-15T15:20:23.383 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mgr.y", 2024-04-15T15:20:23.383 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQBDRR1mNFKxHxAAsjYdZCnA7fcIVzXB9FWxXg==", 2024-04-15T15:20:23.383 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-04-15T15:20:23.383 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow *", 2024-04-15T15:20:23.383 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile mgr", 2024-04-15T15:20:23.383 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2024-04-15T15:20:23.384 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-04-15T15:20:23.384 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-04-15T15:20:23.384 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-04-15T15:20:23.384 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-04-15T15:20:23.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.362+0000 7f32dc6a5640 1 -- 172.21.15.105:0/1802128190 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f32c006c920 msgr2=0x7f32c006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:23.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.362+0000 7f32dc6a5640 1 --2- 172.21.15.105:0/1802128190 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f32c006c920 0x7f32c006ede0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f32d41f6050 tx=0x7f32c8009040 comp rx=0 tx=0).stop 2024-04-15T15:20:23.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.362+0000 7f32dc6a5640 1 -- 172.21.15.105:0/1802128190 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32d40afc70 msgr2=0x7f32d40ce310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:23.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.362+0000 7f32dc6a5640 1 --2- 172.21.15.105:0/1802128190 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32d40afc70 0x7f32d40ce310 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f32c4000f80 tx=0x7f32c4004a40 comp rx=0 tx=0).stop 2024-04-15T15:20:23.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.363+0000 7f32dc6a5640 1 -- 172.21.15.105:0/1802128190 shutdown_connections 2024-04-15T15:20:23.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.363+0000 7f32dc6a5640 1 --2- 172.21.15.105:0/1802128190 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32d4160d10 0x7f32d413bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:23.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.363+0000 7f32dc6a5640 1 --2- 172.21.15.105:0/1802128190 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f32c006c920 0x7f32c006ede0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:23.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.363+0000 7f32dc6a5640 1 --2- 172.21.15.105:0/1802128190 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32d40d1420 0x7f32d40cd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:23.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.363+0000 7f32dc6a5640 1 --2- 172.21.15.105:0/1802128190 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32d40afc70 0x7f32d40ce310 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:23.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.363+0000 7f32dc6a5640 1 -- 172.21.15.105:0/1802128190 >> 172.21.15.105:0/1802128190 conn(0x7f32d409aa40 msgr2=0x7f32d41424d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:23.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.363+0000 7f32dc6a5640 1 -- 172.21.15.105:0/1802128190 shutdown_connections 2024-04-15T15:20:23.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.363+0000 7f32dc6a5640 1 -- 172.21.15.105:0/1802128190 wait complete. 2024-04-15T15:20:23.389 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-15T15:20:23.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.491+0000 7f7a4f57c640 1 Processor -- start 2024-04-15T15:20:23.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.491+0000 7f7a4f57c640 1 -- start start 2024-04-15T15:20:23.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.492+0000 7f7a4f57c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a4815ecf0 0x7f7a4806e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:23.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.492+0000 7f7a4f57c640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a4806d2a0 con 0x7f7a480cd360 2024-04-15T15:20:23.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.492+0000 7f7a4f57c640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a4806d580 con 0x7f7a480d0f10 2024-04-15T15:20:23.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.492+0000 7f7a4f57c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f7a4806d860 con 0x7f7a4815ecf0 2024-04-15T15:20:23.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.492+0000 7f7a4db19640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f7a480cd360 0x7f7a48162960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:44332/0 (socket says 172.21.15.105:44332) 2024-04-15T15:20:23.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.492+0000 7f7a4db19640 1 -- 172.21.15.105:0/354365796 learned_addr learned my addr 172.21.15.105:0/354365796 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:20:23.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.492+0000 7f7a4d318640 1 --2- 172.21.15.105:0/354365796 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a4815ecf0 0x7f7a4806e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:23.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.492+0000 7f7a3ffff640 1 -- 172.21.15.105:0/354365796 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 169274701 0 0) 0x7f7a4806d2a0 con 0x7f7a480cd360 2024-04-15T15:20:23.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.493+0000 7f7a3ffff640 1 -- 172.21.15.105:0/354365796 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a34003520 con 0x7f7a480cd360 2024-04-15T15:20:23.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.493+0000 7f7a3ffff640 1 -- 172.21.15.105:0/354365796 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3020492670 0 0) 0x7f7a4806d580 con 0x7f7a480d0f10 2024-04-15T15:20:23.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.493+0000 7f7a3ffff640 1 -- 172.21.15.105:0/354365796 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a4806d2a0 con 0x7f7a480d0f10 2024-04-15T15:20:23.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.493+0000 7f7a3ffff640 1 -- 172.21.15.105:0/354365796 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3191345187 0 0) 0x7f7a34003520 con 0x7f7a480cd360 2024-04-15T15:20:23.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.493+0000 7f7a4d318640 1 -- 172.21.15.105:0/354365796 >> v1:172.21.15.154:6789/0 conn(0x7f7a480d0f10 legacy=0x7f7a480ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:23.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.493+0000 7f7a4d318640 1 -- 172.21.15.105:0/354365796 >> v1:172.21.15.105:6789/0 conn(0x7f7a480cd360 legacy=0x7f7a48162960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:23.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.493+0000 7f7a4d318640 1 -- 172.21.15.105:0/354365796 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a4806db40 con 0x7f7a4815ecf0 2024-04-15T15:20:23.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.494+0000 7f7a4d318640 1 --2- 172.21.15.105:0/354365796 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a4815ecf0 0x7f7a4806e060 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7a44009dc0 tx=0x7f7a4402bde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ddc4ede3e0bbcb server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:23.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.494+0000 7f7a3ffff640 1 -- 172.21.15.105:0/354365796 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a44039070 con 0x7f7a4815ecf0 2024-04-15T15:20:23.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.494+0000 7f7a3ffff640 1 -- 172.21.15.105:0/354365796 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a4402c910 con 0x7f7a4815ecf0 2024-04-15T15:20:23.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.494+0000 7f7a3ffff640 1 -- 172.21.15.105:0/354365796 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a400015f0 con 0x7f7a4815ecf0 2024-04-15T15:20:23.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.494+0000 7f7a4f57c640 1 -- 172.21.15.105:0/354365796 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a4815ecf0 msgr2=0x7f7a4806e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:23.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.494+0000 7f7a4f57c640 1 --2- 172.21.15.105:0/354365796 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a4815ecf0 0x7f7a4806e060 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7a44009dc0 tx=0x7f7a4402bde0 comp rx=0 tx=0).stop 2024-04-15T15:20:23.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.494+0000 7f7a4f57c640 1 -- 172.21.15.105:0/354365796 shutdown_connections 2024-04-15T15:20:23.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.494+0000 7f7a4f57c640 1 --2- 172.21.15.105:0/354365796 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a4815ecf0 0x7f7a4806e060 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:23.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.494+0000 7f7a4f57c640 1 -- 172.21.15.105:0/354365796 >> 172.21.15.105:0/354365796 conn(0x7f7a480d2600 msgr2=0x7f7a480d2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:23.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.494+0000 7f7a4f57c640 1 -- 172.21.15.105:0/354365796 shutdown_connections 2024-04-15T15:20:23.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.494+0000 7f7a4f57c640 1 -- 172.21.15.105:0/354365796 wait complete. 2024-04-15T15:20:23.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.495+0000 7f7a4f57c640 1 Processor -- start 2024-04-15T15:20:23.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.496+0000 7f7a4f57c640 1 -- start start 2024-04-15T15:20:23.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.496+0000 7f7a4f57c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a480cd360 0x7f7a481f7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:23.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.496+0000 7f7a4f57c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a480d0f10 0x7f7a480cbe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:23.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.496+0000 7f7a4f57c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a4815ecf0 0x7f7a480ca910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:23.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.496+0000 7f7a4f57c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4815f0d0 con 0x7f7a480d0f10 2024-04-15T15:20:23.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.496+0000 7f7a4f57c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7a480d1370 con 0x7f7a4815ecf0 2024-04-15T15:20:23.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.496+0000 7f7a4f57c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f7a4815f250 con 0x7f7a480cd360 2024-04-15T15:20:23.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.496+0000 7f7a4cb17640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a480d0f10 0x7f7a480cbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:23.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.496+0000 7f7a4cb17640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a480d0f10 0x7f7a480cbe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54716/0 (socket says 172.21.15.105:54716) 2024-04-15T15:20:23.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.496+0000 7f7a4cb17640 1 -- 172.21.15.105:0/3615187377 learned_addr learned my addr 172.21.15.105:0/3615187377 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:23.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.497+0000 7f7a4db19640 1 --2- 172.21.15.105:0/3615187377 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a4815ecf0 0x7f7a480ca910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:23.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.497+0000 7f7a4cb17640 1 -- 172.21.15.105:0/3615187377 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a480cd360 msgr2=0x7f7a481f7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:23.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.497+0000 7f7a4cb17640 1 --2- 172.21.15.105:0/3615187377 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a480cd360 0x7f7a481f7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:23.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.497+0000 7f7a4cb17640 1 -- 172.21.15.105:0/3615187377 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a4815ecf0 msgr2=0x7f7a480ca910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:23.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.497+0000 7f7a4cb17640 1 --2- 172.21.15.105:0/3615187377 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a4815ecf0 0x7f7a480ca910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:23.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.497+0000 7f7a4cb17640 1 -- 172.21.15.105:0/3615187377 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a480c7ca0 con 0x7f7a480d0f10 2024-04-15T15:20:23.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.497+0000 7f7a4cb17640 1 --2- 172.21.15.105:0/3615187377 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a480d0f10 0x7f7a480cbe60 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7a380010e0 tx=0x7f7a3800c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:23.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.497+0000 7f7a3dffb640 1 -- 172.21.15.105:0/3615187377 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a380052d0 con 0x7f7a480d0f10 2024-04-15T15:20:23.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.497+0000 7f7a3dffb640 1 -- 172.21.15.105:0/3615187377 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a38005490 con 0x7f7a480d0f10 2024-04-15T15:20:23.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.497+0000 7f7a3dffb640 1 -- 172.21.15.105:0/3615187377 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a38012870 con 0x7f7a480d0f10 2024-04-15T15:20:23.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.497+0000 7f7a4f57c640 1 -- 172.21.15.105:0/3615187377 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a480c7f80 con 0x7f7a480d0f10 2024-04-15T15:20:23.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.497+0000 7f7a4f57c640 1 -- 172.21.15.105:0/3615187377 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a480c8820 con 0x7f7a480d0f10 2024-04-15T15:20:23.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.498+0000 7f7a4f57c640 1 -- 172.21.15.105:0/3615187377 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a480c8150 con 0x7f7a480d0f10 2024-04-15T15:20:23.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.499+0000 7f7a3dffb640 1 -- 172.21.15.105:0/3615187377 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7a38012a10 con 0x7f7a480d0f10 2024-04-15T15:20:23.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.499+0000 7f7a3dffb640 1 --2- 172.21.15.105:0/3615187377 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f7a2c06c8f0 0x7f7a2c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:23.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.499+0000 7f7a3dffb640 1 -- 172.21.15.105:0/3615187377 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f7a3808b4f0 con 0x7f7a480d0f10 2024-04-15T15:20:23.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.499+0000 7f7a4d318640 1 --2- 172.21.15.105:0/3615187377 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f7a2c06c8f0 0x7f7a2c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:23.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.500+0000 7f7a4d318640 1 --2- 172.21.15.105:0/3615187377 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f7a2c06c8f0 0x7f7a2c06edb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7a44036000 tx=0x7f7a4402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:23.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.504+0000 7f7a3dffb640 1 -- 172.21.15.105:0/3615187377 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7a38059d00 con 0x7f7a480d0f10 2024-04-15T15:20:23.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.745+0000 7f7a4f57c640 1 -- 172.21.15.105:0/3615187377 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f7a480c8a20 con 0x7f7a480d0f10 2024-04-15T15:20:23.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.747+0000 7f7a3dffb640 1 -- 172.21.15.105:0/3615187377 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7f7a480c8a20 con 0x7f7a480d0f10 2024-04-15T15:20:23.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.749+0000 7f7a4f57c640 1 -- 172.21.15.105:0/3615187377 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f7a2c06c8f0 msgr2=0x7f7a2c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:23.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.749+0000 7f7a4f57c640 1 --2- 172.21.15.105:0/3615187377 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f7a2c06c8f0 0x7f7a2c06edb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7a44036000 tx=0x7f7a4402e000 comp rx=0 tx=0).stop 2024-04-15T15:20:23.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.749+0000 7f7a4f57c640 1 -- 172.21.15.105:0/3615187377 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a480d0f10 msgr2=0x7f7a480cbe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:23.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.749+0000 7f7a4f57c640 1 --2- 172.21.15.105:0/3615187377 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a480d0f10 0x7f7a480cbe60 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7a380010e0 tx=0x7f7a3800c980 comp rx=0 tx=0).stop 2024-04-15T15:20:23.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.749+0000 7f7a4f57c640 1 -- 172.21.15.105:0/3615187377 shutdown_connections 2024-04-15T15:20:23.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.749+0000 7f7a4f57c640 1 --2- 172.21.15.105:0/3615187377 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f7a2c06c8f0 0x7f7a2c06edb0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:23.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.749+0000 7f7a4f57c640 1 --2- 172.21.15.105:0/3615187377 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a4815ecf0 0x7f7a480ca910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:23.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.749+0000 7f7a4f57c640 1 --2- 172.21.15.105:0/3615187377 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a480d0f10 0x7f7a480cbe60 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:23.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.749+0000 7f7a4f57c640 1 --2- 172.21.15.105:0/3615187377 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a480cd360 0x7f7a481f7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:23.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.749+0000 7f7a4f57c640 1 -- 172.21.15.105:0/3615187377 >> 172.21.15.105:0/3615187377 conn(0x7f7a480d2600 msgr2=0x7f7a480b7610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:23.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.749+0000 7f7a4f57c640 1 -- 172.21.15.105:0/3615187377 shutdown_connections 2024-04-15T15:20:23.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.749+0000 7f7a4f57c640 1 -- 172.21.15.105:0/3615187377 wait complete. 2024-04-15T15:20:23.761 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-15T15:20:23.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.863+0000 7fe3a3bbd640 1 Processor -- start 2024-04-15T15:20:23.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.863+0000 7fe3a3bbd640 1 -- start start 2024-04-15T15:20:23.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.864+0000 7fe3a3bbd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe39c15eb60 0x7fe39c0d11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:23.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.864+0000 7fe3a3bbd640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe39c15ef40 con 0x7fe39c0d1710 2024-04-15T15:20:23.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.864+0000 7fe3a3bbd640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe39c163260 con 0x7fe39c162d40 2024-04-15T15:20:23.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.864+0000 7fe3a3bbd640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fe39c0d1b60 con 0x7fe39c15eb60 2024-04-15T15:20:23.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.864+0000 7fe3a1158640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fe39c0d1710 0x7fe39c162630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:44342/0 (socket says 172.21.15.105:44342) 2024-04-15T15:20:23.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.864+0000 7fe3a1158640 1 -- 172.21.15.105:0/558272122 learned_addr learned my addr 172.21.15.105:0/558272122 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:20:23.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.865+0000 7fe3a1959640 1 --2- 172.21.15.105:0/558272122 >> v2:172.21.15.154:3300/0 conn(0x7fe39c15eb60 0x7fe39c0d11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:23.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.865+0000 7fe3a0957640 1 -- 172.21.15.105:0/558272122 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2677489611 0 0) 0x7fe39c15ef40 con 0x7fe39c0d1710 2024-04-15T15:20:23.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.865+0000 7fe3a0957640 1 -- 172.21.15.105:0/558272122 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe384003520 con 0x7fe39c0d1710 2024-04-15T15:20:23.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.865+0000 7fe3a0957640 1 -- 172.21.15.105:0/558272122 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 485893135 0 0) 0x7fe384003520 con 0x7fe39c0d1710 2024-04-15T15:20:23.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.865+0000 7fe3a0957640 1 -- 172.21.15.105:0/558272122 >> v2:172.21.15.154:3300/0 conn(0x7fe39c15eb60 msgr2=0x7fe39c0d11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:23.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.865+0000 7fe3a0957640 1 --2- 172.21.15.105:0/558272122 >> v2:172.21.15.154:3300/0 conn(0x7fe39c15eb60 0x7fe39c0d11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:23.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.865+0000 7fe3a0957640 1 -- 172.21.15.105:0/558272122 >> v1:172.21.15.154:6789/0 conn(0x7fe39c162d40 legacy=0x7fe39c0d3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:23.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.865+0000 7fe3a0957640 1 -- 172.21.15.105:0/558272122 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe39c0d2970 con 0x7fe39c0d1710 2024-04-15T15:20:23.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.865+0000 7fe3a0957640 1 -- 172.21.15.105:0/558272122 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7fe390001670 con 0x7fe39c0d1710 2024-04-15T15:20:23.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.865+0000 7fe3a0957640 1 -- 172.21.15.105:0/558272122 >> v1:172.21.15.105:6789/0 conn(0x7fe39c0d1710 legacy=0x7fe39c162630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:23.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.865+0000 7fe3a0957640 1 --2- 172.21.15.105:0/558272122 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3840039b0 0x7fe384003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:23.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.865+0000 7fe3a0957640 1 --2- 172.21.15.105:0/558272122 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe384004800 0x7fe384004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:23.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.865+0000 7fe3a0957640 1 --2- 172.21.15.105:0/558272122 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3840051c0 0x7fe38400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:23.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.865+0000 7fe3a0957640 1 -- 172.21.15.105:0/558272122 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe39c0d1b60 con 0x7fe3840051c0 2024-04-15T15:20:23.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.865+0000 7fe3a0957640 1 -- 172.21.15.105:0/558272122 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe38400c060 con 0x7fe384004800 2024-04-15T15:20:23.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.865+0000 7fe3a0957640 1 -- 172.21.15.105:0/558272122 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fe38400c260 con 0x7fe3840039b0 2024-04-15T15:20:23.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.865+0000 7fe3a215a640 1 --2- 172.21.15.105:0/558272122 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3840051c0 0x7fe38400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:23.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.865+0000 7fe3a1158640 1 --2- 172.21.15.105:0/558272122 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe384004800 0x7fe384004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:23.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.865+0000 7fe3a1959640 1 --2- 172.21.15.105:0/558272122 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3840039b0 0x7fe384003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:23.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.865+0000 7fe3a215a640 1 -- 172.21.15.105:0/558272122 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3840039b0 msgr2=0x7fe384003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:23.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.865+0000 7fe3a215a640 1 --2- 172.21.15.105:0/558272122 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3840039b0 0x7fe384003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:23.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.865+0000 7fe3a215a640 1 -- 172.21.15.105:0/558272122 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe384004800 msgr2=0x7fe384004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:23.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.865+0000 7fe3a215a640 1 --2- 172.21.15.105:0/558272122 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe384004800 0x7fe384004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:23.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.865+0000 7fe3a215a640 1 -- 172.21.15.105:0/558272122 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe384003520 con 0x7fe3840051c0 2024-04-15T15:20:23.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.866+0000 7fe3a215a640 1 -- 172.21.15.105:0/558272122 reap_dead start 2024-04-15T15:20:23.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.866+0000 7fe3a215a640 1 --2- 172.21.15.105:0/558272122 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3840051c0 0x7fe38400b9c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe394001750 tx=0x7fe384005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d9ecc08d25202d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:23.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.866+0000 7fe3a0957640 1 -- 172.21.15.105:0/558272122 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe394001370 con 0x7fe3840051c0 2024-04-15T15:20:23.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.866+0000 7fe3a0957640 1 -- 172.21.15.105:0/558272122 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3940052f0 con 0x7fe3840051c0 2024-04-15T15:20:23.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.866+0000 7fe3a0957640 1 -- 172.21.15.105:0/558272122 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe394003830 con 0x7fe3840051c0 2024-04-15T15:20:23.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.867+0000 7fe3a3bbd640 1 -- 172.21.15.105:0/558272122 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3840051c0 msgr2=0x7fe38400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:23.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.867+0000 7fe3a3bbd640 1 --2- 172.21.15.105:0/558272122 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3840051c0 0x7fe38400b9c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe394001750 tx=0x7fe384005830 comp rx=0 tx=0).stop 2024-04-15T15:20:23.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.867+0000 7fe3a3bbd640 1 -- 172.21.15.105:0/558272122 shutdown_connections 2024-04-15T15:20:23.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.867+0000 7fe3a3bbd640 1 --2- 172.21.15.105:0/558272122 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3840051c0 0x7fe38400b9c0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:23.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.867+0000 7fe3a3bbd640 1 -- 172.21.15.105:0/558272122 >> 172.21.15.105:0/558272122 conn(0x7fe39c067720 msgr2=0x7fe39c067b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:23.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.867+0000 7fe3a3bbd640 1 -- 172.21.15.105:0/558272122 shutdown_connections 2024-04-15T15:20:23.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.867+0000 7fe3a3bbd640 1 -- 172.21.15.105:0/558272122 wait complete. 2024-04-15T15:20:23.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.868+0000 7fe3a3bbd640 1 Processor -- start 2024-04-15T15:20:23.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.868+0000 7fe3a3bbd640 1 -- start start 2024-04-15T15:20:23.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.869+0000 7fe3a3bbd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3840051c0 0x7fe39c14a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:23.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.869+0000 7fe3a3bbd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe39c14af90 0x7fe39c14cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:23.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.869+0000 7fe3a3bbd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe39c14b410 0x7fe39c14d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:23.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.869+0000 7fe3a3bbd640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe39c0d2fc0 con 0x7fe3840051c0 2024-04-15T15:20:23.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.869+0000 7fe3a3bbd640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe39c163540 con 0x7fe39c14af90 2024-04-15T15:20:23.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.869+0000 7fe3a3bbd640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fe39c0b78f0 con 0x7fe39c14b410 2024-04-15T15:20:23.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.869+0000 7fe3a1959640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3840051c0 0x7fe39c14a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:23.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.869+0000 7fe3a1959640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3840051c0 0x7fe39c14a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54734/0 (socket says 172.21.15.105:54734) 2024-04-15T15:20:23.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.869+0000 7fe3a1959640 1 -- 172.21.15.105:0/3947694045 learned_addr learned my addr 172.21.15.105:0/3947694045 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:23.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.869+0000 7fe3a1959640 1 -- 172.21.15.105:0/3947694045 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe39c14b410 msgr2=0x7fe39c14d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:23.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.869+0000 7fe3a1959640 1 --2- 172.21.15.105:0/3947694045 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe39c14b410 0x7fe39c14d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:23.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.869+0000 7fe3a1959640 1 -- 172.21.15.105:0/3947694045 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe39c14af90 msgr2=0x7fe39c14cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:23.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.869+0000 7fe3a1959640 1 --2- 172.21.15.105:0/3947694045 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe39c14af90 0x7fe39c14cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:23.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.869+0000 7fe3a1959640 1 -- 172.21.15.105:0/3947694045 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe39c14acb0 con 0x7fe3840051c0 2024-04-15T15:20:23.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.869+0000 7fe3a1959640 1 --2- 172.21.15.105:0/3947694045 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3840051c0 0x7fe39c14a770 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe38c009fb0 tx=0x7fe38c0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:23.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.869+0000 7fe39a7fc640 1 -- 172.21.15.105:0/3947694045 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe38c01a020 con 0x7fe3840051c0 2024-04-15T15:20:23.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.870+0000 7fe39a7fc640 1 -- 172.21.15.105:0/3947694045 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe38c016470 con 0x7fe3840051c0 2024-04-15T15:20:23.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.870+0000 7fe3a3bbd640 1 -- 172.21.15.105:0/3947694045 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe39c0b5230 con 0x7fe3840051c0 2024-04-15T15:20:23.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.870+0000 7fe39a7fc640 1 -- 172.21.15.105:0/3947694045 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe38c016860 con 0x7fe3840051c0 2024-04-15T15:20:23.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.870+0000 7fe3a3bbd640 1 -- 172.21.15.105:0/3947694045 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe39c0b57f0 con 0x7fe3840051c0 2024-04-15T15:20:23.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.871+0000 7fe3a3bbd640 1 -- 172.21.15.105:0/3947694045 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe39c068a40 con 0x7fe3840051c0 2024-04-15T15:20:23.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.871+0000 7fe39a7fc640 1 -- 172.21.15.105:0/3947694045 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe38c005d10 con 0x7fe3840051c0 2024-04-15T15:20:23.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.871+0000 7fe39a7fc640 1 --2- 172.21.15.105:0/3947694045 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fe38806c8f0 0x7fe38806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:23.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.872+0000 7fe39a7fc640 1 -- 172.21.15.105:0/3947694045 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7fe38c08c750 con 0x7fe3840051c0 2024-04-15T15:20:23.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.872+0000 7fe3a1158640 1 --2- 172.21.15.105:0/3947694045 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fe38806c8f0 0x7fe38806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:23.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.872+0000 7fe3a1158640 1 --2- 172.21.15.105:0/3947694045 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fe38806c8f0 0x7fe38806edb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe390000f60 tx=0x7fe390000ed0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:23.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:23.876+0000 7fe39a7fc640 1 -- 172.21.15.105:0/3947694045 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe38c024030 con 0x7fe3840051c0 2024-04-15T15:20:24.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.120+0000 7fe3a3bbd640 1 -- 172.21.15.105:0/3947694045 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7fe39c0d23f0 con 0x7fe3840051c0 2024-04-15T15:20:24.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.122+0000 7fe39a7fc640 1 -- 172.21.15.105:0/3947694045 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7fe39c14acb0 con 0x7fe3840051c0 2024-04-15T15:20:24.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.124+0000 7fe3a3bbd640 1 -- 172.21.15.105:0/3947694045 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fe38806c8f0 msgr2=0x7fe38806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:24.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.124+0000 7fe3a3bbd640 1 --2- 172.21.15.105:0/3947694045 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fe38806c8f0 0x7fe38806edb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe390000f60 tx=0x7fe390000ed0 comp rx=0 tx=0).stop 2024-04-15T15:20:24.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.124+0000 7fe3a3bbd640 1 -- 172.21.15.105:0/3947694045 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3840051c0 msgr2=0x7fe39c14a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:24.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.124+0000 7fe3a3bbd640 1 --2- 172.21.15.105:0/3947694045 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3840051c0 0x7fe39c14a770 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe38c009fb0 tx=0x7fe38c0115a0 comp rx=0 tx=0).stop 2024-04-15T15:20:24.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.124+0000 7fe3a3bbd640 1 -- 172.21.15.105:0/3947694045 shutdown_connections 2024-04-15T15:20:24.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.124+0000 7fe3a3bbd640 1 --2- 172.21.15.105:0/3947694045 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe39c14b410 0x7fe39c14d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:24.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.124+0000 7fe3a3bbd640 1 --2- 172.21.15.105:0/3947694045 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fe38806c8f0 0x7fe38806edb0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:24.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.124+0000 7fe3a3bbd640 1 --2- 172.21.15.105:0/3947694045 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe39c14af90 0x7fe39c14cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:24.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.124+0000 7fe3a3bbd640 1 --2- 172.21.15.105:0/3947694045 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3840051c0 0x7fe39c14a770 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:24.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.124+0000 7fe3a3bbd640 1 -- 172.21.15.105:0/3947694045 >> 172.21.15.105:0/3947694045 conn(0x7fe39c067720 msgr2=0x7fe39c067b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:24.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.124+0000 7fe3a3bbd640 1 -- 172.21.15.105:0/3947694045 shutdown_connections 2024-04-15T15:20:24.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.124+0000 7fe3a3bbd640 1 -- 172.21.15.105:0/3947694045 wait complete. 2024-04-15T15:20:24.138 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-15T15:20:24.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.241+0000 7f3f61663640 1 Processor -- start 2024-04-15T15:20:24.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.241+0000 7f3f61663640 1 -- start start 2024-04-15T15:20:24.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.241+0000 7f3f61663640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3f5c15eb60 0x7f3f5c0d11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:24.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.241+0000 7f3f61663640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f5c15ef40 con 0x7f3f5c0d1710 2024-04-15T15:20:24.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.241+0000 7f3f61663640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f5c163260 con 0x7f3f5c162d40 2024-04-15T15:20:24.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.241+0000 7f3f61663640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f3f5c0d1b60 con 0x7f3f5c15eb60 2024-04-15T15:20:24.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.242+0000 7f3f5affd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3f5c15eb60 0x7f3f5c0d11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:24.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.242+0000 7f3f527fc640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f3f5c0d1710 0x7f3f5c162630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:44344/0 (socket says 172.21.15.105:44344) 2024-04-15T15:20:24.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.242+0000 7f3f527fc640 1 -- 172.21.15.105:0/3235664829 learned_addr learned my addr 172.21.15.105:0/3235664829 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:20:24.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.242+0000 7f3f5affd640 1 -- 172.21.15.105:0/3235664829 >> v1:172.21.15.154:6789/0 conn(0x7f3f5c162d40 legacy=0x7f3f5c0d3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:24.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.242+0000 7f3f5affd640 1 -- 172.21.15.105:0/3235664829 >> v1:172.21.15.105:6789/0 conn(0x7f3f5c0d1710 legacy=0x7f3f5c162630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:24.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.242+0000 7f3f5a7fc640 1 -- 172.21.15.105:0/3235664829 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1557623509 0 0) 0x7f3f5c15ef40 con 0x7f3f5c0d1710 2024-04-15T15:20:24.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.242+0000 7f3f5affd640 1 -- 172.21.15.105:0/3235664829 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f5c0d2970 con 0x7f3f5c15eb60 2024-04-15T15:20:24.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.243+0000 7f3f5affd640 1 --2- 172.21.15.105:0/3235664829 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f5c15eb60 0x7f3f5c0d11d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c0097a0 tx=0x7f3f4c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=639c080d233aa92d server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:24.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.243+0000 7f3f5a7fc640 1 -- 172.21.15.105:0/3235664829 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c035020 con 0x7f3f5c15eb60 2024-04-15T15:20:24.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.243+0000 7f3f5a7fc640 1 -- 172.21.15.105:0/3235664829 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f4c031830 con 0x7f3f5c15eb60 2024-04-15T15:20:24.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.243+0000 7f3f5a7fc640 1 -- 172.21.15.105:0/3235664829 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c031c00 con 0x7f3f5c15eb60 2024-04-15T15:20:24.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.243+0000 7f3f61663640 1 -- 172.21.15.105:0/3235664829 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f5c15eb60 msgr2=0x7f3f5c0d11d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:24.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.243+0000 7f3f61663640 1 --2- 172.21.15.105:0/3235664829 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f5c15eb60 0x7f3f5c0d11d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c0097a0 tx=0x7f3f4c02b8d0 comp rx=0 tx=0).stop 2024-04-15T15:20:24.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.243+0000 7f3f61663640 1 -- 172.21.15.105:0/3235664829 shutdown_connections 2024-04-15T15:20:24.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.243+0000 7f3f61663640 1 --2- 172.21.15.105:0/3235664829 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f5c15eb60 0x7f3f5c0d11d0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:24.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.243+0000 7f3f61663640 1 -- 172.21.15.105:0/3235664829 >> 172.21.15.105:0/3235664829 conn(0x7f3f5c067720 msgr2=0x7f3f5c067b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:24.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.243+0000 7f3f61663640 1 -- 172.21.15.105:0/3235664829 shutdown_connections 2024-04-15T15:20:24.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.243+0000 7f3f61663640 1 -- 172.21.15.105:0/3235664829 wait complete. 2024-04-15T15:20:24.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.245+0000 7f3f61663640 1 Processor -- start 2024-04-15T15:20:24.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.245+0000 7f3f61663640 1 -- start start 2024-04-15T15:20:24.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.245+0000 7f3f61663640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3f5c0d1710 0x7f3f5c0cb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:24.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.245+0000 7f3f61663640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f5c15eb60 0x7f3f5c15bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:24.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.245+0000 7f3f61663640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f5c162d40 0x7f3f5c0b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:24.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.245+0000 7f3f61663640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3f5c0d2fc0 con 0x7f3f5c162d40 2024-04-15T15:20:24.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.245+0000 7f3f61663640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3f5c163540 con 0x7f3f5c15eb60 2024-04-15T15:20:24.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.245+0000 7f3f61663640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f3f5c0b78f0 con 0x7f3f5c0d1710 2024-04-15T15:20:24.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.246+0000 7f3f5b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f5c162d40 0x7f3f5c0b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:24.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.246+0000 7f3f5b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f5c162d40 0x7f3f5c0b2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54750/0 (socket says 172.21.15.105:54750) 2024-04-15T15:20:24.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.246+0000 7f3f5b7fe640 1 -- 172.21.15.105:0/4161569980 learned_addr learned my addr 172.21.15.105:0/4161569980 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:24.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.246+0000 7f3f527fc640 1 --2- 172.21.15.105:0/4161569980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f5c15eb60 0x7f3f5c15bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:24.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.246+0000 7f3f5affd640 1 --2- 172.21.15.105:0/4161569980 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3f5c0d1710 0x7f3f5c0cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:24.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.246+0000 7f3f5b7fe640 1 -- 172.21.15.105:0/4161569980 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3f5c0d1710 msgr2=0x7f3f5c0cb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:24.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.246+0000 7f3f5b7fe640 1 --2- 172.21.15.105:0/4161569980 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3f5c0d1710 0x7f3f5c0cb890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:24.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.246+0000 7f3f5b7fe640 1 -- 172.21.15.105:0/4161569980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f5c15eb60 msgr2=0x7f3f5c15bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:24.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.246+0000 7f3f5b7fe640 1 --2- 172.21.15.105:0/4161569980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f5c15eb60 0x7f3f5c15bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:24.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.246+0000 7f3f5b7fe640 1 -- 172.21.15.105:0/4161569980 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f5c0cbe70 con 0x7f3f5c162d40 2024-04-15T15:20:24.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.246+0000 7f3f527fc640 1 --2- 172.21.15.105:0/4161569980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f5c15eb60 0x7f3f5c15bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:24.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.246+0000 7f3f5b7fe640 1 --2- 172.21.15.105:0/4161569980 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f5c162d40 0x7f3f5c0b2fc0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f3f5400b990 tx=0x7f3f5400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:24.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.246+0000 7f3f53fff640 1 -- 172.21.15.105:0/4161569980 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f54019070 con 0x7f3f5c162d40 2024-04-15T15:20:24.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.246+0000 7f3f53fff640 1 -- 172.21.15.105:0/4161569980 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f5400cba0 con 0x7f3f5c162d40 2024-04-15T15:20:24.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.246+0000 7f3f61663640 1 -- 172.21.15.105:0/4161569980 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f5c0cc150 con 0x7f3f5c162d40 2024-04-15T15:20:24.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.246+0000 7f3f53fff640 1 -- 172.21.15.105:0/4161569980 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f54014440 con 0x7f3f5c162d40 2024-04-15T15:20:24.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.246+0000 7f3f61663640 1 -- 172.21.15.105:0/4161569980 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f5c0cc710 con 0x7f3f5c162d40 2024-04-15T15:20:24.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.247+0000 7f3f61663640 1 -- 172.21.15.105:0/4161569980 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f5c068a40 con 0x7f3f5c162d40 2024-04-15T15:20:24.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.248+0000 7f3f53fff640 1 -- 172.21.15.105:0/4161569980 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3f5401c070 con 0x7f3f5c162d40 2024-04-15T15:20:24.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.248+0000 7f3f53fff640 1 --2- 172.21.15.105:0/4161569980 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f3f3c06c8f0 0x7f3f3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:24.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.248+0000 7f3f53fff640 1 -- 172.21.15.105:0/4161569980 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f3f5408b4b0 con 0x7f3f5c162d40 2024-04-15T15:20:24.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.248+0000 7f3f5affd640 1 --2- 172.21.15.105:0/4161569980 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f3f3c06c8f0 0x7f3f3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:24.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.248+0000 7f3f5affd640 1 --2- 172.21.15.105:0/4161569980 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f3f3c06c8f0 0x7f3f3c06edb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c036000 tx=0x7f3f4c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:24.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.252+0000 7f3f53fff640 1 -- 172.21.15.105:0/4161569980 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3f54059cc0 con 0x7f3f5c162d40 2024-04-15T15:20:24.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.495+0000 7f3f61663640 1 -- 172.21.15.105:0/4161569980 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f3f5c161f70 con 0x7f3f5c162d40 2024-04-15T15:20:24.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.499+0000 7f3f53fff640 1 -- 172.21.15.105:0/4161569980 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7f3f540595c0 con 0x7f3f5c162d40 2024-04-15T15:20:24.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.501+0000 7f3f61663640 1 -- 172.21.15.105:0/4161569980 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f3f3c06c8f0 msgr2=0x7f3f3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:24.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.501+0000 7f3f61663640 1 --2- 172.21.15.105:0/4161569980 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f3f3c06c8f0 0x7f3f3c06edb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c036000 tx=0x7f3f4c008030 comp rx=0 tx=0).stop 2024-04-15T15:20:24.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.501+0000 7f3f61663640 1 -- 172.21.15.105:0/4161569980 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f5c162d40 msgr2=0x7f3f5c0b2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:24.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.501+0000 7f3f61663640 1 --2- 172.21.15.105:0/4161569980 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f5c162d40 0x7f3f5c0b2fc0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f3f5400b990 tx=0x7f3f5400be50 comp rx=0 tx=0).stop 2024-04-15T15:20:24.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.501+0000 7f3f61663640 1 -- 172.21.15.105:0/4161569980 shutdown_connections 2024-04-15T15:20:24.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.501+0000 7f3f61663640 1 --2- 172.21.15.105:0/4161569980 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f3f3c06c8f0 0x7f3f3c06edb0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:24.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.501+0000 7f3f61663640 1 --2- 172.21.15.105:0/4161569980 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f5c162d40 0x7f3f5c0b2fc0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:24.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.501+0000 7f3f61663640 1 --2- 172.21.15.105:0/4161569980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f5c15eb60 0x7f3f5c15bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:24.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.501+0000 7f3f61663640 1 --2- 172.21.15.105:0/4161569980 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3f5c0d1710 0x7f3f5c0cb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:24.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.501+0000 7f3f61663640 1 -- 172.21.15.105:0/4161569980 >> 172.21.15.105:0/4161569980 conn(0x7f3f5c067720 msgr2=0x7f3f5c067b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:24.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.501+0000 7f3f61663640 1 -- 172.21.15.105:0/4161569980 shutdown_connections 2024-04-15T15:20:24.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.501+0000 7f3f61663640 1 -- 172.21.15.105:0/4161569980 wait complete. 2024-04-15T15:20:24.514 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-15T15:20:24.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.616+0000 7f621e2b1640 1 Processor -- start 2024-04-15T15:20:24.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.616+0000 7f621e2b1640 1 -- start start 2024-04-15T15:20:24.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.617+0000 7f621e2b1640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62180b3c80 0x7f6218167480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:24.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.617+0000 7f621e2b1640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62180cdd40 con 0x7f62180d07e0 2024-04-15T15:20:24.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.617+0000 7f621e2b1640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62180ce020 con 0x7f621815c9a0 2024-04-15T15:20:24.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.617+0000 7f621e2b1640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f62180ce670 con 0x7f62180b3c80 2024-04-15T15:20:24.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.617+0000 7f6217fff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f62180d07e0 0x7f62180d0bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:44352/0 (socket says 172.21.15.105:44352) 2024-04-15T15:20:24.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.617+0000 7f6217fff640 1 -- 172.21.15.105:0/2130549037 learned_addr learned my addr 172.21.15.105:0/2130549037 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:20:24.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.617+0000 7f6216ffd640 1 -- 172.21.15.105:0/2130549037 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 834457075 0 0) 0x7f62180cdd40 con 0x7f62180d07e0 2024-04-15T15:20:24.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.617+0000 7f6216ffd640 1 -- 172.21.15.105:0/2130549037 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6200003520 con 0x7f62180d07e0 2024-04-15T15:20:24.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.617+0000 7f621c84e640 1 --2- 172.21.15.105:0/2130549037 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62180b3c80 0x7f6218167480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:24.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.617+0000 7f6216ffd640 1 -- 172.21.15.105:0/2130549037 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2940960401 0 0) 0x7f6200003520 con 0x7f62180d07e0 2024-04-15T15:20:24.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.617+0000 7f6216ffd640 1 -- 172.21.15.105:0/2130549037 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62180b3c80 msgr2=0x7f6218167480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:24.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.617+0000 7f6216ffd640 1 --2- 172.21.15.105:0/2130549037 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62180b3c80 0x7f6218167480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:24.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.617+0000 7f6216ffd640 1 -- 172.21.15.105:0/2130549037 >> v1:172.21.15.154:6789/0 conn(0x7f621815c9a0 legacy=0x7f62180c7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:24.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.617+0000 7f6216ffd640 1 -- 172.21.15.105:0/2130549037 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62180ce300 con 0x7f62180d07e0 2024-04-15T15:20:24.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.617+0000 7f6216ffd640 1 -- 172.21.15.105:0/2130549037 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7f6208001670 con 0x7f62180d07e0 2024-04-15T15:20:24.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.617+0000 7f6216ffd640 1 -- 172.21.15.105:0/2130549037 >> v1:172.21.15.105:6789/0 conn(0x7f62180d07e0 legacy=0x7f62180d0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:24.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.617+0000 7f6216ffd640 1 --2- 172.21.15.105:0/2130549037 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62000039e0 0x7f6200003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:24.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.617+0000 7f6216ffd640 1 --2- 172.21.15.105:0/2130549037 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6200004800 0x7f6200004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:24.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.618+0000 7f6216ffd640 1 --2- 172.21.15.105:0/2130549037 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62180b3c80 0x7f6200005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:24.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.618+0000 7f6216ffd640 1 -- 172.21.15.105:0/2130549037 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f62180ce670 con 0x7f6200004800 2024-04-15T15:20:24.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.618+0000 7f6216ffd640 1 -- 172.21.15.105:0/2130549037 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f620000baf0 con 0x7f62000039e0 2024-04-15T15:20:24.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.618+0000 7f6216ffd640 1 -- 172.21.15.105:0/2130549037 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f620000bcf0 con 0x7f62180b3c80 2024-04-15T15:20:24.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.618+0000 7f62177fe640 1 --2- 172.21.15.105:0/2130549037 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6200004800 0x7f6200004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:24.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.618+0000 7f6217fff640 1 --2- 172.21.15.105:0/2130549037 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62000039e0 0x7f6200003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:24.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.618+0000 7f621c84e640 1 --2- 172.21.15.105:0/2130549037 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62180b3c80 0x7f6200005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:24.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.618+0000 7f62177fe640 1 -- 172.21.15.105:0/2130549037 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62180b3c80 msgr2=0x7f6200005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:24.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.618+0000 7f62177fe640 1 --2- 172.21.15.105:0/2130549037 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62180b3c80 0x7f6200005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:24.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.618+0000 7f62177fe640 1 -- 172.21.15.105:0/2130549037 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62000039e0 msgr2=0x7f6200003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:24.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.618+0000 7f62177fe640 1 --2- 172.21.15.105:0/2130549037 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62000039e0 0x7f6200003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:24.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.618+0000 7f62177fe640 1 -- 172.21.15.105:0/2130549037 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6200003520 con 0x7f6200004800 2024-04-15T15:20:24.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.619+0000 7f62177fe640 1 --2- 172.21.15.105:0/2130549037 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6200004800 0x7f6200004c80 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f620c001720 tx=0x7f6200005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a28c6a4a56fd4b50 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:24.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.619+0000 7f6216ffd640 1 -- 172.21.15.105:0/2130549037 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620c001370 con 0x7f6200004800 2024-04-15T15:20:24.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.619+0000 7f6216ffd640 1 -- 172.21.15.105:0/2130549037 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f620c0052d0 con 0x7f6200004800 2024-04-15T15:20:24.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.619+0000 7f6216ffd640 1 -- 172.21.15.105:0/2130549037 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620c003830 con 0x7f6200004800 2024-04-15T15:20:24.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.619+0000 7f621e2b1640 1 -- 172.21.15.105:0/2130549037 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6200004800 msgr2=0x7f6200004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:24.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.619+0000 7f621e2b1640 1 --2- 172.21.15.105:0/2130549037 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6200004800 0x7f6200004c80 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f620c001720 tx=0x7f6200005830 comp rx=0 tx=0).stop 2024-04-15T15:20:24.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.619+0000 7f621c84e640 1 -- 172.21.15.105:0/2130549037 reap_dead start 2024-04-15T15:20:24.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.619+0000 7f621e2b1640 1 -- 172.21.15.105:0/2130549037 shutdown_connections 2024-04-15T15:20:24.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.619+0000 7f621e2b1640 1 -- 172.21.15.105:0/2130549037 >> 172.21.15.105:0/2130549037 conn(0x7f621809aa40 msgr2=0x7f62180b30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:24.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.619+0000 7f621e2b1640 1 -- 172.21.15.105:0/2130549037 shutdown_connections 2024-04-15T15:20:24.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.619+0000 7f621e2b1640 1 -- 172.21.15.105:0/2130549037 wait complete. 2024-04-15T15:20:24.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.620+0000 7f621e2b1640 1 Processor -- start 2024-04-15T15:20:24.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.621+0000 7f621e2b1640 1 -- start start 2024-04-15T15:20:24.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.621+0000 7f621e2b1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6200004800 0x7f6218161d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:24.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.621+0000 7f621e2b1640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6218161530 0x7f6218163ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:24.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.621+0000 7f621e2b1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6218152690 0x7f6218152b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:24.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.621+0000 7f621e2b1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f62180cec80 con 0x7f6200004800 2024-04-15T15:20:24.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.621+0000 7f621e2b1640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f62180ceb00 con 0x7f6218152690 2024-04-15T15:20:24.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.621+0000 7f621e2b1640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f6218165470 con 0x7f6218161530 2024-04-15T15:20:24.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.621+0000 7f6217fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6200004800 0x7f6218161d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:24.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.622+0000 7f6217fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6200004800 0x7f6218161d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54764/0 (socket says 172.21.15.105:54764) 2024-04-15T15:20:24.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.622+0000 7f6217fff640 1 -- 172.21.15.105:0/2240882286 learned_addr learned my addr 172.21.15.105:0/2240882286 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:24.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.622+0000 7f62177fe640 1 --2- 172.21.15.105:0/2240882286 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6218161530 0x7f6218163ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:24.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.622+0000 7f621c84e640 1 --2- 172.21.15.105:0/2240882286 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6218152690 0x7f6218152b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:24.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.622+0000 7f6217fff640 1 -- 172.21.15.105:0/2240882286 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6218161530 msgr2=0x7f6218163ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:24.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.622+0000 7f6217fff640 1 --2- 172.21.15.105:0/2240882286 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6218161530 0x7f6218163ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:24.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.622+0000 7f6217fff640 1 -- 172.21.15.105:0/2240882286 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6218152690 msgr2=0x7f6218152b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:24.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.622+0000 7f6217fff640 1 --2- 172.21.15.105:0/2240882286 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6218152690 0x7f6218152b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:24.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.622+0000 7f6217fff640 1 -- 172.21.15.105:0/2240882286 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f621806fb70 con 0x7f6200004800 2024-04-15T15:20:24.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.622+0000 7f621c84e640 1 --2- 172.21.15.105:0/2240882286 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6218152690 0x7f6218152b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:20:24.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.622+0000 7f6217fff640 1 --2- 172.21.15.105:0/2240882286 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6200004800 0x7f6218161d60 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f62080072c0 tx=0x7f6208007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:24.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.622+0000 7f6214ff9640 1 -- 172.21.15.105:0/2240882286 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6208001660 con 0x7f6200004800 2024-04-15T15:20:24.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.622+0000 7f6214ff9640 1 -- 172.21.15.105:0/2240882286 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6208018070 con 0x7f6200004800 2024-04-15T15:20:24.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.622+0000 7f621e2b1640 1 -- 172.21.15.105:0/2240882286 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f621806fe50 con 0x7f6200004800 2024-04-15T15:20:24.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.622+0000 7f6214ff9640 1 -- 172.21.15.105:0/2240882286 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620800fb80 con 0x7f6200004800 2024-04-15T15:20:24.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.622+0000 7f621e2b1640 1 -- 172.21.15.105:0/2240882286 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6218070410 con 0x7f6200004800 2024-04-15T15:20:24.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.623+0000 7f621e2b1640 1 -- 172.21.15.105:0/2240882286 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6218068a40 con 0x7f6200004800 2024-04-15T15:20:24.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.624+0000 7f6214ff9640 1 -- 172.21.15.105:0/2240882286 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f620800fd20 con 0x7f6200004800 2024-04-15T15:20:24.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.624+0000 7f6214ff9640 1 --2- 172.21.15.105:0/2240882286 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f620406ca90 0x7f620406ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:24.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.624+0000 7f6214ff9640 1 -- 172.21.15.105:0/2240882286 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f620808b840 con 0x7f6200004800 2024-04-15T15:20:24.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.624+0000 7f62177fe640 1 --2- 172.21.15.105:0/2240882286 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f620406ca90 0x7f620406ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:24.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.624+0000 7f62177fe640 1 --2- 172.21.15.105:0/2240882286 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f620406ca90 0x7f620406ef50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f620c033000 tx=0x7f620c009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:24.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.628+0000 7f6214ff9640 1 -- 172.21.15.105:0/2240882286 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6208059fc0 con 0x7f6200004800 2024-04-15T15:20:24.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.870+0000 7f621e2b1640 1 -- 172.21.15.105:0/2240882286 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f62180691f0 con 0x7f6200004800 2024-04-15T15:20:24.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.875+0000 7f6214ff9640 1 -- 172.21.15.105:0/2240882286 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f62080598c0 con 0x7f6200004800 2024-04-15T15:20:24.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.876+0000 7f621e2b1640 1 -- 172.21.15.105:0/2240882286 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f620406ca90 msgr2=0x7f620406ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:24.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.876+0000 7f621e2b1640 1 --2- 172.21.15.105:0/2240882286 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f620406ca90 0x7f620406ef50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f620c033000 tx=0x7f620c009030 comp rx=0 tx=0).stop 2024-04-15T15:20:24.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.876+0000 7f621e2b1640 1 -- 172.21.15.105:0/2240882286 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6200004800 msgr2=0x7f6218161d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:24.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.876+0000 7f621e2b1640 1 --2- 172.21.15.105:0/2240882286 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6200004800 0x7f6218161d60 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f62080072c0 tx=0x7f6208007780 comp rx=0 tx=0).stop 2024-04-15T15:20:24.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.876+0000 7f621e2b1640 1 -- 172.21.15.105:0/2240882286 shutdown_connections 2024-04-15T15:20:24.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.876+0000 7f621e2b1640 1 --2- 172.21.15.105:0/2240882286 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6218161530 0x7f6218163ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:24.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.876+0000 7f621e2b1640 1 --2- 172.21.15.105:0/2240882286 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f620406ca90 0x7f620406ef50 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:24.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.876+0000 7f621e2b1640 1 --2- 172.21.15.105:0/2240882286 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6218152690 0x7f6218152b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:24.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.876+0000 7f621e2b1640 1 --2- 172.21.15.105:0/2240882286 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6200004800 0x7f6218161d60 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:24.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.876+0000 7f621e2b1640 1 -- 172.21.15.105:0/2240882286 >> 172.21.15.105:0/2240882286 conn(0x7f621809aa40 msgr2=0x7f621815f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:24.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.876+0000 7f621e2b1640 1 -- 172.21.15.105:0/2240882286 shutdown_connections 2024-04-15T15:20:24.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.877+0000 7f621e2b1640 1 -- 172.21.15.105:0/2240882286 wait complete. 2024-04-15T15:20:24.889 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-15T15:20:24.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.991+0000 7effb8e88640 1 Processor -- start 2024-04-15T15:20:24.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.991+0000 7effb8e88640 1 -- start start 2024-04-15T15:20:24.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.992+0000 7effb8e88640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7effb40d0540 0x7effb406b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:24.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.992+0000 7effb8e88640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7effb40b3c80 0x7effb406bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:24.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.992+0000 7effb8e88640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effb40c8270 con 0x7effb40ce800 2024-04-15T15:20:24.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.992+0000 7effb8e88640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7effb40d0920 con 0x7effb40d0540 2024-04-15T15:20:24.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.992+0000 7effb8e88640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7effb40c7cb0 con 0x7effb40b3c80 2024-04-15T15:20:24.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.993+0000 7effb259c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7effb40d0540 0x7effb406b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:24.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.993+0000 7effabfff640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7effb40b3c80 0x7effb406bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:24.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.993+0000 7effb259c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7effb40d0540 0x7effb406b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.105:40612/0 (socket says 172.21.15.105:40612) 2024-04-15T15:20:24.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.993+0000 7effb2d9d640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7effb40ce800 0x7effb40c6310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.105:60848/0 (socket says 172.21.15.105:60848) 2024-04-15T15:20:24.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.993+0000 7effb259c640 1 -- 172.21.15.105:0/3664774307 learned_addr learned my addr 172.21.15.105:0/3664774307 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:24.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.993+0000 7effb259c640 1 -- 172.21.15.105:0/3664774307 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7effb40b3c80 msgr2=0x7effb406bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:24.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.993+0000 7effb259c640 1 --2- 172.21.15.105:0/3664774307 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7effb40b3c80 0x7effb406bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:24.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.993+0000 7effb259c640 1 -- 172.21.15.105:0/3664774307 >> v1:172.21.15.154:6789/0 conn(0x7effb40ce800 legacy=0x7effb40c6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:24.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.993+0000 7effb259c640 1 -- 172.21.15.105:0/3664774307 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effb40c7f90 con 0x7effb40d0540 2024-04-15T15:20:24.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.993+0000 7effabfff640 1 --2- 172.21.15.105:0/3664774307 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7effb40b3c80 0x7effb406bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:20:24.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.994+0000 7effb259c640 1 --2- 172.21.15.105:0/3664774307 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effb40d0540 0x7effb406b9c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7effa4009870 tx=0x7effa402b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83b400ed2e83de25 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:24.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.996+0000 7effb1d9b640 1 -- 172.21.15.105:0/3664774307 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa4034070 con 0x7effb40d0540 2024-04-15T15:20:24.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.996+0000 7effb1d9b640 1 -- 172.21.15.105:0/3664774307 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effa4030840 con 0x7effb40d0540 2024-04-15T15:20:24.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.996+0000 7effb1d9b640 1 -- 172.21.15.105:0/3664774307 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa4030c10 con 0x7effb40d0540 2024-04-15T15:20:24.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.997+0000 7effb8e88640 1 -- 172.21.15.105:0/3664774307 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effb40d0540 msgr2=0x7effb406b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:24.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.997+0000 7effb8e88640 1 --2- 172.21.15.105:0/3664774307 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effb40d0540 0x7effb406b9c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7effa4009870 tx=0x7effa402b980 comp rx=0 tx=0).stop 2024-04-15T15:20:24.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.997+0000 7effb8e88640 1 -- 172.21.15.105:0/3664774307 shutdown_connections 2024-04-15T15:20:24.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.997+0000 7effb8e88640 1 --2- 172.21.15.105:0/3664774307 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7effb40b3c80 0x7effb406bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:24.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.997+0000 7effb8e88640 1 --2- 172.21.15.105:0/3664774307 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effb40d0540 0x7effb406b9c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:24.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.997+0000 7effb8e88640 1 -- 172.21.15.105:0/3664774307 >> 172.21.15.105:0/3664774307 conn(0x7effb409aa40 msgr2=0x7effb40b30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:24.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.997+0000 7effb8e88640 1 -- 172.21.15.105:0/3664774307 shutdown_connections 2024-04-15T15:20:24.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.997+0000 7effb8e88640 1 -- 172.21.15.105:0/3664774307 wait complete. 2024-04-15T15:20:25.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.998+0000 7effb8e88640 1 Processor -- start 2024-04-15T15:20:25.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.998+0000 7effb8e88640 1 -- start start 2024-04-15T15:20:25.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.999+0000 7effb8e88640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effb40b3c80 0x7effb40b6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:25.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.999+0000 7effb8e88640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effb40ce800 0x7effb414e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:25.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.999+0000 7effb8e88640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7effb40d0540 0x7effb41e5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:25.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.999+0000 7effb8e88640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7effb4070e40 con 0x7effb40ce800 2024-04-15T15:20:25.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.999+0000 7effb8e88640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7effb406f180 con 0x7effb40b3c80 2024-04-15T15:20:25.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.999+0000 7effb8e88640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7effb4070fc0 con 0x7effb40d0540 2024-04-15T15:20:25.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.999+0000 7effabfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effb40ce800 0x7effb414e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:25.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.999+0000 7effabfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effb40ce800 0x7effb414e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54770/0 (socket says 172.21.15.105:54770) 2024-04-15T15:20:25.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.999+0000 7effabfff640 1 -- 172.21.15.105:0/2419998746 learned_addr learned my addr 172.21.15.105:0/2419998746 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:25.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.999+0000 7effb259c640 1 --2- 172.21.15.105:0/2419998746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effb40b3c80 0x7effb40b6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:25.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.999+0000 7effb2d9d640 1 --2- 172.21.15.105:0/2419998746 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7effb40d0540 0x7effb41e5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:25.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.999+0000 7effabfff640 1 -- 172.21.15.105:0/2419998746 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7effb40d0540 msgr2=0x7effb41e5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:25.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.999+0000 7effabfff640 1 --2- 172.21.15.105:0/2419998746 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7effb40d0540 0x7effb41e5210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:25.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.999+0000 7effabfff640 1 -- 172.21.15.105:0/2419998746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effb40b3c80 msgr2=0x7effb40b6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:25.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.999+0000 7effabfff640 1 --2- 172.21.15.105:0/2419998746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effb40b3c80 0x7effb40b6ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:25.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.999+0000 7effabfff640 1 -- 172.21.15.105:0/2419998746 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effb40b5090 con 0x7effb40ce800 2024-04-15T15:20:25.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:24.999+0000 7effb259c640 1 --2- 172.21.15.105:0/2419998746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effb40b3c80 0x7effb40b6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:25.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.000+0000 7effabfff640 1 --2- 172.21.15.105:0/2419998746 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effb40ce800 0x7effb414e060 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7eff9c00e990 tx=0x7eff9c00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:25.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.000+0000 7effaaffd640 1 -- 172.21.15.105:0/2419998746 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff9c017070 con 0x7effb40ce800 2024-04-15T15:20:25.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.000+0000 7effaaffd640 1 -- 172.21.15.105:0/2419998746 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff9c00c9e0 con 0x7effb40ce800 2024-04-15T15:20:25.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.000+0000 7effaaffd640 1 -- 172.21.15.105:0/2419998746 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff9c00cdb0 con 0x7effb40ce800 2024-04-15T15:20:25.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.000+0000 7effb8e88640 1 -- 172.21.15.105:0/2419998746 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effb40b5370 con 0x7effb40ce800 2024-04-15T15:20:25.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.000+0000 7effb8e88640 1 -- 172.21.15.105:0/2419998746 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effb40b5930 con 0x7effb40ce800 2024-04-15T15:20:25.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.001+0000 7effb8e88640 1 -- 172.21.15.105:0/2419998746 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effb4073a80 con 0x7effb40ce800 2024-04-15T15:20:25.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.001+0000 7effaaffd640 1 -- 172.21.15.105:0/2419998746 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7eff9c01a070 con 0x7effb40ce800 2024-04-15T15:20:25.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.002+0000 7effaaffd640 1 --2- 172.21.15.105:0/2419998746 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7eff9406c8f0 0x7eff9406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:25.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.002+0000 7effb259c640 1 --2- 172.21.15.105:0/2419998746 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7eff9406c8f0 0x7eff9406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:25.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.002+0000 7effaaffd640 1 -- 172.21.15.105:0/2419998746 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7eff9c08bac0 con 0x7effb40ce800 2024-04-15T15:20:25.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.002+0000 7effb259c640 1 --2- 172.21.15.105:0/2419998746 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7eff9406c8f0 0x7eff9406edb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7effa4036000 tx=0x7effa403b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:25.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.006+0000 7effaaffd640 1 -- 172.21.15.105:0/2419998746 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7effb4073a80 con 0x7effb40ce800 2024-04-15T15:20:25.116 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-04-15T15:20:25.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.248+0000 7effb8e88640 1 -- 172.21.15.105:0/2419998746 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7effb4068a40 con 0x7effb40ce800 2024-04-15T15:20:25.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.252+0000 7effaaffd640 1 -- 172.21.15.105:0/2419998746 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) ==== 97+0+0 (secure 0 0 0) 0x7eff9c05a240 con 0x7effb40ce800 2024-04-15T15:20:25.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.253+0000 7effb8e88640 1 -- 172.21.15.105:0/2419998746 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7eff9406c8f0 msgr2=0x7eff9406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:25.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.253+0000 7effb8e88640 1 --2- 172.21.15.105:0/2419998746 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7eff9406c8f0 0x7eff9406edb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7effa4036000 tx=0x7effa403b040 comp rx=0 tx=0).stop 2024-04-15T15:20:25.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.253+0000 7effb8e88640 1 -- 172.21.15.105:0/2419998746 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effb40ce800 msgr2=0x7effb414e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:25.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.253+0000 7effb8e88640 1 --2- 172.21.15.105:0/2419998746 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effb40ce800 0x7effb414e060 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7eff9c00e990 tx=0x7eff9c00ee50 comp rx=0 tx=0).stop 2024-04-15T15:20:25.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.254+0000 7effb8e88640 1 -- 172.21.15.105:0/2419998746 shutdown_connections 2024-04-15T15:20:25.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.254+0000 7effb8e88640 1 --2- 172.21.15.105:0/2419998746 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7effb40d0540 0x7effb41e5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:25.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.254+0000 7effb8e88640 1 --2- 172.21.15.105:0/2419998746 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7eff9406c8f0 0x7eff9406edb0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:25.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.254+0000 7effb8e88640 1 --2- 172.21.15.105:0/2419998746 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effb40ce800 0x7effb414e060 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:25.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.254+0000 7effb8e88640 1 --2- 172.21.15.105:0/2419998746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effb40b3c80 0x7effb40b6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:25.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.254+0000 7effb8e88640 1 -- 172.21.15.105:0/2419998746 >> 172.21.15.105:0/2419998746 conn(0x7effb409aa40 msgr2=0x7effb415b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:25.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.254+0000 7effb8e88640 1 -- 172.21.15.105:0/2419998746 shutdown_connections 2024-04-15T15:20:25.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.254+0000 7effb8e88640 1 -- 172.21.15.105:0/2419998746 wait complete. 2024-04-15T15:20:25.266 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-15T15:20:25.345 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-15T15:20:25.348 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-15T15:20:25.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.370+0000 7f664aceb640 1 Processor -- start 2024-04-15T15:20:25.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.370+0000 7f664aceb640 1 -- start start 2024-04-15T15:20:25.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.371+0000 7f664aceb640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f664415c7b0 0x7f66440d09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:25.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.371+0000 7f664aceb640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f66440d0f30 0x7f66440b25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:25.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.371+0000 7f664aceb640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f66440b2ae0 0x7f6644160410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:25.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.371+0000 7f664aceb640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f66440ccf40 con 0x7f664415c7b0 2024-04-15T15:20:25.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.371+0000 7f664aceb640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f66440b32d0 con 0x7f66440d0f30 2024-04-15T15:20:25.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.371+0000 7f664aceb640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f66440b35b0 con 0x7f66440b2ae0 2024-04-15T15:20:25.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.371+0000 7f6648a87640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f664415c7b0 0x7f66440d09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:25.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.371+0000 7f6648a87640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f664415c7b0 0x7f66440d09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54774/0 (socket says 172.21.15.105:54774) 2024-04-15T15:20:25.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.371+0000 7f6648a87640 1 -- 172.21.15.105:0/1408443646 learned_addr learned my addr 172.21.15.105:0/1408443646 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:25.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.371+0000 7f6643fff640 1 --2- 172.21.15.105:0/1408443646 >> v2:172.21.15.154:3300/0 conn(0x7f66440d0f30 0x7f66440b25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:25.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.371+0000 7f6649288640 1 --2- 172.21.15.105:0/1408443646 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f66440b2ae0 0x7f6644160410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:25.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.371+0000 7f6648a87640 1 -- 172.21.15.105:0/1408443646 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f66440b2ae0 msgr2=0x7f6644160410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:25.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.371+0000 7f6648a87640 1 --2- 172.21.15.105:0/1408443646 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f66440b2ae0 0x7f6644160410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:25.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.371+0000 7f6648a87640 1 -- 172.21.15.105:0/1408443646 >> v2:172.21.15.154:3300/0 conn(0x7f66440d0f30 msgr2=0x7f66440b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:25.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.371+0000 7f6648a87640 1 --2- 172.21.15.105:0/1408443646 >> v2:172.21.15.154:3300/0 conn(0x7f66440d0f30 0x7f66440b25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:25.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.371+0000 7f6648a87640 1 -- 172.21.15.105:0/1408443646 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66440b3890 con 0x7f664415c7b0 2024-04-15T15:20:25.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.372+0000 7f6648a87640 1 --2- 172.21.15.105:0/1408443646 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f664415c7b0 0x7f66440d09f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6644072530 tx=0x7f663802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=559eaf327386da85 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:25.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.372+0000 7f66437fe640 1 -- 172.21.15.105:0/1408443646 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6638035020 con 0x7f664415c7b0 2024-04-15T15:20:25.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.372+0000 7f66437fe640 1 -- 172.21.15.105:0/1408443646 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6638031870 con 0x7f664415c7b0 2024-04-15T15:20:25.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.372+0000 7f66437fe640 1 -- 172.21.15.105:0/1408443646 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6638031c40 con 0x7f664415c7b0 2024-04-15T15:20:25.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.373+0000 7f664aceb640 1 -- 172.21.15.105:0/1408443646 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f664415c7b0 msgr2=0x7f66440d09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:25.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.373+0000 7f664aceb640 1 --2- 172.21.15.105:0/1408443646 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f664415c7b0 0x7f66440d09f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6644072530 tx=0x7f663802b910 comp rx=0 tx=0).stop 2024-04-15T15:20:25.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.373+0000 7f664aceb640 1 -- 172.21.15.105:0/1408443646 shutdown_connections 2024-04-15T15:20:25.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.373+0000 7f664aceb640 1 --2- 172.21.15.105:0/1408443646 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f66440b2ae0 0x7f6644160410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:25.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.373+0000 7f664aceb640 1 --2- 172.21.15.105:0/1408443646 >> v2:172.21.15.154:3300/0 conn(0x7f66440d0f30 0x7f66440b25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:25.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.373+0000 7f664aceb640 1 --2- 172.21.15.105:0/1408443646 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f664415c7b0 0x7f66440d09f0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:25.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.373+0000 7f664aceb640 1 -- 172.21.15.105:0/1408443646 >> 172.21.15.105:0/1408443646 conn(0x7f66440d25a0 msgr2=0x7f66440d29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:25.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.373+0000 7f664aceb640 1 -- 172.21.15.105:0/1408443646 shutdown_connections 2024-04-15T15:20:25.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.373+0000 7f664aceb640 1 -- 172.21.15.105:0/1408443646 wait complete. 2024-04-15T15:20:25.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.374+0000 7f664aceb640 1 Processor -- start 2024-04-15T15:20:25.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.374+0000 7f664aceb640 1 -- start start 2024-04-15T15:20:25.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.375+0000 7f664aceb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66440b2ae0 0x7f66441f4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:25.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.375+0000 7f664aceb640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f66440d0f30 0x7f664415ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:25.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.375+0000 7f664aceb640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f664415c7b0 0x7f664415f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:25.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.375+0000 7f664aceb640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f664415cb90 con 0x7f66440b2ae0 2024-04-15T15:20:25.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.375+0000 7f664aceb640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f66440b2f70 con 0x7f664415c7b0 2024-04-15T15:20:25.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.375+0000 7f664aceb640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f664415cd10 con 0x7f66440d0f30 2024-04-15T15:20:25.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.375+0000 7f6648a87640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66440b2ae0 0x7f66441f4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:25.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.375+0000 7f6648a87640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66440b2ae0 0x7f66441f4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54786/0 (socket says 172.21.15.105:54786) 2024-04-15T15:20:25.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.375+0000 7f6648a87640 1 -- 172.21.15.105:0/2715726607 learned_addr learned my addr 172.21.15.105:0/2715726607 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:25.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.375+0000 7f6643fff640 1 --2- 172.21.15.105:0/2715726607 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f66440d0f30 0x7f664415ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:25.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.375+0000 7f6649288640 1 --2- 172.21.15.105:0/2715726607 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f664415c7b0 0x7f664415f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:25.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.375+0000 7f6648a87640 1 -- 172.21.15.105:0/2715726607 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f66440d0f30 msgr2=0x7f664415ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:25.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.376+0000 7f6648a87640 1 --2- 172.21.15.105:0/2715726607 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f66440d0f30 0x7f664415ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:25.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.376+0000 7f6648a87640 1 -- 172.21.15.105:0/2715726607 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f664415c7b0 msgr2=0x7f664415f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:25.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.376+0000 7f6648a87640 1 --2- 172.21.15.105:0/2715726607 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f664415c7b0 0x7f664415f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:25.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.376+0000 7f6648a87640 1 -- 172.21.15.105:0/2715726607 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f664406ea00 con 0x7f66440b2ae0 2024-04-15T15:20:25.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.376+0000 7f6648a87640 1 --2- 172.21.15.105:0/2715726607 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66440b2ae0 0x7f66441f4640 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6638000c00 tx=0x7f66380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:25.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.376+0000 7f66417fa640 1 -- 172.21.15.105:0/2715726607 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6638035040 con 0x7f66440b2ae0 2024-04-15T15:20:25.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.376+0000 7f66417fa640 1 -- 172.21.15.105:0/2715726607 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6638030d00 con 0x7f66440b2ae0 2024-04-15T15:20:25.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.376+0000 7f664aceb640 1 -- 172.21.15.105:0/2715726607 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f664406e720 con 0x7f66440b2ae0 2024-04-15T15:20:25.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.376+0000 7f66417fa640 1 -- 172.21.15.105:0/2715726607 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66380456c0 con 0x7f66440b2ae0 2024-04-15T15:20:25.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.376+0000 7f664aceb640 1 -- 172.21.15.105:0/2715726607 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66440b57e0 con 0x7f66440b2ae0 2024-04-15T15:20:25.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.377+0000 7f664aceb640 1 -- 172.21.15.105:0/2715726607 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f664415c110 con 0x7f66440b2ae0 2024-04-15T15:20:25.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.377+0000 7f66417fa640 1 -- 172.21.15.105:0/2715726607 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6638038020 con 0x7f66440b2ae0 2024-04-15T15:20:25.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.378+0000 7f66417fa640 1 --2- 172.21.15.105:0/2715726607 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f662806c8f0 0x7f662806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:25.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.378+0000 7f66417fa640 1 -- 172.21.15.105:0/2715726607 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f66380ada10 con 0x7f66440b2ae0 2024-04-15T15:20:25.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.378+0000 7f6643fff640 1 --2- 172.21.15.105:0/2715726607 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f662806c8f0 0x7f662806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:25.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.378+0000 7f6643fff640 1 --2- 172.21.15.105:0/2715726607 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f662806c8f0 0x7f662806edb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6630006fd0 tx=0x7f6630008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:25.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.382+0000 7f66417fa640 1 -- 172.21.15.105:0/2715726607 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f663807c220 con 0x7f66440b2ae0 2024-04-15T15:20:25.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.628+0000 7f664aceb640 1 -- 172.21.15.105:0/2715726607 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f6644069150 con 0x7f66440b2ae0 2024-04-15T15:20:25.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.632+0000 7f66417fa640 1 -- 172.21.15.105:0/2715726607 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7f663807bb20 con 0x7f66440b2ae0 2024-04-15T15:20:25.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.633+0000 7f664aceb640 1 -- 172.21.15.105:0/2715726607 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f662806c8f0 msgr2=0x7f662806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:25.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.633+0000 7f664aceb640 1 --2- 172.21.15.105:0/2715726607 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f662806c8f0 0x7f662806edb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6630006fd0 tx=0x7f6630008040 comp rx=0 tx=0).stop 2024-04-15T15:20:25.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.633+0000 7f664aceb640 1 -- 172.21.15.105:0/2715726607 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66440b2ae0 msgr2=0x7f66441f4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:25.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.633+0000 7f664aceb640 1 --2- 172.21.15.105:0/2715726607 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66440b2ae0 0x7f66441f4640 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6638000c00 tx=0x7f66380046c0 comp rx=0 tx=0).stop 2024-04-15T15:20:25.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.634+0000 7f664aceb640 1 -- 172.21.15.105:0/2715726607 shutdown_connections 2024-04-15T15:20:25.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.634+0000 7f664aceb640 1 --2- 172.21.15.105:0/2715726607 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f66440d0f30 0x7f664415ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:25.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.634+0000 7f664aceb640 1 --2- 172.21.15.105:0/2715726607 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f662806c8f0 0x7f662806edb0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:25.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.634+0000 7f664aceb640 1 --2- 172.21.15.105:0/2715726607 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f664415c7b0 0x7f664415f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:25.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.634+0000 7f664aceb640 1 --2- 172.21.15.105:0/2715726607 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66440b2ae0 0x7f66441f4640 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:25.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.634+0000 7f664aceb640 1 -- 172.21.15.105:0/2715726607 >> 172.21.15.105:0/2715726607 conn(0x7f66440d25a0 msgr2=0x7f66440cd270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:25.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.634+0000 7f664aceb640 1 -- 172.21.15.105:0/2715726607 shutdown_connections 2024-04-15T15:20:25.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.634+0000 7f664aceb640 1 -- 172.21.15.105:0/2715726607 wait complete. 2024-04-15T15:20:25.647 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:20:25.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.748+0000 7f44ef09a640 1 Processor -- start 2024-04-15T15:20:25.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.748+0000 7f44ef09a640 1 -- start start 2024-04-15T15:20:25.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.749+0000 7f44ef09a640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f44e815e940 0x7f44e80b1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:25.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.749+0000 7f44ef09a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f44e80b2430 0x7f44e80cef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:25.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.749+0000 7f44ef09a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44e809afe0 con 0x7f44e809aa40 2024-04-15T15:20:25.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.749+0000 7f44ef09a640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f44e80b2120 con 0x7f44e815e940 2024-04-15T15:20:25.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.749+0000 7f44ef09a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f44e809b2c0 con 0x7f44e80b2430 2024-04-15T15:20:25.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.749+0000 7f44ed637640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f44e809aa40 0x7f44e80d0cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:44358/0 (socket says 172.21.15.105:44358) 2024-04-15T15:20:25.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.749+0000 7f44ed637640 1 -- 172.21.15.105:0/3371099510 learned_addr learned my addr 172.21.15.105:0/3371099510 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:20:25.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.749+0000 7f44ece36640 1 --2- 172.21.15.105:0/3371099510 >> v2:172.21.15.154:3300/0 conn(0x7f44e815e940 0x7f44e80b1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:25.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.749+0000 7f44e7fff640 1 --2- 172.21.15.105:0/3371099510 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f44e80b2430 0x7f44e80cef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:25.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.749+0000 7f44e77fe640 1 -- 172.21.15.105:0/3371099510 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 982812131 0 0) 0x7f44e809afe0 con 0x7f44e809aa40 2024-04-15T15:20:25.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.749+0000 7f44e77fe640 1 -- 172.21.15.105:0/3371099510 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f44d0003520 con 0x7f44e809aa40 2024-04-15T15:20:25.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.750+0000 7f44e77fe640 1 -- 172.21.15.105:0/3371099510 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2106448100 0 0) 0x7f44d0003520 con 0x7f44e809aa40 2024-04-15T15:20:25.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.750+0000 7f44e77fe640 1 -- 172.21.15.105:0/3371099510 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f44e80b2430 msgr2=0x7f44e80cef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:25.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.750+0000 7f44e77fe640 1 --2- 172.21.15.105:0/3371099510 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f44e80b2430 0x7f44e80cef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:25.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.750+0000 7f44e77fe640 1 -- 172.21.15.105:0/3371099510 >> v2:172.21.15.154:3300/0 conn(0x7f44e815e940 msgr2=0x7f44e80b1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:25.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.750+0000 7f44e77fe640 1 --2- 172.21.15.105:0/3371099510 >> v2:172.21.15.154:3300/0 conn(0x7f44e815e940 0x7f44e80b1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:25.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.750+0000 7f44e77fe640 1 -- 172.21.15.105:0/3371099510 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44e809b5a0 con 0x7f44e809aa40 2024-04-15T15:20:25.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.750+0000 7f44e77fe640 1 -- 172.21.15.105:0/3371099510 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7f44e0001670 con 0x7f44e809aa40 2024-04-15T15:20:25.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.750+0000 7f44e77fe640 1 -- 172.21.15.105:0/3371099510 >> v1:172.21.15.105:6789/0 conn(0x7f44e809aa40 legacy=0x7f44e80d0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:25.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.750+0000 7f44e77fe640 1 --2- 172.21.15.105:0/3371099510 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44d00039b0 0x7f44d0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:25.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.750+0000 7f44e77fe640 1 --2- 172.21.15.105:0/3371099510 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f44e80b2430 0x7f44d0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:25.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.750+0000 7f44e77fe640 1 --2- 172.21.15.105:0/3371099510 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44d0004d40 0x7f44d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:25.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.750+0000 7f44e77fe640 1 -- 172.21.15.105:0/3371099510 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f44e80b2120 con 0x7f44d00039b0 2024-04-15T15:20:25.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.750+0000 7f44e77fe640 1 -- 172.21.15.105:0/3371099510 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f44e809b2c0 con 0x7f44d0004d40 2024-04-15T15:20:25.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.750+0000 7f44e77fe640 1 -- 172.21.15.105:0/3371099510 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f44d000b9c0 con 0x7f44e80b2430 2024-04-15T15:20:25.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.750+0000 7f44ece36640 1 --2- 172.21.15.105:0/3371099510 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44d00039b0 0x7f44d0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:25.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.750+0000 7f44e7fff640 1 --2- 172.21.15.105:0/3371099510 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f44e80b2430 0x7f44d0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:25.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.750+0000 7f44ed637640 1 --2- 172.21.15.105:0/3371099510 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44d0004d40 0x7f44d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:25.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.750+0000 7f44ece36640 1 -- 172.21.15.105:0/3371099510 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f44e80b2430 msgr2=0x7f44d0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:25.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.750+0000 7f44ece36640 1 --2- 172.21.15.105:0/3371099510 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f44e80b2430 0x7f44d0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:25.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.750+0000 7f44ece36640 1 -- 172.21.15.105:0/3371099510 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44d0004d40 msgr2=0x7f44d00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:25.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.750+0000 7f44ece36640 1 --2- 172.21.15.105:0/3371099510 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44d0004d40 0x7f44d00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:25.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.750+0000 7f44ece36640 1 -- 172.21.15.105:0/3371099510 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44d0003520 con 0x7f44d00039b0 2024-04-15T15:20:25.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.751+0000 7f44ece36640 1 --2- 172.21.15.105:0/3371099510 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44d00039b0 0x7f44d0003dd0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f44d8009fc0 tx=0x7f44d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86bc37f2430b57b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:25.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.751+0000 7f44e77fe640 1 -- 172.21.15.105:0/3371099510 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d8039070 con 0x7f44d00039b0 2024-04-15T15:20:25.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.751+0000 7f44e77fe640 1 -- 172.21.15.105:0/3371099510 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44d802cc60 con 0x7f44d00039b0 2024-04-15T15:20:25.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.751+0000 7f44e77fe640 1 -- 172.21.15.105:0/3371099510 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d8009c10 con 0x7f44d00039b0 2024-04-15T15:20:25.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.752+0000 7f44ef09a640 1 -- 172.21.15.105:0/3371099510 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44d00039b0 msgr2=0x7f44d0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:25.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.752+0000 7f44ef09a640 1 --2- 172.21.15.105:0/3371099510 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44d00039b0 0x7f44d0003dd0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f44d8009fc0 tx=0x7f44d0005830 comp rx=0 tx=0).stop 2024-04-15T15:20:25.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.752+0000 7f44ed637640 1 -- 172.21.15.105:0/3371099510 reap_dead start 2024-04-15T15:20:25.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.752+0000 7f44ef09a640 1 -- 172.21.15.105:0/3371099510 shutdown_connections 2024-04-15T15:20:25.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.752+0000 7f44ef09a640 1 -- 172.21.15.105:0/3371099510 >> 172.21.15.105:0/3371099510 conn(0x7f44e80d25a0 msgr2=0x7f44e80d29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:25.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.752+0000 7f44ef09a640 1 -- 172.21.15.105:0/3371099510 shutdown_connections 2024-04-15T15:20:25.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.752+0000 7f44ef09a640 1 -- 172.21.15.105:0/3371099510 wait complete. 2024-04-15T15:20:25.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.753+0000 7f44ef09a640 1 Processor -- start 2024-04-15T15:20:25.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.753+0000 7f44ef09a640 1 -- start start 2024-04-15T15:20:25.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.754+0000 7f44ef09a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44d00039b0 0x7f44e814bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:25.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.754+0000 7f44ef09a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f44e809ea50 0x7f44e809eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:25.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.754+0000 7f44ef09a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44e815f360 0x7f44e815f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:25.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.754+0000 7f44ef09a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f44e80d0730 con 0x7f44d00039b0 2024-04-15T15:20:25.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.754+0000 7f44ef09a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f44e815ed20 con 0x7f44e815f360 2024-04-15T15:20:25.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.754+0000 7f44ef09a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f44e80d13d0 con 0x7f44e809ea50 2024-04-15T15:20:25.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.754+0000 7f44ece36640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44d00039b0 0x7f44e814bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:25.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.754+0000 7f44ece36640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44d00039b0 0x7f44e814bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54798/0 (socket says 172.21.15.105:54798) 2024-04-15T15:20:25.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.754+0000 7f44ece36640 1 -- 172.21.15.105:0/4018879725 learned_addr learned my addr 172.21.15.105:0/4018879725 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:25.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.754+0000 7f44e7fff640 1 --2- 172.21.15.105:0/4018879725 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f44e809ea50 0x7f44e809eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:25.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.754+0000 7f44ed637640 1 --2- 172.21.15.105:0/4018879725 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44e815f360 0x7f44e815f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:25.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.754+0000 7f44ece36640 1 -- 172.21.15.105:0/4018879725 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f44e809ea50 msgr2=0x7f44e809eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:25.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.754+0000 7f44ece36640 1 --2- 172.21.15.105:0/4018879725 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f44e809ea50 0x7f44e809eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:25.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.754+0000 7f44ece36640 1 -- 172.21.15.105:0/4018879725 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44e815f360 msgr2=0x7f44e815f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:25.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.754+0000 7f44ece36640 1 --2- 172.21.15.105:0/4018879725 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44e815f360 0x7f44e815f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:25.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.754+0000 7f44ece36640 1 -- 172.21.15.105:0/4018879725 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44e809f730 con 0x7f44d00039b0 2024-04-15T15:20:25.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.754+0000 7f44ece36640 1 --2- 172.21.15.105:0/4018879725 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44d00039b0 0x7f44e814bad0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f44d8037040 tx=0x7f44d8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:25.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.755+0000 7f44e57fa640 1 -- 172.21.15.105:0/4018879725 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d8039050 con 0x7f44d00039b0 2024-04-15T15:20:25.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.755+0000 7f44e57fa640 1 -- 172.21.15.105:0/4018879725 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44d803a040 con 0x7f44d00039b0 2024-04-15T15:20:25.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.755+0000 7f44ef09a640 1 -- 172.21.15.105:0/4018879725 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44e809fa10 con 0x7f44d00039b0 2024-04-15T15:20:25.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.755+0000 7f44e57fa640 1 -- 172.21.15.105:0/4018879725 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d8009c10 con 0x7f44d00039b0 2024-04-15T15:20:25.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.755+0000 7f44ef09a640 1 -- 172.21.15.105:0/4018879725 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44e809ffd0 con 0x7f44d00039b0 2024-04-15T15:20:25.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.756+0000 7f44ef09a640 1 -- 172.21.15.105:0/4018879725 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44e8069150 con 0x7f44d00039b0 2024-04-15T15:20:25.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.756+0000 7f44e57fa640 1 -- 172.21.15.105:0/4018879725 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f44d8047030 con 0x7f44d00039b0 2024-04-15T15:20:25.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.756+0000 7f44e57fa640 1 --2- 172.21.15.105:0/4018879725 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f44d406c920 0x7f44d406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:25.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.756+0000 7f44e57fa640 1 -- 172.21.15.105:0/4018879725 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f44d80ada60 con 0x7f44d00039b0 2024-04-15T15:20:25.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.756+0000 7f44e7fff640 1 --2- 172.21.15.105:0/4018879725 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f44d406c920 0x7f44d406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:25.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.757+0000 7f44e7fff640 1 --2- 172.21.15.105:0/4018879725 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f44d406c920 0x7f44d406ede0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f44e815bc60 tx=0x7f44dc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:25.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.761+0000 7f44e57fa640 1 -- 172.21.15.105:0/4018879725 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f44d804e060 con 0x7f44d00039b0 2024-04-15T15:20:25.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.937+0000 7f44ef09a640 1 -- 172.21.15.105:0/4018879725 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f44e806d260 con 0x7f44d00039b0 2024-04-15T15:20:25.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.938+0000 7f44e57fa640 1 -- 172.21.15.105:0/4018879725 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) ==== 74+0+12939 (secure 0 0 0) 0x7f44d807c270 con 0x7f44d00039b0 2024-04-15T15:20:25.940 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:20:25.940 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":30,"fsid":"09e4d2a4-a324-4793-af77-644b25e520e1","created":"2024-04-15T15:19:01.524543+0000","modified":"2024-04-15T15:20:22.982120+0000","last_up_change":"2024-04-15T15:19:12.825815+0000","last_in_change":"2024-04-15T15:19:03.788126+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:19:13.748585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6f6292b-2d7e-4aa0-9d1c-6ddd0f97287c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6801","nonce":2873040054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6803","nonce":2873040054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6807","nonce":2873040054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6805","nonce":2873040054}]},"public_addr":"172.21.15.105:6801/2873040054","cluster_addr":"172.21.15.105:6803/2873040054","heartbeat_back_addr":"172.21.15.105:6807/2873040054","heartbeat_front_addr":"172.21.15.105:6805/2873040054","state":["exists","up"]},{"osd":1,"uuid":"8900966a-b393-41e3-8f93-e5a43bd23622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6818","nonce":2494994479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6821","nonce":2494994479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6828","nonce":2494994479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6823","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6825","nonce":2494994479}]},"public_addr":"172.21.15.105:6818/2494994479","cluster_addr":"172.21.15.105:6821/2494994479","heartbeat_back_addr":"172.21.15.105:6828/2494994479","heartbeat_front_addr":"172.21.15.105:6825/2494994479","state":["exists","up"]},{"osd":2,"uuid":"81a5dc7a-9325-4ff5-941d-a2d5fba21f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6809","nonce":3798408699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6811","nonce":3798408699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6815","nonce":3798408699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6813","nonce":3798408699}]},"public_addr":"172.21.15.105:6809/3798408699","cluster_addr":"172.21.15.105:6811/3798408699","heartbeat_back_addr":"172.21.15.105:6815/3798408699","heartbeat_front_addr":"172.21.15.105:6813/3798408699","state":["exists","up"]},{"osd":3,"uuid":"df766ed2-5c71-4805-96f4-3d4970edc960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6819","nonce":1920720186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6824","nonce":1920720186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6831","nonce":1920720186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6827","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6829","nonce":1920720186}]},"public_addr":"172.21.15.105:6819/1920720186","cluster_addr":"172.21.15.105:6824/1920720186","heartbeat_back_addr":"172.21.15.105:6831/1920720186","heartbeat_front_addr":"172.21.15.105:6829/1920720186","state":["exists","up"]},{"osd":4,"uuid":"e1039e03-28b8-49df-b951-c91930b93c9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6810","nonce":3516494456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6813","nonce":3516494456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6820","nonce":3516494456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6817","nonce":3516494456}]},"public_addr":"172.21.15.154:6810/3516494456","cluster_addr":"172.21.15.154:6813/3516494456","heartbeat_back_addr":"172.21.15.154:6820/3516494456","heartbeat_front_addr":"172.21.15.154:6817/3516494456","state":["exists","up"]},{"osd":5,"uuid":"a9f5eaf1-66ff-405a-a9e4-ec3340812a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6801","nonce":540367751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6803","nonce":540367751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6807","nonce":540367751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6805","nonce":540367751}]},"public_addr":"172.21.15.154:6801/540367751","cluster_addr":"172.21.15.154:6803/540367751","heartbeat_back_addr":"172.21.15.154:6807/540367751","heartbeat_front_addr":"172.21.15.154:6805/540367751","state":["exists","up"]},{"osd":6,"uuid":"267d02ea-675f-428f-aae8-e0375e4d4feb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6825","nonce":1550829005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6827","nonce":1550829005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6831","nonce":1550829005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6829","nonce":1550829005}]},"public_addr":"172.21.15.154:6825/1550829005","cluster_addr":"172.21.15.154:6827/1550829005","heartbeat_back_addr":"172.21.15.154:6831/1550829005","heartbeat_front_addr":"172.21.15.154:6829/1550829005","state":["exists","up"]},{"osd":7,"uuid":"759ac6cc-896a-423c-885f-9fe90d8dcb14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6809","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6811","nonce":3920089903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6816","nonce":3920089903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6823","nonce":3920089903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6819","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6821","nonce":3920089903}]},"public_addr":"172.21.15.154:6811/3920089903","cluster_addr":"172.21.15.154:6816/3920089903","heartbeat_back_addr":"172.21.15.154:6823/3920089903","heartbeat_front_addr":"172.21.15.154:6821/3920089903","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.949509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.984072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.907278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.947391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.975447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.998896+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:09.028739+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.894119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:20:25.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.940+0000 7f44ef09a640 1 -- 172.21.15.105:0/4018879725 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f44d406c920 msgr2=0x7f44d406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:25.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.940+0000 7f44ef09a640 1 --2- 172.21.15.105:0/4018879725 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f44d406c920 0x7f44d406ede0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f44e815bc60 tx=0x7f44dc008040 comp rx=0 tx=0).stop 2024-04-15T15:20:25.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.940+0000 7f44ef09a640 1 -- 172.21.15.105:0/4018879725 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44d00039b0 msgr2=0x7f44e814bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:25.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.940+0000 7f44ef09a640 1 --2- 172.21.15.105:0/4018879725 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44d00039b0 0x7f44e814bad0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f44d8037040 tx=0x7f44d8009750 comp rx=0 tx=0).stop 2024-04-15T15:20:25.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.941+0000 7f44ef09a640 1 -- 172.21.15.105:0/4018879725 shutdown_connections 2024-04-15T15:20:25.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.941+0000 7f44ef09a640 1 --2- 172.21.15.105:0/4018879725 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f44e809ea50 0x7f44e809eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:25.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.941+0000 7f44ef09a640 1 --2- 172.21.15.105:0/4018879725 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f44d406c920 0x7f44d406ede0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:25.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.941+0000 7f44ef09a640 1 --2- 172.21.15.105:0/4018879725 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44e815f360 0x7f44e815f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:25.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.941+0000 7f44ef09a640 1 --2- 172.21.15.105:0/4018879725 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44d00039b0 0x7f44e814bad0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:25.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.941+0000 7f44ef09a640 1 -- 172.21.15.105:0/4018879725 >> 172.21.15.105:0/4018879725 conn(0x7f44e80d25a0 msgr2=0x7f44e80b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:25.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.941+0000 7f44ef09a640 1 -- 172.21.15.105:0/4018879725 shutdown_connections 2024-04-15T15:20:25.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:25.941+0000 7f44ef09a640 1 -- 172.21.15.105:0/4018879725 wait complete. 2024-04-15T15:20:25.953 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:19:13.748585+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:20:25.954 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:20:26.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.058+0000 7fa6b318f640 1 Processor -- start 2024-04-15T15:20:26.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.058+0000 7fa6b318f640 1 -- start start 2024-04-15T15:20:26.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.059+0000 7fa6b318f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6ac06f6a0 0x7fa6ac0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:26.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.059+0000 7fa6b318f640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6ac0b40a0 con 0x7fa6ac0cd1d0 2024-04-15T15:20:26.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.059+0000 7fa6b318f640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6ac0b4380 con 0x7fa6ac144c90 2024-04-15T15:20:26.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.059+0000 7fa6b318f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fa6ac0b3e50 con 0x7fa6ac06f6a0 2024-04-15T15:20:26.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.059+0000 7fa6b172c640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fa6ac0cd1d0 0x7fa6ac0cd620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:44362/0 (socket says 172.21.15.105:44362) 2024-04-15T15:20:26.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.059+0000 7fa6b172c640 1 -- 172.21.15.105:0/813005446 learned_addr learned my addr 172.21.15.105:0/813005446 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:20:26.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.059+0000 7fa6abfff640 1 --2- 172.21.15.105:0/813005446 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6ac06f6a0 0x7fa6ac0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:26.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.059+0000 7fa6ab7fe640 1 -- 172.21.15.105:0/813005446 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2726985578 0 0) 0x7fa6ac0b40a0 con 0x7fa6ac0cd1d0 2024-04-15T15:20:26.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.059+0000 7fa6ab7fe640 1 -- 172.21.15.105:0/813005446 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa69c003520 con 0x7fa6ac0cd1d0 2024-04-15T15:20:26.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.059+0000 7fa6ab7fe640 1 -- 172.21.15.105:0/813005446 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2063845420 0 0) 0x7fa69c003520 con 0x7fa6ac0cd1d0 2024-04-15T15:20:26.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.059+0000 7fa6ab7fe640 1 -- 172.21.15.105:0/813005446 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6ac06f6a0 msgr2=0x7fa6ac0b4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:26.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.059+0000 7fa6ab7fe640 1 --2- 172.21.15.105:0/813005446 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6ac06f6a0 0x7fa6ac0b4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.059+0000 7fa6ab7fe640 1 -- 172.21.15.105:0/813005446 >> v1:172.21.15.154:6789/0 conn(0x7fa6ac144c90 legacy=0x7fa6ac06ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:26.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.059+0000 7fa6ab7fe640 1 -- 172.21.15.105:0/813005446 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6ac0b4660 con 0x7fa6ac0cd1d0 2024-04-15T15:20:26.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.059+0000 7fa6ab7fe640 1 -- 172.21.15.105:0/813005446 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7fa6a40019f0 con 0x7fa6ac0cd1d0 2024-04-15T15:20:26.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.059+0000 7fa6ab7fe640 1 -- 172.21.15.105:0/813005446 >> v1:172.21.15.105:6789/0 conn(0x7fa6ac0cd1d0 legacy=0x7fa6ac0cd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:26.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.060+0000 7fa6ab7fe640 1 --2- 172.21.15.105:0/813005446 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa69c0039e0 0x7fa69c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:26.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.060+0000 7fa6ab7fe640 1 --2- 172.21.15.105:0/813005446 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6ac06f6a0 0x7fa69c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:26.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.060+0000 7fa6ab7fe640 1 --2- 172.21.15.105:0/813005446 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa69c004d40 0x7fa69c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:26.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.060+0000 7fa6ab7fe640 1 -- 172.21.15.105:0/813005446 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ac0b3e50 con 0x7fa69c0039e0 2024-04-15T15:20:26.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.060+0000 7fa6ab7fe640 1 -- 172.21.15.105:0/813005446 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa69c00baa0 con 0x7fa69c004d40 2024-04-15T15:20:26.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.060+0000 7fa6ab7fe640 1 -- 172.21.15.105:0/813005446 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fa69c00deb0 con 0x7fa6ac06f6a0 2024-04-15T15:20:26.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.060+0000 7fa6b0f2b640 1 --2- 172.21.15.105:0/813005446 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa69c0039e0 0x7fa69c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:26.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.060+0000 7fa6b172c640 1 --2- 172.21.15.105:0/813005446 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa69c004d40 0x7fa69c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:26.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.060+0000 7fa6abfff640 1 --2- 172.21.15.105:0/813005446 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6ac06f6a0 0x7fa69c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:26.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.060+0000 7fa6b0f2b640 1 -- 172.21.15.105:0/813005446 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6ac06f6a0 msgr2=0x7fa69c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:26.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.060+0000 7fa6b0f2b640 1 --2- 172.21.15.105:0/813005446 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6ac06f6a0 0x7fa69c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.060+0000 7fa6b0f2b640 1 -- 172.21.15.105:0/813005446 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa69c004d40 msgr2=0x7fa69c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:26.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.060+0000 7fa6b0f2b640 1 --2- 172.21.15.105:0/813005446 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa69c004d40 0x7fa69c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.060+0000 7fa6b0f2b640 1 -- 172.21.15.105:0/813005446 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa69c003520 con 0x7fa69c0039e0 2024-04-15T15:20:26.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.060+0000 7fa6b172c640 1 --2- 172.21.15.105:0/813005446 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa69c004d40 0x7fa69c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:20:26.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.060+0000 7fa6abfff640 1 --2- 172.21.15.105:0/813005446 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6ac06f6a0 0x7fa69c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:20:26.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.061+0000 7fa6b0f2b640 1 --2- 172.21.15.105:0/813005446 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa69c0039e0 0x7fa69c003dc0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0001860 tx=0x7fa69c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c372e40249e6478c server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:26.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.061+0000 7fa6ab7fe640 1 -- 172.21.15.105:0/813005446 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a002f050 con 0x7fa69c0039e0 2024-04-15T15:20:26.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.061+0000 7fa6ab7fe640 1 -- 172.21.15.105:0/813005446 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6a0033650 con 0x7fa69c0039e0 2024-04-15T15:20:26.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.061+0000 7fa6ab7fe640 1 -- 172.21.15.105:0/813005446 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a0001430 con 0x7fa69c0039e0 2024-04-15T15:20:26.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.061+0000 7fa6b318f640 1 -- 172.21.15.105:0/813005446 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa69c0039e0 msgr2=0x7fa69c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:26.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.061+0000 7fa6b318f640 1 --2- 172.21.15.105:0/813005446 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa69c0039e0 0x7fa69c003dc0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0001860 tx=0x7fa69c005830 comp rx=0 tx=0).stop 2024-04-15T15:20:26.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.061+0000 7fa6b172c640 1 -- 172.21.15.105:0/813005446 reap_dead start 2024-04-15T15:20:26.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.061+0000 7fa6b318f640 1 -- 172.21.15.105:0/813005446 shutdown_connections 2024-04-15T15:20:26.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.061+0000 7fa6b318f640 1 -- 172.21.15.105:0/813005446 >> 172.21.15.105:0/813005446 conn(0x7fa6ac067d50 msgr2=0x7fa6ac06b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:26.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.061+0000 7fa6b318f640 1 -- 172.21.15.105:0/813005446 shutdown_connections 2024-04-15T15:20:26.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.061+0000 7fa6b318f640 1 -- 172.21.15.105:0/813005446 wait complete. 2024-04-15T15:20:26.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.063+0000 7fa6b318f640 1 Processor -- start 2024-04-15T15:20:26.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.063+0000 7fa6b318f640 1 -- start start 2024-04-15T15:20:26.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.063+0000 7fa6b318f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa69c0039e0 0x7fa6ac15bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:26.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.063+0000 7fa6b318f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6ac154ca0 0x7fa6ac0ae7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:26.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.063+0000 7fa6b318f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6ac155120 0x7fa6ac15e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:26.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.063+0000 7fa6b318f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ac1451f0 con 0x7fa6ac154ca0 2024-04-15T15:20:26.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.063+0000 7fa6b318f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ac145070 con 0x7fa69c0039e0 2024-04-15T15:20:26.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.063+0000 7fa6b318f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fa6ac0b4e60 con 0x7fa6ac155120 2024-04-15T15:20:26.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.064+0000 7fa6abfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6ac154ca0 0x7fa6ac0ae7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:26.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.064+0000 7fa6abfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6ac154ca0 0x7fa6ac0ae7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54820/0 (socket says 172.21.15.105:54820) 2024-04-15T15:20:26.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.064+0000 7fa6abfff640 1 -- 172.21.15.105:0/3038890084 learned_addr learned my addr 172.21.15.105:0/3038890084 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:26.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.064+0000 7fa6b0f2b640 1 --2- 172.21.15.105:0/3038890084 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa69c0039e0 0x7fa6ac15bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:26.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.064+0000 7fa6b172c640 1 --2- 172.21.15.105:0/3038890084 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6ac155120 0x7fa6ac15e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:26.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.064+0000 7fa6abfff640 1 -- 172.21.15.105:0/3038890084 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6ac155120 msgr2=0x7fa6ac15e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:26.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.064+0000 7fa6abfff640 1 --2- 172.21.15.105:0/3038890084 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6ac155120 0x7fa6ac15e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.064+0000 7fa6abfff640 1 -- 172.21.15.105:0/3038890084 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa69c0039e0 msgr2=0x7fa6ac15bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:26.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.064+0000 7fa6abfff640 1 --2- 172.21.15.105:0/3038890084 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa69c0039e0 0x7fa6ac15bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.064+0000 7fa6abfff640 1 -- 172.21.15.105:0/3038890084 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6ac13cb30 con 0x7fa6ac154ca0 2024-04-15T15:20:26.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.064+0000 7fa6abfff640 1 --2- 172.21.15.105:0/3038890084 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6ac154ca0 0x7fa6ac0ae7d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa6980116d0 tx=0x7fa698011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:26.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.064+0000 7fa6a97fa640 1 -- 172.21.15.105:0/3038890084 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69800f9c0 con 0x7fa6ac154ca0 2024-04-15T15:20:26.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.064+0000 7fa6a97fa640 1 -- 172.21.15.105:0/3038890084 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa69800fb60 con 0x7fa6ac154ca0 2024-04-15T15:20:26.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.064+0000 7fa6b318f640 1 -- 172.21.15.105:0/3038890084 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6ac13cdb0 con 0x7fa6ac154ca0 2024-04-15T15:20:26.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.064+0000 7fa6a97fa640 1 -- 172.21.15.105:0/3038890084 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa698015460 con 0x7fa6ac154ca0 2024-04-15T15:20:26.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.064+0000 7fa6b318f640 1 -- 172.21.15.105:0/3038890084 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6ac13d370 con 0x7fa6ac154ca0 2024-04-15T15:20:26.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.065+0000 7fa6b318f640 1 -- 172.21.15.105:0/3038890084 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6ac068ab0 con 0x7fa6ac154ca0 2024-04-15T15:20:26.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.066+0000 7fa6a97fa640 1 -- 172.21.15.105:0/3038890084 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa698015600 con 0x7fa6ac154ca0 2024-04-15T15:20:26.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.066+0000 7fa6a97fa640 1 --2- 172.21.15.105:0/3038890084 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fa69006c920 0x7fa69006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:26.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.066+0000 7fa6a97fa640 1 -- 172.21.15.105:0/3038890084 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7fa69808c0a0 con 0x7fa6ac154ca0 2024-04-15T15:20:26.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.066+0000 7fa6b0f2b640 1 --2- 172.21.15.105:0/3038890084 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fa69006c920 0x7fa69006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:26.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.067+0000 7fa6b0f2b640 1 --2- 172.21.15.105:0/3038890084 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fa69006c920 0x7fa69006ede0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0036000 tx=0x7fa6a0035040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:26.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.070+0000 7fa6a97fa640 1 -- 172.21.15.105:0/3038890084 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa6ac13cb30 con 0x7fa6ac154ca0 2024-04-15T15:20:26.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.246+0000 7fa6b318f640 1 -- 172.21.15.105:0/3038890084 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa6ac0cc870 con 0x7fa6ac154ca0 2024-04-15T15:20:26.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.246+0000 7fa6a97fa640 1 -- 172.21.15.105:0/3038890084 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) ==== 93+0+10 (secure 0 0 0) 0x7fa698019030 con 0x7fa6ac154ca0 2024-04-15T15:20:26.248 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-04-15T15:20:26.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.248+0000 7fa6b318f640 1 -- 172.21.15.105:0/3038890084 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fa69006c920 msgr2=0x7fa69006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:26.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.248+0000 7fa6b318f640 1 --2- 172.21.15.105:0/3038890084 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fa69006c920 0x7fa69006ede0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0036000 tx=0x7fa6a0035040 comp rx=0 tx=0).stop 2024-04-15T15:20:26.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.248+0000 7fa6b318f640 1 -- 172.21.15.105:0/3038890084 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6ac154ca0 msgr2=0x7fa6ac0ae7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:26.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.248+0000 7fa6b318f640 1 --2- 172.21.15.105:0/3038890084 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6ac154ca0 0x7fa6ac0ae7d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa6980116d0 tx=0x7fa698011b90 comp rx=0 tx=0).stop 2024-04-15T15:20:26.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.248+0000 7fa6b318f640 1 -- 172.21.15.105:0/3038890084 shutdown_connections 2024-04-15T15:20:26.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.248+0000 7fa6b318f640 1 --2- 172.21.15.105:0/3038890084 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6ac155120 0x7fa6ac15e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.248+0000 7fa6b318f640 1 --2- 172.21.15.105:0/3038890084 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fa69006c920 0x7fa69006ede0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.248+0000 7fa6b318f640 1 --2- 172.21.15.105:0/3038890084 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6ac154ca0 0x7fa6ac0ae7d0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.248+0000 7fa6b318f640 1 --2- 172.21.15.105:0/3038890084 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa69c0039e0 0x7fa6ac15bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.248+0000 7fa6b318f640 1 -- 172.21.15.105:0/3038890084 >> 172.21.15.105:0/3038890084 conn(0x7fa6ac067d50 msgr2=0x7fa6ac06e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:26.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.248+0000 7fa6b318f640 1 -- 172.21.15.105:0/3038890084 shutdown_connections 2024-04-15T15:20:26.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.248+0000 7fa6b318f640 1 -- 172.21.15.105:0/3038890084 wait complete. 2024-04-15T15:20:26.261 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-15T15:20:26.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.363+0000 7fbdf04ba640 1 Processor -- start 2024-04-15T15:20:26.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.363+0000 7fbdf04ba640 1 -- start start 2024-04-15T15:20:26.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.364+0000 7fbdf04ba640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbde816b740 0x7fbde806de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:26.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.364+0000 7fbdf04ba640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbde80ceeb0 0x7fbde8072280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:26.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.364+0000 7fbdf04ba640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbde80cf310 0x7fbde80727c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:26.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.364+0000 7fbdf04ba640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fbde806f070 con 0x7fbde80ceeb0 2024-04-15T15:20:26.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.364+0000 7fbdf04ba640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fbde806e460 con 0x7fbde80cf310 2024-04-15T15:20:26.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.364+0000 7fbdf04ba640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fbde806e740 con 0x7fbde816b740 2024-04-15T15:20:26.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.364+0000 7fbdeda55640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbde80ceeb0 0x7fbde8072280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:26.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.364+0000 7fbdeda55640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbde80ceeb0 0x7fbde8072280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54826/0 (socket says 172.21.15.105:54826) 2024-04-15T15:20:26.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.364+0000 7fbdeda55640 1 -- 172.21.15.105:0/3215101827 learned_addr learned my addr 172.21.15.105:0/3215101827 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:26.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.364+0000 7fbdeea57640 1 --2- 172.21.15.105:0/3215101827 >> v2:172.21.15.154:3300/0 conn(0x7fbde80cf310 0x7fbde80727c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:26.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.364+0000 7fbdee256640 1 --2- 172.21.15.105:0/3215101827 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbde816b740 0x7fbde806de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:26.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.484+0000 7fbdeda55640 1 -- 172.21.15.105:0/3215101827 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbde816b740 msgr2=0x7fbde806de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:26.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.484+0000 7fbdeda55640 1 --2- 172.21.15.105:0/3215101827 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbde816b740 0x7fbde806de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.484+0000 7fbdeda55640 1 -- 172.21.15.105:0/3215101827 >> v2:172.21.15.154:3300/0 conn(0x7fbde80cf310 msgr2=0x7fbde80727c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:26.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.484+0000 7fbdeda55640 1 --2- 172.21.15.105:0/3215101827 >> v2:172.21.15.154:3300/0 conn(0x7fbde80cf310 0x7fbde80727c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.484+0000 7fbdeda55640 1 -- 172.21.15.105:0/3215101827 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbde806ea20 con 0x7fbde80ceeb0 2024-04-15T15:20:26.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.484+0000 7fbdeda55640 1 --2- 172.21.15.105:0/3215101827 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbde80ceeb0 0x7fbde8072280 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fbde8072e70 tx=0x7fbdd802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c3fa0df19b7f245 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:26.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.485+0000 7fbded254640 1 -- 172.21.15.105:0/3215101827 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd8039070 con 0x7fbde80ceeb0 2024-04-15T15:20:26.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.485+0000 7fbded254640 1 -- 172.21.15.105:0/3215101827 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdd802cc60 con 0x7fbde80ceeb0 2024-04-15T15:20:26.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.485+0000 7fbded254640 1 -- 172.21.15.105:0/3215101827 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd8034660 con 0x7fbde80ceeb0 2024-04-15T15:20:26.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.485+0000 7fbdf04ba640 1 -- 172.21.15.105:0/3215101827 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbde80ceeb0 msgr2=0x7fbde8072280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:26.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.485+0000 7fbdf04ba640 1 --2- 172.21.15.105:0/3215101827 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbde80ceeb0 0x7fbde8072280 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fbde8072e70 tx=0x7fbdd802beb0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.485+0000 7fbdf04ba640 1 -- 172.21.15.105:0/3215101827 shutdown_connections 2024-04-15T15:20:26.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.485+0000 7fbdf04ba640 1 --2- 172.21.15.105:0/3215101827 >> v2:172.21.15.154:3300/0 conn(0x7fbde80cf310 0x7fbde80727c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.485+0000 7fbdf04ba640 1 --2- 172.21.15.105:0/3215101827 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbde80ceeb0 0x7fbde8072280 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.485+0000 7fbdf04ba640 1 --2- 172.21.15.105:0/3215101827 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbde816b740 0x7fbde806de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.485+0000 7fbdf04ba640 1 -- 172.21.15.105:0/3215101827 >> 172.21.15.105:0/3215101827 conn(0x7fbde806bde0 msgr2=0x7fbde806c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:26.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.485+0000 7fbdf04ba640 1 -- 172.21.15.105:0/3215101827 shutdown_connections 2024-04-15T15:20:26.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.485+0000 7fbdf04ba640 1 -- 172.21.15.105:0/3215101827 wait complete. 2024-04-15T15:20:26.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.486+0000 7fbdf04ba640 1 Processor -- start 2024-04-15T15:20:26.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.487+0000 7fbdf04ba640 1 -- start start 2024-04-15T15:20:26.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.487+0000 7fbdf04ba640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbde80ceeb0 0x7fbde80b1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:26.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.487+0000 7fbdf04ba640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbde80cf310 0x7fbde806fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:26.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.487+0000 7fbdf04ba640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbde816b740 0x7fbde80700a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:26.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.487+0000 7fbdf04ba640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbde80b3660 con 0x7fbde816b740 2024-04-15T15:20:26.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.487+0000 7fbdf04ba640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbde80b34e0 con 0x7fbde80cf310 2024-04-15T15:20:26.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.487+0000 7fbdf04ba640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fbde80b37e0 con 0x7fbde80ceeb0 2024-04-15T15:20:26.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.487+0000 7fbdeea57640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbde816b740 0x7fbde80700a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:26.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.487+0000 7fbdeea57640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbde816b740 0x7fbde80700a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54840/0 (socket says 172.21.15.105:54840) 2024-04-15T15:20:26.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.487+0000 7fbdeea57640 1 -- 172.21.15.105:0/1793862683 learned_addr learned my addr 172.21.15.105:0/1793862683 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:26.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.488+0000 7fbdee256640 1 --2- 172.21.15.105:0/1793862683 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbde80ceeb0 0x7fbde80b1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:26.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.488+0000 7fbdeda55640 1 --2- 172.21.15.105:0/1793862683 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbde80cf310 0x7fbde806fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:26.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.488+0000 7fbdeea57640 1 -- 172.21.15.105:0/1793862683 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbde80ceeb0 msgr2=0x7fbde80b1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:26.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.488+0000 7fbdeea57640 1 --2- 172.21.15.105:0/1793862683 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbde80ceeb0 0x7fbde80b1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.488+0000 7fbdeea57640 1 -- 172.21.15.105:0/1793862683 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbde80cf310 msgr2=0x7fbde806fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:26.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.488+0000 7fbdeea57640 1 --2- 172.21.15.105:0/1793862683 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbde80cf310 0x7fbde806fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.488+0000 7fbdeea57640 1 -- 172.21.15.105:0/1793862683 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbde815bc30 con 0x7fbde816b740 2024-04-15T15:20:26.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.488+0000 7fbdeda55640 1 --2- 172.21.15.105:0/1793862683 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbde80cf310 0x7fbde806fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:26.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.488+0000 7fbdeea57640 1 --2- 172.21.15.105:0/1793862683 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbde816b740 0x7fbde80700a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fbde000fb90 tx=0x7fbde000d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:26.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.488+0000 7fbddeffd640 1 -- 172.21.15.105:0/1793862683 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde0018020 con 0x7fbde816b740 2024-04-15T15:20:26.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.488+0000 7fbddeffd640 1 -- 172.21.15.105:0/1793862683 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbde000fe90 con 0x7fbde816b740 2024-04-15T15:20:26.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.488+0000 7fbdf04ba640 1 -- 172.21.15.105:0/1793862683 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbde815e870 con 0x7fbde816b740 2024-04-15T15:20:26.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.488+0000 7fbddeffd640 1 -- 172.21.15.105:0/1793862683 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde000de70 con 0x7fbde816b740 2024-04-15T15:20:26.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.488+0000 7fbdf04ba640 1 -- 172.21.15.105:0/1793862683 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbde815c1d0 con 0x7fbde816b740 2024-04-15T15:20:26.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.489+0000 7fbdf04ba640 1 -- 172.21.15.105:0/1793862683 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbde8068a40 con 0x7fbde816b740 2024-04-15T15:20:26.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.489+0000 7fbddeffd640 1 -- 172.21.15.105:0/1793862683 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbde001b020 con 0x7fbde816b740 2024-04-15T15:20:26.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.490+0000 7fbddeffd640 1 --2- 172.21.15.105:0/1793862683 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fbdcc06c8f0 0x7fbdcc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:26.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.490+0000 7fbddeffd640 1 -- 172.21.15.105:0/1793862683 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7fbde008bd80 con 0x7fbde816b740 2024-04-15T15:20:26.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.490+0000 7fbdee256640 1 --2- 172.21.15.105:0/1793862683 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fbdcc06c8f0 0x7fbdcc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:26.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.490+0000 7fbdee256640 1 --2- 172.21.15.105:0/1793862683 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fbdcc06c8f0 0x7fbdcc06edb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbde4004230 tx=0x7fbde400d590 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:26.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.494+0000 7fbddeffd640 1 -- 172.21.15.105:0/1793862683 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbde0022030 con 0x7fbde816b740 2024-04-15T15:20:26.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.674+0000 7fbdf04ba640 1 -- 172.21.15.105:0/1793862683 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fbde806f640 con 0x7fbde816b740 2024-04-15T15:20:26.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.674+0000 7fbddeffd640 1 -- 172.21.15.105:0/1793862683 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v30) ==== 84+0+65 (secure 0 0 0) 0x7fbde005a500 con 0x7fbde816b740 2024-04-15T15:20:26.676 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:20:26.676 INFO:teuthology.orchestra.run.smithi105.stdout:[ 2024-04-15T15:20:26.676 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-04-15T15:20:26.677 INFO:teuthology.orchestra.run.smithi105.stdout: "poolnum": 1, 2024-04-15T15:20:26.677 INFO:teuthology.orchestra.run.smithi105.stdout: "poolname": ".mgr" 2024-04-15T15:20:26.677 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-04-15T15:20:26.677 INFO:teuthology.orchestra.run.smithi105.stdout:] 2024-04-15T15:20:26.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.676+0000 7fbdf04ba640 1 -- 172.21.15.105:0/1793862683 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fbdcc06c8f0 msgr2=0x7fbdcc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:26.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.676+0000 7fbdf04ba640 1 --2- 172.21.15.105:0/1793862683 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fbdcc06c8f0 0x7fbdcc06edb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbde4004230 tx=0x7fbde400d590 comp rx=0 tx=0).stop 2024-04-15T15:20:26.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.676+0000 7fbdf04ba640 1 -- 172.21.15.105:0/1793862683 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbde816b740 msgr2=0x7fbde80700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:26.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.676+0000 7fbdf04ba640 1 --2- 172.21.15.105:0/1793862683 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbde816b740 0x7fbde80700a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fbde000fb90 tx=0x7fbde000d520 comp rx=0 tx=0).stop 2024-04-15T15:20:26.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.676+0000 7fbdf04ba640 1 -- 172.21.15.105:0/1793862683 shutdown_connections 2024-04-15T15:20:26.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.676+0000 7fbdf04ba640 1 --2- 172.21.15.105:0/1793862683 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fbdcc06c8f0 0x7fbdcc06edb0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.676+0000 7fbdf04ba640 1 --2- 172.21.15.105:0/1793862683 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbde816b740 0x7fbde80700a0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.676+0000 7fbdf04ba640 1 --2- 172.21.15.105:0/1793862683 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbde80cf310 0x7fbde806fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.676+0000 7fbdf04ba640 1 --2- 172.21.15.105:0/1793862683 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbde80ceeb0 0x7fbde80b1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.676+0000 7fbdf04ba640 1 -- 172.21.15.105:0/1793862683 >> 172.21.15.105:0/1793862683 conn(0x7fbde806bde0 msgr2=0x7fbde816bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:26.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.676+0000 7fbdf04ba640 1 -- 172.21.15.105:0/1793862683 shutdown_connections 2024-04-15T15:20:26.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.676+0000 7fbdf04ba640 1 -- 172.21.15.105:0/1793862683 wait complete. 2024-04-15T15:20:26.688 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-15T15:20:26.689 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-15T15:20:26.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.793+0000 7ff93b167640 1 Processor -- start 2024-04-15T15:20:26.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.793+0000 7ff93b167640 1 -- start start 2024-04-15T15:20:26.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.794+0000 7ff93b167640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff9340d0820 0x7ff93409b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:26.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.794+0000 7ff93b167640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff93415e250 0x7ff93415e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:26.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.794+0000 7ff93b167640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9340d0f90 con 0x7ff93409aa40 2024-04-15T15:20:26.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.794+0000 7ff93b167640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7ff93409be40 con 0x7ff9340d0820 2024-04-15T15:20:26.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.794+0000 7ff93b167640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7ff9340d1270 con 0x7ff93415e250 2024-04-15T15:20:26.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.794+0000 7ff92bfff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff9340d0820 0x7ff93409b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:26.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.794+0000 7ff92bfff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff9340d0820 0x7ff93409b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54848/0 (socket says 172.21.15.105:54848) 2024-04-15T15:20:26.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.794+0000 7ff92bfff640 1 -- 172.21.15.105:0/993161325 learned_addr learned my addr 172.21.15.105:0/993161325 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:26.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.795+0000 7ff939704640 1 --2- 172.21.15.105:0/993161325 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff93415e250 0x7ff93415e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:26.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.795+0000 7ff92bfff640 1 -- 172.21.15.105:0/993161325 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff93415e250 msgr2=0x7ff93415e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:26.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.795+0000 7ff92bfff640 1 --2- 172.21.15.105:0/993161325 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff93415e250 0x7ff93415e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.795+0000 7ff92bfff640 1 -- 172.21.15.105:0/993161325 >> v1:172.21.15.154:6789/0 conn(0x7ff93409aa40 legacy=0x7ff93409ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:26.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.795+0000 7ff92bfff640 1 -- 172.21.15.105:0/993161325 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9340d1550 con 0x7ff9340d0820 2024-04-15T15:20:26.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.795+0000 7ff92bfff640 1 --2- 172.21.15.105:0/993161325 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9340d0820 0x7ff93409b900 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff9240098a0 tx=0x7ff92402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f338a9e8cd5723f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:26.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.796+0000 7ff92b7fe640 1 -- 172.21.15.105:0/993161325 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff924035020 con 0x7ff9340d0820 2024-04-15T15:20:26.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.796+0000 7ff92b7fe640 1 -- 172.21.15.105:0/993161325 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff924031640 con 0x7ff9340d0820 2024-04-15T15:20:26.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.796+0000 7ff92b7fe640 1 -- 172.21.15.105:0/993161325 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92402d040 con 0x7ff9340d0820 2024-04-15T15:20:26.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.796+0000 7ff93b167640 1 -- 172.21.15.105:0/993161325 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9340d0820 msgr2=0x7ff93409b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:26.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.796+0000 7ff93b167640 1 --2- 172.21.15.105:0/993161325 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9340d0820 0x7ff93409b900 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff9240098a0 tx=0x7ff92402b8f0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.796+0000 7ff93b167640 1 -- 172.21.15.105:0/993161325 shutdown_connections 2024-04-15T15:20:26.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.796+0000 7ff93b167640 1 --2- 172.21.15.105:0/993161325 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff93415e250 0x7ff93415e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.796+0000 7ff93b167640 1 --2- 172.21.15.105:0/993161325 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9340d0820 0x7ff93409b900 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.796+0000 7ff93b167640 1 -- 172.21.15.105:0/993161325 >> 172.21.15.105:0/993161325 conn(0x7ff9340b40d0 msgr2=0x7ff9340b44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:26.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.796+0000 7ff93b167640 1 -- 172.21.15.105:0/993161325 shutdown_connections 2024-04-15T15:20:26.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.796+0000 7ff93b167640 1 -- 172.21.15.105:0/993161325 wait complete. 2024-04-15T15:20:26.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.797+0000 7ff93b167640 1 Processor -- start 2024-04-15T15:20:26.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.797+0000 7ff93b167640 1 -- start start 2024-04-15T15:20:26.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.798+0000 7ff93b167640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff93409aa40 0x7ff9340ca940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:26.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.798+0000 7ff93b167640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9340d0820 0x7ff934161900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:26.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.798+0000 7ff93b167640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff93415e250 0x7ff934156c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:26.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.798+0000 7ff93b167640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff9340d0cb0 con 0x7ff9340d0820 2024-04-15T15:20:26.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.798+0000 7ff93b167640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff9340d1b10 con 0x7ff93415e250 2024-04-15T15:20:26.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.798+0000 7ff93b167640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7ff9340b3e10 con 0x7ff93409aa40 2024-04-15T15:20:26.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.798+0000 7ff92bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9340d0820 0x7ff934161900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:26.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.798+0000 7ff92bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9340d0820 0x7ff934161900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54862/0 (socket says 172.21.15.105:54862) 2024-04-15T15:20:26.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.798+0000 7ff92bfff640 1 -- 172.21.15.105:0/3218832042 learned_addr learned my addr 172.21.15.105:0/3218832042 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:26.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.799+0000 7ff938f03640 1 --2- 172.21.15.105:0/3218832042 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff93409aa40 0x7ff9340ca940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:26.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.799+0000 7ff939704640 1 --2- 172.21.15.105:0/3218832042 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff93415e250 0x7ff934156c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:26.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.799+0000 7ff92bfff640 1 -- 172.21.15.105:0/3218832042 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff93409aa40 msgr2=0x7ff9340ca940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:26.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.799+0000 7ff92bfff640 1 --2- 172.21.15.105:0/3218832042 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff93409aa40 0x7ff9340ca940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.799+0000 7ff92bfff640 1 -- 172.21.15.105:0/3218832042 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff93415e250 msgr2=0x7ff934156c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:26.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.799+0000 7ff92bfff640 1 --2- 172.21.15.105:0/3218832042 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff93415e250 0x7ff934156c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.799+0000 7ff92bfff640 1 -- 172.21.15.105:0/3218832042 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9341f82c0 con 0x7ff9340d0820 2024-04-15T15:20:26.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.799+0000 7ff92bfff640 1 --2- 172.21.15.105:0/3218832042 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9340d0820 0x7ff934161900 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff924000c00 tx=0x7ff9240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:26.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.799+0000 7ff9297fa640 1 -- 172.21.15.105:0/3218832042 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff924040070 con 0x7ff9340d0820 2024-04-15T15:20:26.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.799+0000 7ff9297fa640 1 -- 172.21.15.105:0/3218832042 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9240399f0 con 0x7ff9340d0820 2024-04-15T15:20:26.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.799+0000 7ff93b167640 1 -- 172.21.15.105:0/3218832042 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9341f85a0 con 0x7ff9340d0820 2024-04-15T15:20:26.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.799+0000 7ff9297fa640 1 -- 172.21.15.105:0/3218832042 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff924035040 con 0x7ff9340d0820 2024-04-15T15:20:26.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.799+0000 7ff93b167640 1 -- 172.21.15.105:0/3218832042 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9341f8b60 con 0x7ff9340d0820 2024-04-15T15:20:26.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.800+0000 7ff93b167640 1 -- 172.21.15.105:0/3218832042 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff93406e1a0 con 0x7ff9340d0820 2024-04-15T15:20:26.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.801+0000 7ff9297fa640 1 -- 172.21.15.105:0/3218832042 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff924038020 con 0x7ff9340d0820 2024-04-15T15:20:26.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.801+0000 7ff9297fa640 1 --2- 172.21.15.105:0/3218832042 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff92006c8f0 0x7ff92006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:26.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.801+0000 7ff9297fa640 1 -- 172.21.15.105:0/3218832042 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7ff9240ad050 con 0x7ff9340d0820 2024-04-15T15:20:26.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.801+0000 7ff938f03640 1 --2- 172.21.15.105:0/3218832042 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff92006c8f0 0x7ff92006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:26.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.801+0000 7ff938f03640 1 --2- 172.21.15.105:0/3218832042 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff92006c8f0 0x7ff92006edb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff930006fd0 tx=0x7ff930009060 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:26.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.805+0000 7ff9297fa640 1 -- 172.21.15.105:0/3218832042 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff93406e1a0 con 0x7ff9340d0820 2024-04-15T15:20:26.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:26.979+0000 7ff93b167640 1 -- 172.21.15.105:0/3218832042 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7ff934069150 con 0x7ff9340d0820 2024-04-15T15:20:27.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.636+0000 7ff9297fa640 1 -- 172.21.15.105:0/3218832042 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v31) ==== 138+0+0 (secure 0 0 0) 0x7ff92407b180 con 0x7ff9340d0820 2024-04-15T15:20:27.639 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_metadata' created 2024-04-15T15:20:27.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.639+0000 7ff93b167640 1 -- 172.21.15.105:0/3218832042 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff92006c8f0 msgr2=0x7ff92006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:27.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.639+0000 7ff93b167640 1 --2- 172.21.15.105:0/3218832042 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff92006c8f0 0x7ff92006edb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff930006fd0 tx=0x7ff930009060 comp rx=0 tx=0).stop 2024-04-15T15:20:27.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.639+0000 7ff93b167640 1 -- 172.21.15.105:0/3218832042 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9340d0820 msgr2=0x7ff934161900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:27.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.639+0000 7ff93b167640 1 --2- 172.21.15.105:0/3218832042 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9340d0820 0x7ff934161900 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff924000c00 tx=0x7ff9240046c0 comp rx=0 tx=0).stop 2024-04-15T15:20:27.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.639+0000 7ff93b167640 1 -- 172.21.15.105:0/3218832042 shutdown_connections 2024-04-15T15:20:27.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.639+0000 7ff93b167640 1 --2- 172.21.15.105:0/3218832042 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff92006c8f0 0x7ff92006edb0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:27.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.639+0000 7ff93b167640 1 --2- 172.21.15.105:0/3218832042 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff93415e250 0x7ff934156c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:27.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.639+0000 7ff93b167640 1 --2- 172.21.15.105:0/3218832042 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9340d0820 0x7ff934161900 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:27.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.639+0000 7ff93b167640 1 --2- 172.21.15.105:0/3218832042 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff93409aa40 0x7ff9340ca940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:27.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.639+0000 7ff93b167640 1 -- 172.21.15.105:0/3218832042 >> 172.21.15.105:0/3218832042 conn(0x7ff9340b40d0 msgr2=0x7ff9340b7a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:27.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.639+0000 7ff93b167640 1 -- 172.21.15.105:0/3218832042 shutdown_connections 2024-04-15T15:20:27.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.639+0000 7ff93b167640 1 -- 172.21.15.105:0/3218832042 wait complete. 2024-04-15T15:20:27.656 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-15T15:20:27.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.768+0000 7f5bf0647640 1 Processor -- start 2024-04-15T15:20:27.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.768+0000 7f5bf0647640 1 -- start start 2024-04-15T15:20:27.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.769+0000 7f5bf0647640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5be815e9f0 0x7f5be806e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:27.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.769+0000 7f5bf0647640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5be80b4890 0x7f5be80b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:27.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.769+0000 7f5bf0647640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5be80aee60 con 0x7f5be80b52d0 2024-04-15T15:20:27.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.769+0000 7f5bf0647640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f5be80af140 con 0x7f5be80b4890 2024-04-15T15:20:27.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.769+0000 7f5bf0647640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f5be80af420 con 0x7f5be815e9f0 2024-04-15T15:20:27.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.769+0000 7f5bedbe2640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5be80b4890 0x7f5be80b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:27.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.769+0000 7f5bee3e3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5be815e9f0 0x7f5be806e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:27.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.769+0000 7f5beebe4640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f5be80b52d0 0x7f5be80aff00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.105:60896/0 (socket says 172.21.15.105:60896) 2024-04-15T15:20:27.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.769+0000 7f5beebe4640 1 -- 172.21.15.105:0/288767580 learned_addr learned my addr 172.21.15.105:0/288767580 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:20:27.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.770+0000 7f5bed3e1640 1 -- 172.21.15.105:0/288767580 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1202314400 0 0) 0x7f5be80aee60 con 0x7f5be80b52d0 2024-04-15T15:20:27.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.770+0000 7f5bed3e1640 1 -- 172.21.15.105:0/288767580 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5bd4003520 con 0x7f5be80b52d0 2024-04-15T15:20:27.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.770+0000 7f5bee3e3640 1 -- 172.21.15.105:0/288767580 >> v2:172.21.15.154:3300/0 conn(0x7f5be80b4890 msgr2=0x7f5be80b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:27.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.770+0000 7f5bee3e3640 1 --2- 172.21.15.105:0/288767580 >> v2:172.21.15.154:3300/0 conn(0x7f5be80b4890 0x7f5be80b4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:27.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.770+0000 7f5bee3e3640 1 -- 172.21.15.105:0/288767580 >> v1:172.21.15.154:6789/0 conn(0x7f5be80b52d0 legacy=0x7f5be80aff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:27.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.770+0000 7f5bee3e3640 1 -- 172.21.15.105:0/288767580 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5be80af700 con 0x7f5be815e9f0 2024-04-15T15:20:27.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.770+0000 7f5bedbe2640 1 --2- 172.21.15.105:0/288767580 >> v2:172.21.15.154:3300/0 conn(0x7f5be80b4890 0x7f5be80b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:20:27.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.771+0000 7f5bee3e3640 1 --2- 172.21.15.105:0/288767580 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5be815e9f0 0x7f5be806e5c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5be4009d90 tx=0x7f5be402be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1bbe6d4c7732032 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:27.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.771+0000 7f5bed3e1640 1 -- 172.21.15.105:0/288767580 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be4039070 con 0x7f5be815e9f0 2024-04-15T15:20:27.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.771+0000 7f5bed3e1640 1 -- 172.21.15.105:0/288767580 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5be402cbc0 con 0x7f5be815e9f0 2024-04-15T15:20:27.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.771+0000 7f5bed3e1640 1 -- 172.21.15.105:0/288767580 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be4034440 con 0x7f5be815e9f0 2024-04-15T15:20:27.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.771+0000 7f5bf0647640 1 -- 172.21.15.105:0/288767580 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5be815e9f0 msgr2=0x7f5be806e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:27.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.771+0000 7f5bf0647640 1 --2- 172.21.15.105:0/288767580 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5be815e9f0 0x7f5be806e5c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5be4009d90 tx=0x7f5be402be10 comp rx=0 tx=0).stop 2024-04-15T15:20:27.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.771+0000 7f5bf0647640 1 -- 172.21.15.105:0/288767580 shutdown_connections 2024-04-15T15:20:27.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.771+0000 7f5bf0647640 1 --2- 172.21.15.105:0/288767580 >> v2:172.21.15.154:3300/0 conn(0x7f5be80b4890 0x7f5be80b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:27.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.771+0000 7f5bf0647640 1 --2- 172.21.15.105:0/288767580 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5be815e9f0 0x7f5be806e5c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:27.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.771+0000 7f5bf0647640 1 -- 172.21.15.105:0/288767580 >> 172.21.15.105:0/288767580 conn(0x7f5be8073570 msgr2=0x7f5be806b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:27.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.771+0000 7f5bf0647640 1 -- 172.21.15.105:0/288767580 shutdown_connections 2024-04-15T15:20:27.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.771+0000 7f5bf0647640 1 -- 172.21.15.105:0/288767580 wait complete. 2024-04-15T15:20:27.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.773+0000 7f5bf0647640 1 Processor -- start 2024-04-15T15:20:27.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.773+0000 7f5bf0647640 1 -- start start 2024-04-15T15:20:27.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.773+0000 7f5bf0647640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5be80b4890 0x7f5be806c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:27.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.773+0000 7f5bf0647640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5be80b52d0 0x7f5be81f4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:27.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.773+0000 7f5bf0647640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5be815e9f0 0x7f5be8158b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:27.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.773+0000 7f5bf0647640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5be806ec80 con 0x7f5be80b4890 2024-04-15T15:20:27.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.773+0000 7f5bf0647640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5be806eb00 con 0x7f5be80b52d0 2024-04-15T15:20:27.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.773+0000 7f5bf0647640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f5be80b0610 con 0x7f5be815e9f0 2024-04-15T15:20:27.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.773+0000 7f5bee3e3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5be80b4890 0x7f5be806c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:27.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.774+0000 7f5bee3e3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5be80b4890 0x7f5be806c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54868/0 (socket says 172.21.15.105:54868) 2024-04-15T15:20:27.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.774+0000 7f5bee3e3640 1 -- 172.21.15.105:0/2128800975 learned_addr learned my addr 172.21.15.105:0/2128800975 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:27.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.774+0000 7f5beebe4640 1 --2- 172.21.15.105:0/2128800975 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5be815e9f0 0x7f5be8158b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:27.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.774+0000 7f5bedbe2640 1 --2- 172.21.15.105:0/2128800975 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5be80b52d0 0x7f5be81f4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:27.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.774+0000 7f5bee3e3640 1 -- 172.21.15.105:0/2128800975 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5be815e9f0 msgr2=0x7f5be8158b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:27.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.774+0000 7f5bee3e3640 1 --2- 172.21.15.105:0/2128800975 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5be815e9f0 0x7f5be8158b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:27.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.774+0000 7f5bee3e3640 1 -- 172.21.15.105:0/2128800975 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5be80b52d0 msgr2=0x7f5be81f4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:27.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.774+0000 7f5bee3e3640 1 --2- 172.21.15.105:0/2128800975 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5be80b52d0 0x7f5be81f4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:27.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.774+0000 7f5bee3e3640 1 -- 172.21.15.105:0/2128800975 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5be81f49a0 con 0x7f5be80b4890 2024-04-15T15:20:27.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.774+0000 7f5bee3e3640 1 --2- 172.21.15.105:0/2128800975 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5be80b4890 0x7f5be806c3a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f5be4000f80 tx=0x7f5be4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:27.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.774+0000 7f5bdeffd640 1 -- 172.21.15.105:0/2128800975 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be4039050 con 0x7f5be80b4890 2024-04-15T15:20:27.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.774+0000 7f5bdeffd640 1 -- 172.21.15.105:0/2128800975 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5be4045070 con 0x7f5be80b4890 2024-04-15T15:20:27.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.774+0000 7f5bf0647640 1 -- 172.21.15.105:0/2128800975 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5be81f4c80 con 0x7f5be80b4890 2024-04-15T15:20:27.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.774+0000 7f5bdeffd640 1 -- 172.21.15.105:0/2128800975 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be4009450 con 0x7f5be80b4890 2024-04-15T15:20:27.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.774+0000 7f5bf0647640 1 -- 172.21.15.105:0/2128800975 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5be81578c0 con 0x7f5be80b4890 2024-04-15T15:20:27.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.775+0000 7f5bf0647640 1 -- 172.21.15.105:0/2128800975 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5be81576b0 con 0x7f5be80b4890 2024-04-15T15:20:27.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.776+0000 7f5bdeffd640 1 -- 172.21.15.105:0/2128800975 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5be404a460 con 0x7f5be80b4890 2024-04-15T15:20:27.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.776+0000 7f5bdeffd640 1 --2- 172.21.15.105:0/2128800975 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f5bcc06c8f0 0x7f5bcc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:27.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.776+0000 7f5bdeffd640 1 -- 172.21.15.105:0/2128800975 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5227+0+0 (secure 0 0 0) 0x7f5be40ae1b0 con 0x7f5be80b4890 2024-04-15T15:20:27.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.776+0000 7f5bedbe2640 1 --2- 172.21.15.105:0/2128800975 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f5bcc06c8f0 0x7f5bcc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:27.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.777+0000 7f5bedbe2640 1 --2- 172.21.15.105:0/2128800975 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f5bcc06c8f0 0x7f5bcc06edb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5be80693a0 tx=0x7f5bd8009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:27.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.781+0000 7f5bdeffd640 1 -- 172.21.15.105:0/2128800975 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5be407c200 con 0x7f5be80b4890 2024-04-15T15:20:27.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:27.957+0000 7f5bf0647640 1 -- 172.21.15.105:0/2128800975 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f5be8068ab0 con 0x7f5be80b4890 2024-04-15T15:20:28.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.643+0000 7f5bdeffd640 1 -- 172.21.15.105:0/2128800975 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v32) ==== 170+0+0 (secure 0 0 0) 0x7f5be4080500 con 0x7f5be80b4890 2024-04-15T15:20:28.645 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_data' created 2024-04-15T15:20:28.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.645+0000 7f5bf0647640 1 -- 172.21.15.105:0/2128800975 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f5bcc06c8f0 msgr2=0x7f5bcc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:28.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.645+0000 7f5bf0647640 1 --2- 172.21.15.105:0/2128800975 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f5bcc06c8f0 0x7f5bcc06edb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5be80693a0 tx=0x7f5bd8009190 comp rx=0 tx=0).stop 2024-04-15T15:20:28.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.645+0000 7f5bf0647640 1 -- 172.21.15.105:0/2128800975 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5be80b4890 msgr2=0x7f5be806c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:28.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.645+0000 7f5bf0647640 1 --2- 172.21.15.105:0/2128800975 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5be80b4890 0x7f5be806c3a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f5be4000f80 tx=0x7f5be4004a40 comp rx=0 tx=0).stop 2024-04-15T15:20:28.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.645+0000 7f5bf0647640 1 -- 172.21.15.105:0/2128800975 shutdown_connections 2024-04-15T15:20:28.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.645+0000 7f5bf0647640 1 --2- 172.21.15.105:0/2128800975 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5be815e9f0 0x7f5be8158b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:28.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.645+0000 7f5bf0647640 1 --2- 172.21.15.105:0/2128800975 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f5bcc06c8f0 0x7f5bcc06edb0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:28.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.645+0000 7f5bf0647640 1 --2- 172.21.15.105:0/2128800975 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5be80b52d0 0x7f5be81f4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:28.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.645+0000 7f5bf0647640 1 --2- 172.21.15.105:0/2128800975 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5be80b4890 0x7f5be806c3a0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:28.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.645+0000 7f5bf0647640 1 -- 172.21.15.105:0/2128800975 >> 172.21.15.105:0/2128800975 conn(0x7f5be8073570 msgr2=0x7f5be80b2380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:28.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.645+0000 7f5bf0647640 1 -- 172.21.15.105:0/2128800975 shutdown_connections 2024-04-15T15:20:28.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.645+0000 7f5bf0647640 1 -- 172.21.15.105:0/2128800975 wait complete. 2024-04-15T15:20:28.667 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-15T15:20:28.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.773+0000 7fd68759e640 1 Processor -- start 2024-04-15T15:20:28.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.773+0000 7fd68759e640 1 -- start start 2024-04-15T15:20:28.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.773+0000 7fd68759e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd68815ea90 0x7fd6880cd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:28.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.773+0000 7fd68759e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd6880b1d00 0x7fd6880d05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:28.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.773+0000 7fd68759e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6880ce5f0 con 0x7fd68809aa40 2024-04-15T15:20:28.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.773+0000 7fd68759e640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fd6880cdd50 con 0x7fd6880b1d00 2024-04-15T15:20:28.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.773+0000 7fd68759e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fd6880ce030 con 0x7fd68815ea90 2024-04-15T15:20:28.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.774+0000 7fd686d9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd6880b1d00 0x7fd6880d05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:28.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.774+0000 7fd686d9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd6880b1d00 0x7fd6880d05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54880/0 (socket says 172.21.15.105:54880) 2024-04-15T15:20:28.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.774+0000 7fd686d9d640 1 -- 172.21.15.105:0/529793250 learned_addr learned my addr 172.21.15.105:0/529793250 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:28.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.774+0000 7fd67ffff640 1 --2- 172.21.15.105:0/529793250 >> v2:172.21.15.154:3300/0 conn(0x7fd68815ea90 0x7fd6880cd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:28.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.774+0000 7fd686d9d640 1 -- 172.21.15.105:0/529793250 >> v2:172.21.15.154:3300/0 conn(0x7fd68815ea90 msgr2=0x7fd6880cd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:28.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.774+0000 7fd686d9d640 1 --2- 172.21.15.105:0/529793250 >> v2:172.21.15.154:3300/0 conn(0x7fd68815ea90 0x7fd6880cd2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:28.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.774+0000 7fd686d9d640 1 -- 172.21.15.105:0/529793250 >> v1:172.21.15.154:6789/0 conn(0x7fd68809aa40 legacy=0x7fd68809ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:28.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.774+0000 7fd686d9d640 1 -- 172.21.15.105:0/529793250 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6880ce310 con 0x7fd6880b1d00 2024-04-15T15:20:28.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.775+0000 7fd686d9d640 1 --2- 172.21.15.105:0/529793250 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6880b1d00 0x7fd6880d05e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd68809b8c0 tx=0x7fd68002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44ae496489f0f199 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:28.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.775+0000 7fd685d9b640 1 -- 172.21.15.105:0/529793250 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68002cb20 con 0x7fd6880b1d00 2024-04-15T15:20:28.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.775+0000 7fd685d9b640 1 -- 172.21.15.105:0/529793250 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd68002ccc0 con 0x7fd6880b1d00 2024-04-15T15:20:28.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.775+0000 7fd685d9b640 1 -- 172.21.15.105:0/529793250 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd680034610 con 0x7fd6880b1d00 2024-04-15T15:20:28.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.775+0000 7fd68759e640 1 -- 172.21.15.105:0/529793250 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6880b1d00 msgr2=0x7fd6880d05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:28.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.775+0000 7fd68759e640 1 --2- 172.21.15.105:0/529793250 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6880b1d00 0x7fd6880d05e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd68809b8c0 tx=0x7fd68002bf80 comp rx=0 tx=0).stop 2024-04-15T15:20:28.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.775+0000 7fd68759e640 1 -- 172.21.15.105:0/529793250 shutdown_connections 2024-04-15T15:20:28.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.775+0000 7fd68759e640 1 --2- 172.21.15.105:0/529793250 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6880b1d00 0x7fd6880d05e0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:28.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.775+0000 7fd68759e640 1 --2- 172.21.15.105:0/529793250 >> v2:172.21.15.154:3300/0 conn(0x7fd68815ea90 0x7fd6880cd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:28.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.775+0000 7fd68759e640 1 -- 172.21.15.105:0/529793250 >> 172.21.15.105:0/529793250 conn(0x7fd6880b3ef0 msgr2=0x7fd6880b4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:28.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.775+0000 7fd68759e640 1 -- 172.21.15.105:0/529793250 shutdown_connections 2024-04-15T15:20:28.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.775+0000 7fd68759e640 1 -- 172.21.15.105:0/529793250 wait complete. 2024-04-15T15:20:28.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.777+0000 7fd68759e640 1 Processor -- start 2024-04-15T15:20:28.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.777+0000 7fd68759e640 1 -- start start 2024-04-15T15:20:28.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.777+0000 7fd68759e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd68809aa40 0x7fd6880ca520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:28.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.777+0000 7fd68759e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6880b1d00 0x7fd688162d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:28.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.777+0000 7fd68759e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd68815ea90 0x7fd68814fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:28.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.777+0000 7fd68759e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd6880cd810 con 0x7fd6880b1d00 2024-04-15T15:20:28.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.777+0000 7fd68759e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd68809bbb0 con 0x7fd68815ea90 2024-04-15T15:20:28.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.777+0000 7fd68759e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fd6880cd990 con 0x7fd68809aa40 2024-04-15T15:20:28.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.778+0000 7fd67ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6880b1d00 0x7fd688162d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:28.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.778+0000 7fd67ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6880b1d00 0x7fd688162d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54894/0 (socket says 172.21.15.105:54894) 2024-04-15T15:20:28.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.778+0000 7fd67ffff640 1 -- 172.21.15.105:0/3928546157 learned_addr learned my addr 172.21.15.105:0/3928546157 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:28.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.778+0000 7fd686d9d640 1 --2- 172.21.15.105:0/3928546157 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd68815ea90 0x7fd68814fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:28.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.778+0000 7fd68659c640 1 --2- 172.21.15.105:0/3928546157 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd68809aa40 0x7fd6880ca520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:28.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.778+0000 7fd67ffff640 1 -- 172.21.15.105:0/3928546157 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd68809aa40 msgr2=0x7fd6880ca520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:28.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.778+0000 7fd67ffff640 1 --2- 172.21.15.105:0/3928546157 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd68809aa40 0x7fd6880ca520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:28.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.778+0000 7fd67ffff640 1 -- 172.21.15.105:0/3928546157 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd68815ea90 msgr2=0x7fd68814fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:28.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.778+0000 7fd67ffff640 1 --2- 172.21.15.105:0/3928546157 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd68815ea90 0x7fd68814fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:28.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.778+0000 7fd67ffff640 1 -- 172.21.15.105:0/3928546157 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6880cabc0 con 0x7fd6880b1d00 2024-04-15T15:20:28.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.778+0000 7fd68659c640 1 --2- 172.21.15.105:0/3928546157 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd68809aa40 0x7fd6880ca520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:20:28.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.778+0000 7fd67ffff640 1 --2- 172.21.15.105:0/3928546157 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6880b1d00 0x7fd688162d90 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd67000d830 tx=0x7fd67000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:28.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.778+0000 7fd67effd640 1 -- 172.21.15.105:0/3928546157 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67000b6c0 con 0x7fd6880b1d00 2024-04-15T15:20:28.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.778+0000 7fd67effd640 1 -- 172.21.15.105:0/3928546157 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd670019070 con 0x7fd6880b1d00 2024-04-15T15:20:28.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.778+0000 7fd68759e640 1 -- 172.21.15.105:0/3928546157 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6880caea0 con 0x7fd6880b1d00 2024-04-15T15:20:28.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.778+0000 7fd67effd640 1 -- 172.21.15.105:0/3928546157 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67000ba80 con 0x7fd6880b1d00 2024-04-15T15:20:28.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.778+0000 7fd68759e640 1 -- 172.21.15.105:0/3928546157 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6880cb460 con 0x7fd6880b1d00 2024-04-15T15:20:28.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.779+0000 7fd68759e640 1 -- 172.21.15.105:0/3928546157 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd688069150 con 0x7fd6880b1d00 2024-04-15T15:20:28.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.780+0000 7fd67effd640 1 -- 172.21.15.105:0/3928546157 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd67000bc20 con 0x7fd6880b1d00 2024-04-15T15:20:28.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.780+0000 7fd67effd640 1 --2- 172.21.15.105:0/3928546157 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fd66806c8f0 0x7fd66806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:28.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.780+0000 7fd68659c640 1 --2- 172.21.15.105:0/3928546157 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fd66806c8f0 0x7fd66806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:28.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.780+0000 7fd67effd640 1 -- 172.21.15.105:0/3928546157 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5609+0+0 (secure 0 0 0) 0x7fd67008bf10 con 0x7fd6880b1d00 2024-04-15T15:20:28.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.780+0000 7fd68659c640 1 --2- 172.21.15.105:0/3928546157 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fd66806c8f0 0x7fd66806edb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd678006fd0 tx=0x7fd678008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:28.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.785+0000 7fd67effd640 1 -- 172.21.15.105:0/3928546157 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd67005a3a0 con 0x7fd6880b1d00 2024-04-15T15:20:29.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:28.999+0000 7fd68759e640 1 -- 172.21.15.105:0/3928546157 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fd6880b4e00 con 0x7fd6880b1d00 2024-04-15T15:20:29.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.644+0000 7fd67effd640 1 -- 172.21.15.105:0/3928546157 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v12) ==== 172+0+0 (secure 0 0 0) 0x7fd670059ca0 con 0x7fd6880b1d00 2024-04-15T15:20:29.646 INFO:teuthology.orchestra.run.smithi105.stderr:new fs with metadata pool 4 and data pool 5 2024-04-15T15:20:29.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.649+0000 7fd68759e640 1 -- 172.21.15.105:0/3928546157 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fd66806c8f0 msgr2=0x7fd66806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:29.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.649+0000 7fd68759e640 1 --2- 172.21.15.105:0/3928546157 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fd66806c8f0 0x7fd66806edb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd678006fd0 tx=0x7fd678008040 comp rx=0 tx=0).stop 2024-04-15T15:20:29.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.649+0000 7fd68759e640 1 -- 172.21.15.105:0/3928546157 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6880b1d00 msgr2=0x7fd688162d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:29.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.649+0000 7fd68759e640 1 --2- 172.21.15.105:0/3928546157 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6880b1d00 0x7fd688162d90 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd67000d830 tx=0x7fd67000dcf0 comp rx=0 tx=0).stop 2024-04-15T15:20:29.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.649+0000 7fd68759e640 1 -- 172.21.15.105:0/3928546157 shutdown_connections 2024-04-15T15:20:29.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.649+0000 7fd68759e640 1 --2- 172.21.15.105:0/3928546157 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fd66806c8f0 0x7fd66806edb0 secure :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd678006fd0 tx=0x7fd678008040 comp rx=0 tx=0).stop 2024-04-15T15:20:29.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.649+0000 7fd68759e640 1 --2- 172.21.15.105:0/3928546157 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd68815ea90 0x7fd68814fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:29.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.649+0000 7fd68759e640 1 --2- 172.21.15.105:0/3928546157 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6880b1d00 0x7fd688162d90 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:29.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.649+0000 7fd68759e640 1 --2- 172.21.15.105:0/3928546157 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd68809aa40 0x7fd6880ca520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:29.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.649+0000 7fd68759e640 1 -- 172.21.15.105:0/3928546157 >> 172.21.15.105:0/3928546157 conn(0x7fd6880b3ef0 msgr2=0x7fd6880702d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:29.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.652+0000 7fd68759e640 1 -- 172.21.15.105:0/3928546157 shutdown_connections 2024-04-15T15:20:29.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.652+0000 7fd68759e640 1 -- 172.21.15.105:0/3928546157 wait complete. 2024-04-15T15:20:29.680 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:20:29.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.783+0000 7f847ad32640 1 Processor -- start 2024-04-15T15:20:29.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.783+0000 7f847ad32640 1 -- start start 2024-04-15T15:20:29.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.784+0000 7f847ad32640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f847415eb60 0x7f84740d11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:29.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.784+0000 7f847ad32640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f84740d1710 0x7f8474162630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:29.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.784+0000 7f847ad32640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8474162b70 0x7f84740b6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:29.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.784+0000 7f847ad32640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f8474163540 con 0x7f84740d1710 2024-04-15T15:20:29.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.784+0000 7f847ad32640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f84740d2fc0 con 0x7f8474162b70 2024-04-15T15:20:29.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.784+0000 7f847ad32640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f84740d1b30 con 0x7f847415eb60 2024-04-15T15:20:29.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.784+0000 7f8473fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f84740d1710 0x7f8474162630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:29.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.784+0000 7f8473fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f84740d1710 0x7f8474162630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54906/0 (socket says 172.21.15.105:54906) 2024-04-15T15:20:29.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.784+0000 7f8473fff640 1 -- 172.21.15.105:0/1878995729 learned_addr learned my addr 172.21.15.105:0/1878995729 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:29.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.784+0000 7f8478ace640 1 --2- 172.21.15.105:0/1878995729 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f847415eb60 0x7f84740d11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:29.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.784+0000 7f84792cf640 1 --2- 172.21.15.105:0/1878995729 >> v2:172.21.15.154:3300/0 conn(0x7f8474162b70 0x7f84740b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:29.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.784+0000 7f8473fff640 1 -- 172.21.15.105:0/1878995729 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f847415eb60 msgr2=0x7f84740d11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:29.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.784+0000 7f8473fff640 1 --2- 172.21.15.105:0/1878995729 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f847415eb60 0x7f84740d11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:29.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.784+0000 7f8473fff640 1 -- 172.21.15.105:0/1878995729 >> v2:172.21.15.154:3300/0 conn(0x7f8474162b70 msgr2=0x7f84740b6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:29.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.784+0000 7f8473fff640 1 --2- 172.21.15.105:0/1878995729 >> v2:172.21.15.154:3300/0 conn(0x7f8474162b70 0x7f84740b6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:29.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.784+0000 7f8473fff640 1 -- 172.21.15.105:0/1878995729 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f847415ef40 con 0x7f84740d1710 2024-04-15T15:20:29.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.785+0000 7f8473fff640 1 --2- 172.21.15.105:0/1878995729 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84740d1710 0x7f8474162630 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f847406e830 tx=0x7f846002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8ef39e25fc97474 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:29.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.785+0000 7f84737fe640 1 -- 172.21.15.105:0/1878995729 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8460035020 con 0x7f84740d1710 2024-04-15T15:20:29.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.785+0000 7f84737fe640 1 -- 172.21.15.105:0/1878995729 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8460031870 con 0x7f84740d1710 2024-04-15T15:20:29.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.785+0000 7f84737fe640 1 -- 172.21.15.105:0/1878995729 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8460031c40 con 0x7f84740d1710 2024-04-15T15:20:29.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.785+0000 7f847ad32640 1 -- 172.21.15.105:0/1878995729 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84740d1710 msgr2=0x7f8474162630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:29.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.785+0000 7f847ad32640 1 --2- 172.21.15.105:0/1878995729 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84740d1710 0x7f8474162630 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f847406e830 tx=0x7f846002b910 comp rx=0 tx=0).stop 2024-04-15T15:20:29.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.786+0000 7f847ad32640 1 -- 172.21.15.105:0/1878995729 shutdown_connections 2024-04-15T15:20:29.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.786+0000 7f847ad32640 1 --2- 172.21.15.105:0/1878995729 >> v2:172.21.15.154:3300/0 conn(0x7f8474162b70 0x7f84740b6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:29.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.786+0000 7f847ad32640 1 --2- 172.21.15.105:0/1878995729 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84740d1710 0x7f8474162630 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:29.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.786+0000 7f847ad32640 1 --2- 172.21.15.105:0/1878995729 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f847415eb60 0x7f84740d11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:29.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.786+0000 7f847ad32640 1 -- 172.21.15.105:0/1878995729 >> 172.21.15.105:0/1878995729 conn(0x7f8474067720 msgr2=0x7f8474067b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:29.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.786+0000 7f847ad32640 1 -- 172.21.15.105:0/1878995729 shutdown_connections 2024-04-15T15:20:29.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.786+0000 7f847ad32640 1 -- 172.21.15.105:0/1878995729 wait complete. 2024-04-15T15:20:29.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.787+0000 7f847ad32640 1 Processor -- start 2024-04-15T15:20:29.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.787+0000 7f847ad32640 1 -- start start 2024-04-15T15:20:29.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.788+0000 7f847ad32640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84740d1710 0x7f84740cd590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:29.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.788+0000 7f847ad32640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f847415eb60 0x7f84740cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:29.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.788+0000 7f847ad32640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8474162b70 0x7f847414ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:29.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.788+0000 7f847ad32640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f84740d2970 con 0x7f8474162b70 2024-04-15T15:20:29.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.788+0000 7f847ad32640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8474162fc0 con 0x7f84740d1710 2024-04-15T15:20:29.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.788+0000 7f847ad32640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f84740d2c50 con 0x7f847415eb60 2024-04-15T15:20:29.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.788+0000 7f84792cf640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8474162b70 0x7f847414ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:29.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.788+0000 7f84792cf640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8474162b70 0x7f847414ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54916/0 (socket says 172.21.15.105:54916) 2024-04-15T15:20:29.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.788+0000 7f84792cf640 1 -- 172.21.15.105:0/3608855741 learned_addr learned my addr 172.21.15.105:0/3608855741 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:29.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.788+0000 7f8473fff640 1 --2- 172.21.15.105:0/3608855741 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f847415eb60 0x7f84740cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:29.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.788+0000 7f8478ace640 1 --2- 172.21.15.105:0/3608855741 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84740d1710 0x7f84740cd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:29.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.788+0000 7f84792cf640 1 -- 172.21.15.105:0/3608855741 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f847415eb60 msgr2=0x7f84740cdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:29.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.788+0000 7f84792cf640 1 --2- 172.21.15.105:0/3608855741 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f847415eb60 0x7f84740cdad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:29.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.788+0000 7f84792cf640 1 -- 172.21.15.105:0/3608855741 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84740d1710 msgr2=0x7f84740cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:29.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.788+0000 7f84792cf640 1 --2- 172.21.15.105:0/3608855741 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84740d1710 0x7f84740cd590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:29.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.788+0000 7f84792cf640 1 -- 172.21.15.105:0/3608855741 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84740ce0e0 con 0x7f8474162b70 2024-04-15T15:20:29.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.788+0000 7f84792cf640 1 --2- 172.21.15.105:0/3608855741 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8474162b70 0x7f847414ca40 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f846c00efc0 tx=0x7f846c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:29.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.788+0000 7f84717fa640 1 -- 172.21.15.105:0/3608855741 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f846c018020 con 0x7f8474162b70 2024-04-15T15:20:29.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.788+0000 7f84717fa640 1 -- 172.21.15.105:0/3608855741 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f846c014480 con 0x7f8474162b70 2024-04-15T15:20:29.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.788+0000 7f847ad32640 1 -- 172.21.15.105:0/3608855741 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84740ce3c0 con 0x7f8474162b70 2024-04-15T15:20:29.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.788+0000 7f84717fa640 1 -- 172.21.15.105:0/3608855741 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f846c014870 con 0x7f8474162b70 2024-04-15T15:20:29.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.788+0000 7f847ad32640 1 -- 172.21.15.105:0/3608855741 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84740ce980 con 0x7f8474162b70 2024-04-15T15:20:29.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.789+0000 7f847ad32640 1 -- 172.21.15.105:0/3608855741 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84740cebc0 con 0x7f8474162b70 2024-04-15T15:20:29.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.790+0000 7f84717fa640 1 -- 172.21.15.105:0/3608855741 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f846c005d10 con 0x7f8474162b70 2024-04-15T15:20:29.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.790+0000 7f84717fa640 1 --2- 172.21.15.105:0/3608855741 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f845806c920 0x7f845806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:29.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.790+0000 7f84717fa640 1 -- 172.21.15.105:0/3608855741 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f846c08c450 con 0x7f8474162b70 2024-04-15T15:20:29.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.790+0000 7f8478ace640 1 --2- 172.21.15.105:0/3608855741 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f845806c920 0x7f845806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:29.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.791+0000 7f8478ace640 1 --2- 172.21.15.105:0/3608855741 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f845806c920 0x7f845806ede0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8468009ac0 tx=0x7f8468009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:29.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.795+0000 7f84717fa640 1 -- 172.21.15.105:0/3608855741 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f84740cebc0 con 0x7f8474162b70 2024-04-15T15:20:29.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.970+0000 7f847ad32640 1 -- 172.21.15.105:0/3608855741 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8474068a40 con 0x7f8474162b70 2024-04-15T15:20:29.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.971+0000 7f84717fa640 1 -- 172.21.15.105:0/3608855741 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16730 (secure 0 0 0) 0x7f846c022070 con 0x7f8474162b70 2024-04-15T15:20:29.973 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:20:29.973 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":33,"fsid":"09e4d2a4-a324-4793-af77-644b25e520e1","created":"2024-04-15T15:19:01.524543+0000","modified":"2024-04-15T15:20:29.000652+0000","last_up_change":"2024-04-15T15:19:12.825815+0000","last_in_change":"2024-04-15T15:19:03.788126+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:19:13.748585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:20:26.983353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-15T15:20:27.961802+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6f6292b-2d7e-4aa0-9d1c-6ddd0f97287c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6801","nonce":2873040054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6803","nonce":2873040054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6807","nonce":2873040054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6805","nonce":2873040054}]},"public_addr":"172.21.15.105:6801/2873040054","cluster_addr":"172.21.15.105:6803/2873040054","heartbeat_back_addr":"172.21.15.105:6807/2873040054","heartbeat_front_addr":"172.21.15.105:6805/2873040054","state":["exists","up"]},{"osd":1,"uuid":"8900966a-b393-41e3-8f93-e5a43bd23622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6818","nonce":2494994479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6821","nonce":2494994479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6828","nonce":2494994479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6823","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6825","nonce":2494994479}]},"public_addr":"172.21.15.105:6818/2494994479","cluster_addr":"172.21.15.105:6821/2494994479","heartbeat_back_addr":"172.21.15.105:6828/2494994479","heartbeat_front_addr":"172.21.15.105:6825/2494994479","state":["exists","up"]},{"osd":2,"uuid":"81a5dc7a-9325-4ff5-941d-a2d5fba21f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6809","nonce":3798408699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6811","nonce":3798408699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6815","nonce":3798408699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6813","nonce":3798408699}]},"public_addr":"172.21.15.105:6809/3798408699","cluster_addr":"172.21.15.105:6811/3798408699","heartbeat_back_addr":"172.21.15.105:6815/3798408699","heartbeat_front_addr":"172.21.15.105:6813/3798408699","state":["exists","up"]},{"osd":3,"uuid":"df766ed2-5c71-4805-96f4-3d4970edc960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6819","nonce":1920720186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6824","nonce":1920720186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6831","nonce":1920720186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6827","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6829","nonce":1920720186}]},"public_addr":"172.21.15.105:6819/1920720186","cluster_addr":"172.21.15.105:6824/1920720186","heartbeat_back_addr":"172.21.15.105:6831/1920720186","heartbeat_front_addr":"172.21.15.105:6829/1920720186","state":["exists","up"]},{"osd":4,"uuid":"e1039e03-28b8-49df-b951-c91930b93c9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6810","nonce":3516494456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6813","nonce":3516494456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6820","nonce":3516494456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6817","nonce":3516494456}]},"public_addr":"172.21.15.154:6810/3516494456","cluster_addr":"172.21.15.154:6813/3516494456","heartbeat_back_addr":"172.21.15.154:6820/3516494456","heartbeat_front_addr":"172.21.15.154:6817/3516494456","state":["exists","up"]},{"osd":5,"uuid":"a9f5eaf1-66ff-405a-a9e4-ec3340812a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6801","nonce":540367751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6803","nonce":540367751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6807","nonce":540367751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6805","nonce":540367751}]},"public_addr":"172.21.15.154:6801/540367751","cluster_addr":"172.21.15.154:6803/540367751","heartbeat_back_addr":"172.21.15.154:6807/540367751","heartbeat_front_addr":"172.21.15.154:6805/540367751","state":["exists","up"]},{"osd":6,"uuid":"267d02ea-675f-428f-aae8-e0375e4d4feb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6825","nonce":1550829005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6827","nonce":1550829005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6831","nonce":1550829005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6829","nonce":1550829005}]},"public_addr":"172.21.15.154:6825/1550829005","cluster_addr":"172.21.15.154:6827/1550829005","heartbeat_back_addr":"172.21.15.154:6831/1550829005","heartbeat_front_addr":"172.21.15.154:6829/1550829005","state":["exists","up"]},{"osd":7,"uuid":"759ac6cc-896a-423c-885f-9fe90d8dcb14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6809","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6811","nonce":3920089903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6816","nonce":3920089903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6823","nonce":3920089903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6819","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6821","nonce":3920089903}]},"public_addr":"172.21.15.154:6811/3920089903","cluster_addr":"172.21.15.154:6816/3920089903","heartbeat_back_addr":"172.21.15.154:6823/3920089903","heartbeat_front_addr":"172.21.15.154:6821/3920089903","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.949509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.984072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.907278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.947391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.975447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.998896+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:09.028739+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.894119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:20:29.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.973+0000 7f847ad32640 1 -- 172.21.15.105:0/3608855741 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f845806c920 msgr2=0x7f845806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:29.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.973+0000 7f847ad32640 1 --2- 172.21.15.105:0/3608855741 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f845806c920 0x7f845806ede0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8468009ac0 tx=0x7f8468009190 comp rx=0 tx=0).stop 2024-04-15T15:20:29.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.973+0000 7f847ad32640 1 -- 172.21.15.105:0/3608855741 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8474162b70 msgr2=0x7f847414ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:29.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.973+0000 7f847ad32640 1 --2- 172.21.15.105:0/3608855741 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8474162b70 0x7f847414ca40 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f846c00efc0 tx=0x7f846c00c490 comp rx=0 tx=0).stop 2024-04-15T15:20:29.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.973+0000 7f847ad32640 1 -- 172.21.15.105:0/3608855741 shutdown_connections 2024-04-15T15:20:29.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.973+0000 7f847ad32640 1 --2- 172.21.15.105:0/3608855741 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f847415eb60 0x7f84740cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:29.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.973+0000 7f847ad32640 1 --2- 172.21.15.105:0/3608855741 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f845806c920 0x7f845806ede0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:29.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.973+0000 7f847ad32640 1 --2- 172.21.15.105:0/3608855741 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8474162b70 0x7f847414ca40 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:29.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.973+0000 7f847ad32640 1 --2- 172.21.15.105:0/3608855741 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84740d1710 0x7f84740cd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:29.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.973+0000 7f847ad32640 1 -- 172.21.15.105:0/3608855741 >> 172.21.15.105:0/3608855741 conn(0x7f8474067720 msgr2=0x7f84740b8990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:29.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.973+0000 7f847ad32640 1 -- 172.21.15.105:0/3608855741 shutdown_connections 2024-04-15T15:20:29.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:29.973+0000 7f847ad32640 1 -- 172.21.15.105:0/3608855741 wait complete. 2024-04-15T15:20:29.986 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:20:30.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.087+0000 7f9a1b9c8640 1 Processor -- start 2024-04-15T15:20:30.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.087+0000 7f9a1b9c8640 1 -- start start 2024-04-15T15:20:30.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.088+0000 7f9a1b9c8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9a1413e680 0x7f9a140cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:30.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.088+0000 7f9a1b9c8640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a1415bc90 con 0x7f9a1406f6a0 2024-04-15T15:20:30.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.088+0000 7f9a1b9c8640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a1415bf70 con 0x7f9a1413ec60 2024-04-15T15:20:30.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.088+0000 7f9a1b9c8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f9a1415c250 con 0x7f9a1413e680 2024-04-15T15:20:30.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.088+0000 7f9a19f65640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f9a1406f6a0 0x7f9a1415f1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:44366/0 (socket says 172.21.15.105:44366) 2024-04-15T15:20:30.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.088+0000 7f9a19f65640 1 -- 172.21.15.105:0/2130568964 learned_addr learned my addr 172.21.15.105:0/2130568964 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:20:30.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.088+0000 7f9a19764640 1 --2- 172.21.15.105:0/2130568964 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9a1413e680 0x7f9a140cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:30.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.088+0000 7f9a0bfff640 1 -- 172.21.15.105:0/2130568964 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2807708404 0 0) 0x7f9a1415bc90 con 0x7f9a1406f6a0 2024-04-15T15:20:30.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.088+0000 7f9a0bfff640 1 -- 172.21.15.105:0/2130568964 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9a00003520 con 0x7f9a1406f6a0 2024-04-15T15:20:30.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.088+0000 7f9a0bfff640 1 -- 172.21.15.105:0/2130568964 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1503863357 0 0) 0x7f9a1415bf70 con 0x7f9a1413ec60 2024-04-15T15:20:30.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.088+0000 7f9a0bfff640 1 -- 172.21.15.105:0/2130568964 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9a1415bc90 con 0x7f9a1413ec60 2024-04-15T15:20:30.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.088+0000 7f9a0bfff640 1 -- 172.21.15.105:0/2130568964 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2246185819 0 0) 0x7f9a00003520 con 0x7f9a1406f6a0 2024-04-15T15:20:30.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.088+0000 7f9a19764640 1 -- 172.21.15.105:0/2130568964 >> v1:172.21.15.154:6789/0 conn(0x7f9a1413ec60 legacy=0x7f9a1406ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:30.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.088+0000 7f9a19764640 1 -- 172.21.15.105:0/2130568964 >> v1:172.21.15.105:6789/0 conn(0x7f9a1406f6a0 legacy=0x7f9a1415f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:30.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.088+0000 7f9a19764640 1 -- 172.21.15.105:0/2130568964 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a1415c530 con 0x7f9a1413e680 2024-04-15T15:20:30.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.089+0000 7f9a19764640 1 --2- 172.21.15.105:0/2130568964 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9a1413e680 0x7f9a140cfb70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9a100097d0 tx=0x7f9a1002b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14aebc9ae5062cac server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:30.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.089+0000 7f9a0bfff640 1 -- 172.21.15.105:0/2130568964 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a10037070 con 0x7f9a1413e680 2024-04-15T15:20:30.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.089+0000 7f9a0bfff640 1 -- 172.21.15.105:0/2130568964 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a10030510 con 0x7f9a1413e680 2024-04-15T15:20:30.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.089+0000 7f9a0bfff640 1 -- 172.21.15.105:0/2130568964 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c0019f0 con 0x7f9a1413e680 2024-04-15T15:20:30.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.090+0000 7f9a1b9c8640 1 -- 172.21.15.105:0/2130568964 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9a1413e680 msgr2=0x7f9a140cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:30.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.090+0000 7f9a1b9c8640 1 --2- 172.21.15.105:0/2130568964 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9a1413e680 0x7f9a140cfb70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9a100097d0 tx=0x7f9a1002b800 comp rx=0 tx=0).stop 2024-04-15T15:20:30.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.090+0000 7f9a1b9c8640 1 -- 172.21.15.105:0/2130568964 shutdown_connections 2024-04-15T15:20:30.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.090+0000 7f9a1b9c8640 1 --2- 172.21.15.105:0/2130568964 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9a1413e680 0x7f9a140cfb70 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:30.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.090+0000 7f9a1b9c8640 1 -- 172.21.15.105:0/2130568964 >> 172.21.15.105:0/2130568964 conn(0x7f9a14067d50 msgr2=0x7f9a1406b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:30.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.090+0000 7f9a1b9c8640 1 -- 172.21.15.105:0/2130568964 shutdown_connections 2024-04-15T15:20:30.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.090+0000 7f9a1b9c8640 1 -- 172.21.15.105:0/2130568964 wait complete. 2024-04-15T15:20:30.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.091+0000 7f9a1b9c8640 1 Processor -- start 2024-04-15T15:20:30.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.091+0000 7f9a1b9c8640 1 -- start start 2024-04-15T15:20:30.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.092+0000 7f9a1b9c8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a1406f6a0 0x7f9a1413d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:30.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.092+0000 7f9a1b9c8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9a1413e680 0x7f9a14156b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:30.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.092+0000 7f9a1b9c8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9a1413ec60 0x7f9a1406de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:30.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.092+0000 7f9a1b9c8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9a140d0230 con 0x7f9a1406f6a0 2024-04-15T15:20:30.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.092+0000 7f9a1b9c8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9a140d00b0 con 0x7f9a1413e680 2024-04-15T15:20:30.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.092+0000 7f9a1b9c8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f9a140cf590 con 0x7f9a1413ec60 2024-04-15T15:20:30.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.092+0000 7f9a19764640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a1406f6a0 0x7f9a1413d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:30.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.092+0000 7f9a19764640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a1406f6a0 0x7f9a1413d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54924/0 (socket says 172.21.15.105:54924) 2024-04-15T15:20:30.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.092+0000 7f9a19764640 1 -- 172.21.15.105:0/408665291 learned_addr learned my addr 172.21.15.105:0/408665291 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:30.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.092+0000 7f9a19f65640 1 --2- 172.21.15.105:0/408665291 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9a1413ec60 0x7f9a1406de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:30.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.092+0000 7f9a18f63640 1 --2- 172.21.15.105:0/408665291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9a1413e680 0x7f9a14156b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:30.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.092+0000 7f9a19764640 1 -- 172.21.15.105:0/408665291 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9a1413ec60 msgr2=0x7f9a1406de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:30.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.092+0000 7f9a19764640 1 --2- 172.21.15.105:0/408665291 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9a1413ec60 0x7f9a1406de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:30.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.092+0000 7f9a19764640 1 -- 172.21.15.105:0/408665291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9a1413e680 msgr2=0x7f9a14156b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:30.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.092+0000 7f9a19764640 1 --2- 172.21.15.105:0/408665291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9a1413e680 0x7f9a14156b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:30.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.092+0000 7f9a19764640 1 -- 172.21.15.105:0/408665291 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a14143690 con 0x7f9a1406f6a0 2024-04-15T15:20:30.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.092+0000 7f9a19764640 1 --2- 172.21.15.105:0/408665291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a1406f6a0 0x7f9a1413d930 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f9a1002b840 tx=0x7f9a100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:30.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.093+0000 7f9a09ffb640 1 -- 172.21.15.105:0/408665291 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a10040020 con 0x7f9a1406f6a0 2024-04-15T15:20:30.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.093+0000 7f9a09ffb640 1 -- 172.21.15.105:0/408665291 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a10038040 con 0x7f9a1406f6a0 2024-04-15T15:20:30.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.093+0000 7f9a1b9c8640 1 -- 172.21.15.105:0/408665291 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a14143970 con 0x7f9a1406f6a0 2024-04-15T15:20:30.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.093+0000 7f9a09ffb640 1 -- 172.21.15.105:0/408665291 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a10037070 con 0x7f9a1406f6a0 2024-04-15T15:20:30.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.093+0000 7f9a1b9c8640 1 -- 172.21.15.105:0/408665291 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a141587c0 con 0x7f9a1406f6a0 2024-04-15T15:20:30.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.094+0000 7f9a1b9c8640 1 -- 172.21.15.105:0/408665291 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a14158d20 con 0x7f9a1406f6a0 2024-04-15T15:20:30.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.094+0000 7f9a09ffb640 1 -- 172.21.15.105:0/408665291 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9a10043020 con 0x7f9a1406f6a0 2024-04-15T15:20:30.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.095+0000 7f9a09ffb640 1 --2- 172.21.15.105:0/408665291 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f99f806c8f0 0x7f99f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:30.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.095+0000 7f9a09ffb640 1 -- 172.21.15.105:0/408665291 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f9a100acfa0 con 0x7f9a1406f6a0 2024-04-15T15:20:30.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.095+0000 7f9a18f63640 1 --2- 172.21.15.105:0/408665291 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f99f806c8f0 0x7f99f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:30.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.095+0000 7f9a18f63640 1 --2- 172.21.15.105:0/408665291 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f99f806c8f0 0x7f99f806edb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9a040015b0 tx=0x7f9a04009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:30.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.099+0000 7f9a09ffb640 1 -- 172.21.15.105:0/408665291 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9a1004a030 con 0x7f9a1406f6a0 2024-04-15T15:20:30.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.273+0000 7f9a1b9c8640 1 -- 172.21.15.105:0/408665291 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9a140b3790 con 0x7f9a1406f6a0 2024-04-15T15:20:30.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.274+0000 7f9a09ffb640 1 -- 172.21.15.105:0/408665291 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16730 (secure 0 0 0) 0x7f9a1007b460 con 0x7f9a1406f6a0 2024-04-15T15:20:30.276 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:20:30.276 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":33,"fsid":"09e4d2a4-a324-4793-af77-644b25e520e1","created":"2024-04-15T15:19:01.524543+0000","modified":"2024-04-15T15:20:29.000652+0000","last_up_change":"2024-04-15T15:19:12.825815+0000","last_in_change":"2024-04-15T15:19:03.788126+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:19:13.748585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:20:26.983353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-15T15:20:27.961802+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6f6292b-2d7e-4aa0-9d1c-6ddd0f97287c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6801","nonce":2873040054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6803","nonce":2873040054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6807","nonce":2873040054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6805","nonce":2873040054}]},"public_addr":"172.21.15.105:6801/2873040054","cluster_addr":"172.21.15.105:6803/2873040054","heartbeat_back_addr":"172.21.15.105:6807/2873040054","heartbeat_front_addr":"172.21.15.105:6805/2873040054","state":["exists","up"]},{"osd":1,"uuid":"8900966a-b393-41e3-8f93-e5a43bd23622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6818","nonce":2494994479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6821","nonce":2494994479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6828","nonce":2494994479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6823","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6825","nonce":2494994479}]},"public_addr":"172.21.15.105:6818/2494994479","cluster_addr":"172.21.15.105:6821/2494994479","heartbeat_back_addr":"172.21.15.105:6828/2494994479","heartbeat_front_addr":"172.21.15.105:6825/2494994479","state":["exists","up"]},{"osd":2,"uuid":"81a5dc7a-9325-4ff5-941d-a2d5fba21f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6809","nonce":3798408699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6811","nonce":3798408699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6815","nonce":3798408699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6813","nonce":3798408699}]},"public_addr":"172.21.15.105:6809/3798408699","cluster_addr":"172.21.15.105:6811/3798408699","heartbeat_back_addr":"172.21.15.105:6815/3798408699","heartbeat_front_addr":"172.21.15.105:6813/3798408699","state":["exists","up"]},{"osd":3,"uuid":"df766ed2-5c71-4805-96f4-3d4970edc960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6819","nonce":1920720186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6824","nonce":1920720186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6831","nonce":1920720186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6827","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6829","nonce":1920720186}]},"public_addr":"172.21.15.105:6819/1920720186","cluster_addr":"172.21.15.105:6824/1920720186","heartbeat_back_addr":"172.21.15.105:6831/1920720186","heartbeat_front_addr":"172.21.15.105:6829/1920720186","state":["exists","up"]},{"osd":4,"uuid":"e1039e03-28b8-49df-b951-c91930b93c9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6810","nonce":3516494456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6813","nonce":3516494456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6820","nonce":3516494456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6817","nonce":3516494456}]},"public_addr":"172.21.15.154:6810/3516494456","cluster_addr":"172.21.15.154:6813/3516494456","heartbeat_back_addr":"172.21.15.154:6820/3516494456","heartbeat_front_addr":"172.21.15.154:6817/3516494456","state":["exists","up"]},{"osd":5,"uuid":"a9f5eaf1-66ff-405a-a9e4-ec3340812a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6801","nonce":540367751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6803","nonce":540367751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6807","nonce":540367751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6805","nonce":540367751}]},"public_addr":"172.21.15.154:6801/540367751","cluster_addr":"172.21.15.154:6803/540367751","heartbeat_back_addr":"172.21.15.154:6807/540367751","heartbeat_front_addr":"172.21.15.154:6805/540367751","state":["exists","up"]},{"osd":6,"uuid":"267d02ea-675f-428f-aae8-e0375e4d4feb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6825","nonce":1550829005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6827","nonce":1550829005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6831","nonce":1550829005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6829","nonce":1550829005}]},"public_addr":"172.21.15.154:6825/1550829005","cluster_addr":"172.21.15.154:6827/1550829005","heartbeat_back_addr":"172.21.15.154:6831/1550829005","heartbeat_front_addr":"172.21.15.154:6829/1550829005","state":["exists","up"]},{"osd":7,"uuid":"759ac6cc-896a-423c-885f-9fe90d8dcb14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6809","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6811","nonce":3920089903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6816","nonce":3920089903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6823","nonce":3920089903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6819","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6821","nonce":3920089903}]},"public_addr":"172.21.15.154:6811/3920089903","cluster_addr":"172.21.15.154:6816/3920089903","heartbeat_back_addr":"172.21.15.154:6823/3920089903","heartbeat_front_addr":"172.21.15.154:6821/3920089903","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.949509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.984072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.907278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.947391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.975447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.998896+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:09.028739+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.894119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:20:30.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.276+0000 7f9a1b9c8640 1 -- 172.21.15.105:0/408665291 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f99f806c8f0 msgr2=0x7f99f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:30.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.276+0000 7f9a1b9c8640 1 --2- 172.21.15.105:0/408665291 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f99f806c8f0 0x7f99f806edb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9a040015b0 tx=0x7f9a04009040 comp rx=0 tx=0).stop 2024-04-15T15:20:30.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.276+0000 7f9a1b9c8640 1 -- 172.21.15.105:0/408665291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a1406f6a0 msgr2=0x7f9a1413d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:30.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.276+0000 7f9a1b9c8640 1 --2- 172.21.15.105:0/408665291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a1406f6a0 0x7f9a1413d930 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f9a1002b840 tx=0x7f9a100046c0 comp rx=0 tx=0).stop 2024-04-15T15:20:30.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.276+0000 7f9a1b9c8640 1 -- 172.21.15.105:0/408665291 shutdown_connections 2024-04-15T15:20:30.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.276+0000 7f9a1b9c8640 1 --2- 172.21.15.105:0/408665291 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9a1413ec60 0x7f9a1406de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:30.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.276+0000 7f9a1b9c8640 1 --2- 172.21.15.105:0/408665291 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f99f806c8f0 0x7f99f806edb0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:30.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.276+0000 7f9a1b9c8640 1 --2- 172.21.15.105:0/408665291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9a1413e680 0x7f9a14156b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:30.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.276+0000 7f9a1b9c8640 1 --2- 172.21.15.105:0/408665291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a1406f6a0 0x7f9a1413d930 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:30.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.276+0000 7f9a1b9c8640 1 -- 172.21.15.105:0/408665291 >> 172.21.15.105:0/408665291 conn(0x7f9a14067d50 msgr2=0x7f9a140b39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:30.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.276+0000 7f9a1b9c8640 1 -- 172.21.15.105:0/408665291 shutdown_connections 2024-04-15T15:20:30.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.276+0000 7f9a1b9c8640 1 -- 172.21.15.105:0/408665291 wait complete. 2024-04-15T15:20:30.289 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-15T15:20:30.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.393+0000 7feeb928f640 1 Processor -- start 2024-04-15T15:20:30.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.393+0000 7feeb928f640 1 -- start start 2024-04-15T15:20:30.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.393+0000 7feeb928f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feeb41620c0 0x7feeb40b7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:30.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.393+0000 7feeb928f640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7feeb41625c0 0x7feeb4162a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:30.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.393+0000 7feeb928f640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feeb40d2fc0 con 0x7feeb40d2b40 2024-04-15T15:20:30.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.393+0000 7feeb928f640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7feeb4162f70 con 0x7feeb41625c0 2024-04-15T15:20:30.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.393+0000 7feeb928f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7feeb40b8c00 con 0x7feeb41620c0 2024-04-15T15:20:30.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.394+0000 7feeb27fc640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7feeb40d2b40 0x7feeb40ba190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:44382/0 (socket says 172.21.15.105:44382) 2024-04-15T15:20:30.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.394+0000 7feeb27fc640 1 -- 172.21.15.105:0/1407870579 learned_addr learned my addr 172.21.15.105:0/1407870579 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:20:30.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.394+0000 7feeb2ffd640 1 --2- 172.21.15.105:0/1407870579 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feeb41620c0 0x7feeb40b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:30.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.394+0000 7feeb1ffb640 1 -- 172.21.15.105:0/1407870579 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3418287197 0 0) 0x7feea8001490 con 0x7feeb40d2b40 2024-04-15T15:20:30.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.394+0000 7feeb1ffb640 1 -- 172.21.15.105:0/1407870579 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fee9c003520 con 0x7feeb40d2b40 2024-04-15T15:20:30.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.394+0000 7feeb37fe640 1 --2- 172.21.15.105:0/1407870579 >> v2:172.21.15.154:3300/0 conn(0x7feeb41625c0 0x7feeb4162a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:30.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.394+0000 7feeb1ffb640 1 -- 172.21.15.105:0/1407870579 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2889577493 0 0) 0x7fee9c003520 con 0x7feeb40d2b40 2024-04-15T15:20:30.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.394+0000 7feeb1ffb640 1 -- 172.21.15.105:0/1407870579 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feeb41620c0 msgr2=0x7feeb40b7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:30.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.394+0000 7feeb1ffb640 1 --2- 172.21.15.105:0/1407870579 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feeb41620c0 0x7feeb40b7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:30.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.394+0000 7feeb1ffb640 1 -- 172.21.15.105:0/1407870579 >> v2:172.21.15.154:3300/0 conn(0x7feeb41625c0 msgr2=0x7feeb4162a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:30.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.394+0000 7feeb1ffb640 1 --2- 172.21.15.105:0/1407870579 >> v2:172.21.15.154:3300/0 conn(0x7feeb41625c0 0x7feeb4162a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:30.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.394+0000 7feeb1ffb640 1 -- 172.21.15.105:0/1407870579 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feeb40b7ff0 con 0x7feeb40d2b40 2024-04-15T15:20:30.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.394+0000 7feeb1ffb640 1 -- 172.21.15.105:0/1407870579 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7feea8001bc0 con 0x7feeb40d2b40 2024-04-15T15:20:30.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.394+0000 7feeb1ffb640 1 -- 172.21.15.105:0/1407870579 >> v1:172.21.15.105:6789/0 conn(0x7feeb40d2b40 legacy=0x7feeb40ba190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:30.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.394+0000 7feeb1ffb640 1 --2- 172.21.15.105:0/1407870579 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee9c0039b0 0x7fee9c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:30.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.394+0000 7feeb1ffb640 1 --2- 172.21.15.105:0/1407870579 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feeb41620c0 0x7fee9c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:30.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.394+0000 7feeb1ffb640 1 --2- 172.21.15.105:0/1407870579 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee9c004d40 0x7fee9c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:30.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.394+0000 7feeb1ffb640 1 -- 172.21.15.105:0/1407870579 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7feeb4162f70 con 0x7fee9c0039b0 2024-04-15T15:20:30.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.394+0000 7feeb1ffb640 1 -- 172.21.15.105:0/1407870579 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7feeb40b8c00 con 0x7fee9c004d40 2024-04-15T15:20:30.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.394+0000 7feeb1ffb640 1 -- 172.21.15.105:0/1407870579 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fee9c00b9c0 con 0x7feeb41620c0 2024-04-15T15:20:30.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.394+0000 7feeb2ffd640 1 --2- 172.21.15.105:0/1407870579 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee9c0039b0 0x7fee9c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:30.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.394+0000 7feeb27fc640 1 --2- 172.21.15.105:0/1407870579 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feeb41620c0 0x7fee9c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:30.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.394+0000 7feeb37fe640 1 --2- 172.21.15.105:0/1407870579 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee9c004d40 0x7fee9c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:30.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.395+0000 7feeb2ffd640 1 -- 172.21.15.105:0/1407870579 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feeb41620c0 msgr2=0x7fee9c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:30.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.395+0000 7feeb2ffd640 1 --2- 172.21.15.105:0/1407870579 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feeb41620c0 0x7fee9c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:30.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.395+0000 7feeb2ffd640 1 -- 172.21.15.105:0/1407870579 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee9c004d40 msgr2=0x7fee9c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:30.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.395+0000 7feeb2ffd640 1 --2- 172.21.15.105:0/1407870579 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee9c004d40 0x7fee9c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:30.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.395+0000 7feeb2ffd640 1 -- 172.21.15.105:0/1407870579 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee9c003520 con 0x7fee9c0039b0 2024-04-15T15:20:30.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.395+0000 7feeb2ffd640 1 --2- 172.21.15.105:0/1407870579 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee9c0039b0 0x7fee9c003dd0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7feea4009ac0 tx=0x7fee9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47748e2ccc732e41 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:30.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.395+0000 7feeb1ffb640 1 -- 172.21.15.105:0/1407870579 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feea4039070 con 0x7fee9c0039b0 2024-04-15T15:20:30.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.395+0000 7feeb1ffb640 1 -- 172.21.15.105:0/1407870579 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feea402c6b0 con 0x7fee9c0039b0 2024-04-15T15:20:30.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.395+0000 7feeb1ffb640 1 -- 172.21.15.105:0/1407870579 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feea4009690 con 0x7fee9c0039b0 2024-04-15T15:20:30.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.396+0000 7feeb928f640 1 -- 172.21.15.105:0/1407870579 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee9c0039b0 msgr2=0x7fee9c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:30.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.396+0000 7feeb928f640 1 --2- 172.21.15.105:0/1407870579 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee9c0039b0 0x7fee9c003dd0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7feea4009ac0 tx=0x7fee9c005830 comp rx=0 tx=0).stop 2024-04-15T15:20:30.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.396+0000 7feeb37fe640 1 -- 172.21.15.105:0/1407870579 reap_dead start 2024-04-15T15:20:30.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.396+0000 7feeb928f640 1 -- 172.21.15.105:0/1407870579 shutdown_connections 2024-04-15T15:20:30.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.396+0000 7feeb928f640 1 -- 172.21.15.105:0/1407870579 >> 172.21.15.105:0/1407870579 conn(0x7feeb406ccb0 msgr2=0x7feeb406d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:30.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.396+0000 7feeb928f640 1 -- 172.21.15.105:0/1407870579 shutdown_connections 2024-04-15T15:20:30.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.396+0000 7feeb928f640 1 -- 172.21.15.105:0/1407870579 wait complete. 2024-04-15T15:20:30.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.397+0000 7feeb928f640 1 Processor -- start 2024-04-15T15:20:30.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.397+0000 7feeb928f640 1 -- start start 2024-04-15T15:20:30.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.398+0000 7feeb928f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee9c0039b0 0x7feeb4145a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:30.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.398+0000 7feeb928f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feeb41f5b70 0x7feeb4157bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:30.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.398+0000 7feeb928f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feeb415eff0 0x7feeb4155ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:30.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.398+0000 7feeb928f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7feeb40b8890 con 0x7feeb415eff0 2024-04-15T15:20:30.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.398+0000 7feeb928f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7feeb40b85b0 con 0x7fee9c0039b0 2024-04-15T15:20:30.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.398+0000 7feeb928f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7feeb40b7910 con 0x7feeb41f5b70 2024-04-15T15:20:30.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.398+0000 7feeb37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feeb415eff0 0x7feeb4155ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:30.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.398+0000 7feeb37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feeb415eff0 0x7feeb4155ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54950/0 (socket says 172.21.15.105:54950) 2024-04-15T15:20:30.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.398+0000 7feeb37fe640 1 -- 172.21.15.105:0/2501744851 learned_addr learned my addr 172.21.15.105:0/2501744851 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:30.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.398+0000 7feeb2ffd640 1 --2- 172.21.15.105:0/2501744851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee9c0039b0 0x7feeb4145a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:30.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.398+0000 7feeb27fc640 1 --2- 172.21.15.105:0/2501744851 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feeb41f5b70 0x7feeb4157bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:30.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.398+0000 7feeb37fe640 1 -- 172.21.15.105:0/2501744851 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feeb41f5b70 msgr2=0x7feeb4157bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:30.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.398+0000 7feeb37fe640 1 --2- 172.21.15.105:0/2501744851 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feeb41f5b70 0x7feeb4157bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:30.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.399+0000 7feeb37fe640 1 -- 172.21.15.105:0/2501744851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee9c0039b0 msgr2=0x7feeb4145a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:30.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.399+0000 7feeb37fe640 1 --2- 172.21.15.105:0/2501744851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee9c0039b0 0x7feeb4145a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:30.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.399+0000 7feeb37fe640 1 -- 172.21.15.105:0/2501744851 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feeb4145fd0 con 0x7feeb415eff0 2024-04-15T15:20:30.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.399+0000 7feeb37fe640 1 --2- 172.21.15.105:0/2501744851 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feeb415eff0 0x7feeb4155ff0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7feeb40b7390 tx=0x7feeac011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:30.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.399+0000 7fee9b7fe640 1 -- 172.21.15.105:0/2501744851 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeac019020 con 0x7feeb415eff0 2024-04-15T15:20:30.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.399+0000 7fee9b7fe640 1 -- 172.21.15.105:0/2501744851 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeac003610 con 0x7feeb415eff0 2024-04-15T15:20:30.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.399+0000 7feeb928f640 1 -- 172.21.15.105:0/2501744851 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feeb4154f50 con 0x7feeb415eff0 2024-04-15T15:20:30.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.399+0000 7fee9b7fe640 1 -- 172.21.15.105:0/2501744851 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeac003a40 con 0x7feeb415eff0 2024-04-15T15:20:30.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.399+0000 7feeb928f640 1 -- 172.21.15.105:0/2501744851 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feeb4155510 con 0x7feeb415eff0 2024-04-15T15:20:30.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.400+0000 7feeb928f640 1 -- 172.21.15.105:0/2501744851 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feeb4068a40 con 0x7feeb415eff0 2024-04-15T15:20:30.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.400+0000 7fee9b7fe640 1 -- 172.21.15.105:0/2501744851 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feeac01c020 con 0x7feeb415eff0 2024-04-15T15:20:30.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.401+0000 7fee9b7fe640 1 --2- 172.21.15.105:0/2501744851 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7feea006c920 0x7feea006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:30.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.401+0000 7fee9b7fe640 1 -- 172.21.15.105:0/2501744851 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7feeac08baa0 con 0x7feeb415eff0 2024-04-15T15:20:30.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.401+0000 7feeb2ffd640 1 --2- 172.21.15.105:0/2501744851 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7feea006c920 0x7feea006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:30.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.401+0000 7feeb2ffd640 1 --2- 172.21.15.105:0/2501744851 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7feea006c920 0x7feea006ede0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7feea4036000 tx=0x7feea4030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:30.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.405+0000 7fee9b7fe640 1 -- 172.21.15.105:0/2501744851 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7feeac023030 con 0x7feeb415eff0 2024-04-15T15:20:30.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.623+0000 7feeb928f640 1 -- 172.21.15.105:0/2501744851 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7feeb40d2410 con 0x7feeb415eff0 2024-04-15T15:20:30.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.663+0000 7fee9b7fe640 1 -- 172.21.15.105:0/2501744851 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v13) ==== 118+0+0 (secure 0 0 0) 0x7feeb4145fd0 con 0x7feeb415eff0 2024-04-15T15:20:30.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.665+0000 7feeb928f640 1 -- 172.21.15.105:0/2501744851 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7feea006c920 msgr2=0x7feea006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:30.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.665+0000 7feeb928f640 1 --2- 172.21.15.105:0/2501744851 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7feea006c920 0x7feea006ede0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7feea4036000 tx=0x7feea4030000 comp rx=0 tx=0).stop 2024-04-15T15:20:30.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.665+0000 7feeb928f640 1 -- 172.21.15.105:0/2501744851 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feeb415eff0 msgr2=0x7feeb4155ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:30.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.665+0000 7feeb928f640 1 --2- 172.21.15.105:0/2501744851 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feeb415eff0 0x7feeb4155ff0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7feeb40b7390 tx=0x7feeac011920 comp rx=0 tx=0).stop 2024-04-15T15:20:30.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.665+0000 7feeb928f640 1 -- 172.21.15.105:0/2501744851 shutdown_connections 2024-04-15T15:20:30.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.665+0000 7feeb928f640 1 --2- 172.21.15.105:0/2501744851 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feeb41f5b70 0x7feeb4157bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:30.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.665+0000 7feeb928f640 1 --2- 172.21.15.105:0/2501744851 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7feea006c920 0x7feea006ede0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:30.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.665+0000 7feeb928f640 1 --2- 172.21.15.105:0/2501744851 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feeb415eff0 0x7feeb4155ff0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:30.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.665+0000 7feeb928f640 1 --2- 172.21.15.105:0/2501744851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee9c0039b0 0x7feeb4145a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:30.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.665+0000 7feeb928f640 1 -- 172.21.15.105:0/2501744851 >> 172.21.15.105:0/2501744851 conn(0x7feeb406ccb0 msgr2=0x7feeb40d0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:30.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.665+0000 7feeb928f640 1 -- 172.21.15.105:0/2501744851 shutdown_connections 2024-04-15T15:20:30.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.665+0000 7feeb928f640 1 -- 172.21.15.105:0/2501744851 wait complete. 2024-04-15T15:20:30.678 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-15T15:20:30.679 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-15T15:20:30.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.782+0000 7f0f79540640 1 Processor -- start 2024-04-15T15:20:30.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.782+0000 7f0f79540640 1 -- start start 2024-04-15T15:20:30.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.783+0000 7f0f79540640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0f740d07c0 0x7f0f7406b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:30.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.783+0000 7f0f79540640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f0f740cd5a0 0x7f0f740d0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:30.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.783+0000 7f0f79540640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f740cf150 con 0x7f0f740ce520 2024-04-15T15:20:30.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.783+0000 7f0f79540640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f0f740d10e0 con 0x7f0f740d07c0 2024-04-15T15:20:30.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.783+0000 7f0f79540640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f0f7406c030 con 0x7f0f740cd5a0 2024-04-15T15:20:30.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.783+0000 7f0f72ffd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0f740d07c0 0x7f0f7406b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:30.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.783+0000 7f0f72ffd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0f740d07c0 0x7f0f7406b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45210/0 (socket says 172.21.15.105:45210) 2024-04-15T15:20:30.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.783+0000 7f0f737fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f0f740ce520 0x7f0f740cba80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:43676/0 (socket says 172.21.15.105:43676) 2024-04-15T15:20:30.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.783+0000 7f0f72ffd640 1 -- 172.21.15.105:0/4220212507 learned_addr learned my addr 172.21.15.105:0/4220212507 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:30.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.784+0000 7f0f72ffd640 1 -- 172.21.15.105:0/4220212507 >> v2:172.21.15.154:3300/0 conn(0x7f0f740cd5a0 msgr2=0x7f0f740d0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:30.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.784+0000 7f0f6bfff640 1 --2- 172.21.15.105:0/4220212507 >> v2:172.21.15.154:3300/0 conn(0x7f0f740cd5a0 0x7f0f740d0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:30.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.784+0000 7f0f72ffd640 1 --2- 172.21.15.105:0/4220212507 >> v2:172.21.15.154:3300/0 conn(0x7f0f740cd5a0 0x7f0f740d0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:30.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.784+0000 7f0f727fc640 1 -- 172.21.15.105:0/4220212507 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2569087609 0 0) 0x7f0f740cf150 con 0x7f0f740ce520 2024-04-15T15:20:30.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.784+0000 7f0f72ffd640 1 -- 172.21.15.105:0/4220212507 >> v1:172.21.15.105:6789/0 conn(0x7f0f740ce520 legacy=0x7f0f740cba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:30.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.784+0000 7f0f72ffd640 1 -- 172.21.15.105:0/4220212507 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f7406c310 con 0x7f0f740d07c0 2024-04-15T15:20:30.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.784+0000 7f0f72ffd640 1 --2- 172.21.15.105:0/4220212507 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f740d07c0 0x7f0f7406b9c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f0f740cfe70 tx=0x7f0f6402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0734a24604792c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:30.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.784+0000 7f0f727fc640 1 -- 172.21.15.105:0/4220212507 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f64034070 con 0x7f0f740d07c0 2024-04-15T15:20:30.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.784+0000 7f0f727fc640 1 -- 172.21.15.105:0/4220212507 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f64030860 con 0x7f0f740d07c0 2024-04-15T15:20:30.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.784+0000 7f0f727fc640 1 -- 172.21.15.105:0/4220212507 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f64030c30 con 0x7f0f740d07c0 2024-04-15T15:20:30.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.785+0000 7f0f79540640 1 -- 172.21.15.105:0/4220212507 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f740d07c0 msgr2=0x7f0f7406b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:30.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.785+0000 7f0f79540640 1 --2- 172.21.15.105:0/4220212507 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f740d07c0 0x7f0f7406b9c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f0f740cfe70 tx=0x7f0f6402b9a0 comp rx=0 tx=0).stop 2024-04-15T15:20:30.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.785+0000 7f0f79540640 1 -- 172.21.15.105:0/4220212507 shutdown_connections 2024-04-15T15:20:30.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.785+0000 7f0f79540640 1 --2- 172.21.15.105:0/4220212507 >> v2:172.21.15.154:3300/0 conn(0x7f0f740cd5a0 0x7f0f740d0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:30.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.785+0000 7f0f79540640 1 --2- 172.21.15.105:0/4220212507 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f740d07c0 0x7f0f7406b9c0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:30.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.785+0000 7f0f79540640 1 -- 172.21.15.105:0/4220212507 >> 172.21.15.105:0/4220212507 conn(0x7f0f740b2fc0 msgr2=0x7f0f740b33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:30.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.785+0000 7f0f79540640 1 -- 172.21.15.105:0/4220212507 shutdown_connections 2024-04-15T15:20:30.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.785+0000 7f0f79540640 1 -- 172.21.15.105:0/4220212507 wait complete. 2024-04-15T15:20:30.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.786+0000 7f0f79540640 1 Processor -- start 2024-04-15T15:20:30.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.786+0000 7f0f79540640 1 -- start start 2024-04-15T15:20:30.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.787+0000 7f0f79540640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f740cd5a0 0x7f0f740caab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:30.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.787+0000 7f0f79540640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0f740ce520 0x7f0f740c4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:30.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.787+0000 7f0f79540640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0f740d07c0 0x7f0f740c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:30.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.787+0000 7f0f79540640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0f740ce000 con 0x7f0f740cd5a0 2024-04-15T15:20:30.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.787+0000 7f0f79540640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0f740cdd20 con 0x7f0f740ce520 2024-04-15T15:20:30.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.787+0000 7f0f79540640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f0f740cc190 con 0x7f0f740d07c0 2024-04-15T15:20:30.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.787+0000 7f0f72ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f740cd5a0 0x7f0f740caab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:30.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.787+0000 7f0f72ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f740cd5a0 0x7f0f740caab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45226/0 (socket says 172.21.15.105:45226) 2024-04-15T15:20:30.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.787+0000 7f0f72ffd640 1 -- 172.21.15.105:0/3756301277 learned_addr learned my addr 172.21.15.105:0/3756301277 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:30.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.787+0000 7f0f72ffd640 1 -- 172.21.15.105:0/3756301277 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0f740d07c0 msgr2=0x7f0f740c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:30.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.787+0000 7f0f72ffd640 1 --2- 172.21.15.105:0/3756301277 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0f740d07c0 0x7f0f740c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:30.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.787+0000 7f0f72ffd640 1 -- 172.21.15.105:0/3756301277 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0f740ce520 msgr2=0x7f0f740c4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:30.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.787+0000 7f0f6bfff640 1 --2- 172.21.15.105:0/3756301277 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0f740ce520 0x7f0f740c4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:30.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.787+0000 7f0f72ffd640 1 --2- 172.21.15.105:0/3756301277 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0f740ce520 0x7f0f740c4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:30.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.787+0000 7f0f72ffd640 1 -- 172.21.15.105:0/3756301277 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f740c9a10 con 0x7f0f740cd5a0 2024-04-15T15:20:30.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.788+0000 7f0f72ffd640 1 --2- 172.21.15.105:0/3756301277 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f740cd5a0 0x7f0f740caab0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0f64000c00 tx=0x7f0f640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:30.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.788+0000 7f0f6b7fe640 1 -- 172.21.15.105:0/3756301277 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f64034040 con 0x7f0f740cd5a0 2024-04-15T15:20:30.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.788+0000 7f0f6b7fe640 1 -- 172.21.15.105:0/3756301277 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f64039d90 con 0x7f0f740cd5a0 2024-04-15T15:20:30.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.788+0000 7f0f79540640 1 -- 172.21.15.105:0/3756301277 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f740c9cf0 con 0x7f0f740cd5a0 2024-04-15T15:20:30.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.788+0000 7f0f6b7fe640 1 -- 172.21.15.105:0/3756301277 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f64043630 con 0x7f0f740cd5a0 2024-04-15T15:20:30.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.788+0000 7f0f79540640 1 -- 172.21.15.105:0/3756301277 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f740ca2b0 con 0x7f0f740cd5a0 2024-04-15T15:20:30.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.789+0000 7f0f79540640 1 -- 172.21.15.105:0/3756301277 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f74068a40 con 0x7f0f740cd5a0 2024-04-15T15:20:30.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.789+0000 7f0f6b7fe640 1 -- 172.21.15.105:0/3756301277 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0f64037070 con 0x7f0f740cd5a0 2024-04-15T15:20:30.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.790+0000 7f0f6b7fe640 1 --2- 172.21.15.105:0/3756301277 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f0f5406c8f0 0x7f0f5406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:30.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.790+0000 7f0f6bfff640 1 --2- 172.21.15.105:0/3756301277 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f0f5406c8f0 0x7f0f5406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:30.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.790+0000 7f0f6b7fe640 1 -- 172.21.15.105:0/3756301277 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f0f640ade10 con 0x7f0f740cd5a0 2024-04-15T15:20:30.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.790+0000 7f0f6bfff640 1 --2- 172.21.15.105:0/3756301277 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f0f5406c8f0 0x7f0f5406edb0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c006fd0 tx=0x7f0f5c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:30.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:30.794+0000 7f0f6b7fe640 1 -- 172.21.15.105:0/3756301277 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0f6403e070 con 0x7f0f740cd5a0 2024-04-15T15:20:31.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.007+0000 7f0f79540640 1 -- 172.21.15.105:0/3756301277 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f0f740702e0 con 0x7f0f740cd5a0 2024-04-15T15:20:31.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.668+0000 7f0f6b7fe640 1 -- 172.21.15.105:0/3756301277 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v14) ==== 122+0+0 (secure 0 0 0) 0x7f0f6407c360 con 0x7f0f740cd5a0 2024-04-15T15:20:31.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.671+0000 7f0f79540640 1 -- 172.21.15.105:0/3756301277 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f0f5406c8f0 msgr2=0x7f0f5406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:31.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.671+0000 7f0f79540640 1 --2- 172.21.15.105:0/3756301277 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f0f5406c8f0 0x7f0f5406edb0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c006fd0 tx=0x7f0f5c008040 comp rx=0 tx=0).stop 2024-04-15T15:20:31.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.671+0000 7f0f79540640 1 -- 172.21.15.105:0/3756301277 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f740cd5a0 msgr2=0x7f0f740caab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:31.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.671+0000 7f0f79540640 1 --2- 172.21.15.105:0/3756301277 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f740cd5a0 0x7f0f740caab0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0f64000c00 tx=0x7f0f640046c0 comp rx=0 tx=0).stop 2024-04-15T15:20:31.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.671+0000 7f0f79540640 1 -- 172.21.15.105:0/3756301277 shutdown_connections 2024-04-15T15:20:31.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.671+0000 7f0f79540640 1 --2- 172.21.15.105:0/3756301277 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0f740d07c0 0x7f0f740c54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:31.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.671+0000 7f0f79540640 1 --2- 172.21.15.105:0/3756301277 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f0f5406c8f0 0x7f0f5406edb0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:31.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.671+0000 7f0f79540640 1 --2- 172.21.15.105:0/3756301277 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0f740ce520 0x7f0f740c4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:31.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.671+0000 7f0f79540640 1 --2- 172.21.15.105:0/3756301277 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f740cd5a0 0x7f0f740caab0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:31.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.671+0000 7f0f79540640 1 -- 172.21.15.105:0/3756301277 >> 172.21.15.105:0/3756301277 conn(0x7f0f740b2fc0 msgr2=0x7f0f7406eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:31.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.671+0000 7f0f79540640 1 -- 172.21.15.105:0/3756301277 shutdown_connections 2024-04-15T15:20:31.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.671+0000 7f0f79540640 1 -- 172.21.15.105:0/3756301277 wait complete. 2024-04-15T15:20:31.684 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-15T15:20:31.685 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-15T15:20:31.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.789+0000 7f41f9959640 1 Processor -- start 2024-04-15T15:20:31.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.789+0000 7f41f9959640 1 -- start start 2024-04-15T15:20:31.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.790+0000 7f41f9959640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f41f415ecf0 0x7f41f406e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:31.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.790+0000 7f41f9959640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f41f40cd360 0x7f41f4162960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:31.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.790+0000 7f41f9959640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41f406d2a0 con 0x7f41f40d0f10 2024-04-15T15:20:31.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.790+0000 7f41f9959640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f41f406d580 con 0x7f41f415ecf0 2024-04-15T15:20:31.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.790+0000 7f41f9959640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f41f406d860 con 0x7f41f40cd360 2024-04-15T15:20:31.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.790+0000 7f41f2ffd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f41f415ecf0 0x7f41f406e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:31.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.790+0000 7f41f2ffd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f41f415ecf0 0x7f41f406e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45228/0 (socket says 172.21.15.105:45228) 2024-04-15T15:20:31.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.790+0000 7f41f2ffd640 1 -- 172.21.15.105:0/1603464236 learned_addr learned my addr 172.21.15.105:0/1603464236 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:31.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.790+0000 7f41ea7fc640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f41f40d0f10 0x7f41f40ccc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:43684/0 (socket says 172.21.15.105:43684) 2024-04-15T15:20:31.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.790+0000 7f41f2ffd640 1 -- 172.21.15.105:0/1603464236 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f41f40cd360 msgr2=0x7f41f4162960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:31.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.790+0000 7f41f2ffd640 1 --2- 172.21.15.105:0/1603464236 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f41f40cd360 0x7f41f4162960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:31.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.790+0000 7f41f2ffd640 1 -- 172.21.15.105:0/1603464236 >> v1:172.21.15.105:6789/0 conn(0x7f41f40d0f10 legacy=0x7f41f40ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:31.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.790+0000 7f41f27fc640 1 -- 172.21.15.105:0/1603464236 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 674737324 0 0) 0x7f41f406d2a0 con 0x7f41f40d0f10 2024-04-15T15:20:31.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.790+0000 7f41f2ffd640 1 -- 172.21.15.105:0/1603464236 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41f406db40 con 0x7f41f415ecf0 2024-04-15T15:20:31.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.791+0000 7f41f2ffd640 1 --2- 172.21.15.105:0/1603464236 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41f415ecf0 0x7f41f406e060 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f41e4009870 tx=0x7f41e402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8631267ff0eaf631 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:31.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.791+0000 7f41f27fc640 1 -- 172.21.15.105:0/1603464236 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e4034070 con 0x7f41f415ecf0 2024-04-15T15:20:31.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.791+0000 7f41f27fc640 1 -- 172.21.15.105:0/1603464236 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41e4030840 con 0x7f41f415ecf0 2024-04-15T15:20:31.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.791+0000 7f41f27fc640 1 -- 172.21.15.105:0/1603464236 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e4030c10 con 0x7f41f415ecf0 2024-04-15T15:20:31.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.792+0000 7f41f9959640 1 -- 172.21.15.105:0/1603464236 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41f415ecf0 msgr2=0x7f41f406e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:31.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.792+0000 7f41f9959640 1 --2- 172.21.15.105:0/1603464236 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41f415ecf0 0x7f41f406e060 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f41e4009870 tx=0x7f41e402b980 comp rx=0 tx=0).stop 2024-04-15T15:20:31.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.792+0000 7f41f9959640 1 -- 172.21.15.105:0/1603464236 shutdown_connections 2024-04-15T15:20:31.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.792+0000 7f41f9959640 1 --2- 172.21.15.105:0/1603464236 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f41f40cd360 0x7f41f4162960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:31.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.792+0000 7f41f9959640 1 --2- 172.21.15.105:0/1603464236 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41f415ecf0 0x7f41f406e060 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:31.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.792+0000 7f41f9959640 1 -- 172.21.15.105:0/1603464236 >> 172.21.15.105:0/1603464236 conn(0x7f41f40d2600 msgr2=0x7f41f40d2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:31.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.792+0000 7f41f9959640 1 -- 172.21.15.105:0/1603464236 shutdown_connections 2024-04-15T15:20:31.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.792+0000 7f41f9959640 1 -- 172.21.15.105:0/1603464236 wait complete. 2024-04-15T15:20:31.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.793+0000 7f41f9959640 1 Processor -- start 2024-04-15T15:20:31.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.793+0000 7f41f9959640 1 -- start start 2024-04-15T15:20:31.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.794+0000 7f41f9959640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41f40cd360 0x7f41f409c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:31.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.794+0000 7f41f9959640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41f40d0f10 0x7f41f41522a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:31.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.794+0000 7f41f9959640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f41f415ecf0 0x7f41f41527e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:31.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.794+0000 7f41f9959640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f41f415f0d0 con 0x7f41f40cd360 2024-04-15T15:20:31.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.794+0000 7f41f9959640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f41f40d1370 con 0x7f41f40d0f10 2024-04-15T15:20:31.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.794+0000 7f41f9959640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f41f415f250 con 0x7f41f415ecf0 2024-04-15T15:20:31.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.794+0000 7f41f2ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41f40cd360 0x7f41f409c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:31.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.794+0000 7f41f2ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41f40cd360 0x7f41f409c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45236/0 (socket says 172.21.15.105:45236) 2024-04-15T15:20:31.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.794+0000 7f41f2ffd640 1 -- 172.21.15.105:0/294980217 learned_addr learned my addr 172.21.15.105:0/294980217 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:31.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.794+0000 7f41f37fe640 1 --2- 172.21.15.105:0/294980217 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f41f415ecf0 0x7f41f41527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:31.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.794+0000 7f41ea7fc640 1 --2- 172.21.15.105:0/294980217 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41f40d0f10 0x7f41f41522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:31.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.794+0000 7f41f2ffd640 1 -- 172.21.15.105:0/294980217 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f41f415ecf0 msgr2=0x7f41f41527e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:31.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.794+0000 7f41f2ffd640 1 --2- 172.21.15.105:0/294980217 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f41f415ecf0 0x7f41f41527e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:31.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.794+0000 7f41f2ffd640 1 -- 172.21.15.105:0/294980217 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41f40d0f10 msgr2=0x7f41f41522a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:31.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.794+0000 7f41f2ffd640 1 --2- 172.21.15.105:0/294980217 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41f40d0f10 0x7f41f41522a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:31.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.794+0000 7f41f2ffd640 1 -- 172.21.15.105:0/294980217 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41f40cb1d0 con 0x7f41f40cd360 2024-04-15T15:20:31.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.795+0000 7f41f2ffd640 1 --2- 172.21.15.105:0/294980217 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41f40cd360 0x7f41f409c520 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f41e4009870 tx=0x7f41e4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:31.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.795+0000 7f41f0ff9640 1 -- 172.21.15.105:0/294980217 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e4048070 con 0x7f41f40cd360 2024-04-15T15:20:31.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.795+0000 7f41f0ff9640 1 -- 172.21.15.105:0/294980217 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41e4030640 con 0x7f41f40cd360 2024-04-15T15:20:31.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.795+0000 7f41f9959640 1 -- 172.21.15.105:0/294980217 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41f40cb4b0 con 0x7f41f40cd360 2024-04-15T15:20:31.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.795+0000 7f41f0ff9640 1 -- 172.21.15.105:0/294980217 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e4034040 con 0x7f41f40cd360 2024-04-15T15:20:31.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.795+0000 7f41f9959640 1 -- 172.21.15.105:0/294980217 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41f40cba70 con 0x7f41f40cd360 2024-04-15T15:20:31.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.796+0000 7f41f9959640 1 -- 172.21.15.105:0/294980217 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41f4068a40 con 0x7f41f40cd360 2024-04-15T15:20:31.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.796+0000 7f41f0ff9640 1 -- 172.21.15.105:0/294980217 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f41e404b070 con 0x7f41f40cd360 2024-04-15T15:20:31.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.797+0000 7f41f0ff9640 1 --2- 172.21.15.105:0/294980217 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f41d406c920 0x7f41d406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:31.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.797+0000 7f41f0ff9640 1 -- 172.21.15.105:0/294980217 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f41e40adbd0 con 0x7f41f40cd360 2024-04-15T15:20:31.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.797+0000 7f41ea7fc640 1 --2- 172.21.15.105:0/294980217 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f41d406c920 0x7f41d406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:31.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.797+0000 7f41ea7fc640 1 --2- 172.21.15.105:0/294980217 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f41d406c920 0x7f41d406ede0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f41f40702c0 tx=0x7f41dc007b90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:31.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:31.801+0000 7f41f0ff9640 1 -- 172.21.15.105:0/294980217 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f41e407c150 con 0x7f41f40cd360 2024-04-15T15:20:31.979 INFO:tasks.ceph.mds.a.smithi105.stderr:2024-04-15T15:20:31.977+0000 7f9b03443640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-15T15:20:32.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.011+0000 7f41f9959640 1 -- 172.21.15.105:0/294980217 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f41f40b8580 con 0x7f41f40cd360 2024-04-15T15:20:32.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.011+0000 7f41f0ff9640 1 -- 172.21.15.105:0/294980217 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+5180 (secure 0 0 0) 0x7f41e407ba50 con 0x7f41f40cd360 2024-04-15T15:20:32.013 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:20:32.014 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4778,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/4848508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":4848508},{"type":"v1","addr":"172.21.15.154:6835","nonce":4848508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5005,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/1127070021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1127070021},{"type":"v1","addr":"172.21.15.154:6833","nonce":1127070021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5193,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6836/3825669416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3825669416},{"type":"v1","addr":"172.21.15.105:6836","nonce":3825669416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-15T15:20:29.000616+0000","modified":"2024-04-15T15:20:31.008997+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5196},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5196":{"gid":5196,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6837/958066041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":958066041},{"type":"v1","addr":"172.21.15.105:6837","nonce":958066041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5196,"qdb_cluster":[5196]},"id":2}]} 2024-04-15T15:20:32.014 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 14 2024-04-15T15:20:32.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.013+0000 7f41f9959640 1 -- 172.21.15.105:0/294980217 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f41d406c920 msgr2=0x7f41d406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:32.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.013+0000 7f41f9959640 1 --2- 172.21.15.105:0/294980217 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f41d406c920 0x7f41d406ede0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f41f40702c0 tx=0x7f41dc007b90 comp rx=0 tx=0).stop 2024-04-15T15:20:32.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.013+0000 7f41f9959640 1 -- 172.21.15.105:0/294980217 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41f40cd360 msgr2=0x7f41f409c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:32.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.013+0000 7f41f9959640 1 --2- 172.21.15.105:0/294980217 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41f40cd360 0x7f41f409c520 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f41e4009870 tx=0x7f41e4004730 comp rx=0 tx=0).stop 2024-04-15T15:20:32.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.013+0000 7f41f9959640 1 -- 172.21.15.105:0/294980217 shutdown_connections 2024-04-15T15:20:32.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.013+0000 7f41f9959640 1 --2- 172.21.15.105:0/294980217 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f41f415ecf0 0x7f41f41527e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:32.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.013+0000 7f41f9959640 1 --2- 172.21.15.105:0/294980217 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f41d406c920 0x7f41d406ede0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:32.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.013+0000 7f41f9959640 1 --2- 172.21.15.105:0/294980217 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41f40d0f10 0x7f41f41522a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:32.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.013+0000 7f41f9959640 1 --2- 172.21.15.105:0/294980217 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41f40cd360 0x7f41f409c520 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:32.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.013+0000 7f41f9959640 1 -- 172.21.15.105:0/294980217 >> 172.21.15.105:0/294980217 conn(0x7f41f40d2600 msgr2=0x7f41f40b7470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:32.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.014+0000 7f41f9959640 1 -- 172.21.15.105:0/294980217 shutdown_connections 2024-04-15T15:20:32.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.014+0000 7f41f9959640 1 -- 172.21.15.105:0/294980217 wait complete. 2024-04-15T15:20:32.026 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:20:32.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.128+0000 7fded9159640 1 Processor -- start 2024-04-15T15:20:32.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.128+0000 7fded9159640 1 -- start start 2024-04-15T15:20:32.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.129+0000 7fded9159640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fded40d1970 0x7fded406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:32.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.129+0000 7fded9159640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fded406da20 0x7fded40cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:32.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.129+0000 7fded9159640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fded415ee90 con 0x7fded415eab0 2024-04-15T15:20:32.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.129+0000 7fded9159640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fded40ccc40 con 0x7fded40d1970 2024-04-15T15:20:32.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.129+0000 7fded9159640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fded40ccf20 con 0x7fded406da20 2024-04-15T15:20:32.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.129+0000 7fded259c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fded40d1970 0x7fded406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:32.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.129+0000 7fded259c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fded40d1970 0x7fded406d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45248/0 (socket says 172.21.15.105:45248) 2024-04-15T15:20:32.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.129+0000 7fded259c640 1 -- 172.21.15.105:0/799535373 learned_addr learned my addr 172.21.15.105:0/799535373 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:32.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.130+0000 7fded359e640 1 --2- 172.21.15.105:0/799535373 >> v2:172.21.15.154:3300/0 conn(0x7fded406da20 0x7fded40cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:32.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.130+0000 7fded259c640 1 -- 172.21.15.105:0/799535373 >> v2:172.21.15.154:3300/0 conn(0x7fded406da20 msgr2=0x7fded40cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:32.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.130+0000 7fded259c640 1 --2- 172.21.15.105:0/799535373 >> v2:172.21.15.154:3300/0 conn(0x7fded406da20 0x7fded40cdce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:32.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.130+0000 7fded259c640 1 -- 172.21.15.105:0/799535373 >> v1:172.21.15.154:6789/0 conn(0x7fded415eab0 legacy=0x7fded40d1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:32.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.130+0000 7fded259c640 1 -- 172.21.15.105:0/799535373 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fded40cd200 con 0x7fded40d1970 2024-04-15T15:20:32.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.130+0000 7fded259c640 1 --2- 172.21.15.105:0/799535373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fded40d1970 0x7fded406d4e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fded40d24f0 tx=0x7fdebc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c82feaef6315d590 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:32.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.130+0000 7fded1d9b640 1 -- 172.21.15.105:0/799535373 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdebc035020 con 0x7fded40d1970 2024-04-15T15:20:32.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.131+0000 7fded1d9b640 1 -- 172.21.15.105:0/799535373 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdebc031870 con 0x7fded40d1970 2024-04-15T15:20:32.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.131+0000 7fded1d9b640 1 -- 172.21.15.105:0/799535373 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdebc031c40 con 0x7fded40d1970 2024-04-15T15:20:32.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.131+0000 7fded9159640 1 -- 172.21.15.105:0/799535373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fded40d1970 msgr2=0x7fded406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:32.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.131+0000 7fded9159640 1 --2- 172.21.15.105:0/799535373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fded40d1970 0x7fded406d4e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fded40d24f0 tx=0x7fdebc02b910 comp rx=0 tx=0).stop 2024-04-15T15:20:32.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.131+0000 7fded9159640 1 -- 172.21.15.105:0/799535373 shutdown_connections 2024-04-15T15:20:32.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.131+0000 7fded9159640 1 --2- 172.21.15.105:0/799535373 >> v2:172.21.15.154:3300/0 conn(0x7fded406da20 0x7fded40cdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:32.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.131+0000 7fded9159640 1 --2- 172.21.15.105:0/799535373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fded40d1970 0x7fded406d4e0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:32.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.131+0000 7fded9159640 1 -- 172.21.15.105:0/799535373 >> 172.21.15.105:0/799535373 conn(0x7fded40b4040 msgr2=0x7fded40b4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:32.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.131+0000 7fded9159640 1 -- 172.21.15.105:0/799535373 shutdown_connections 2024-04-15T15:20:32.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.131+0000 7fded9159640 1 -- 172.21.15.105:0/799535373 wait complete. 2024-04-15T15:20:32.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.132+0000 7fded9159640 1 Processor -- start 2024-04-15T15:20:32.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.133+0000 7fded9159640 1 -- start start 2024-04-15T15:20:32.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.133+0000 7fded9159640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fded406da20 0x7fded40c77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:32.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.133+0000 7fded9159640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fded40d1970 0x7fded40c9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:32.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.133+0000 7fded9159640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fded415eab0 0x7fded41587b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:32.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.133+0000 7fded9159640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fded406cf90 con 0x7fded406da20 2024-04-15T15:20:32.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.133+0000 7fded9159640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fded40cd7c0 con 0x7fded415eab0 2024-04-15T15:20:32.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.133+0000 7fded9159640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fded406de40 con 0x7fded40d1970 2024-04-15T15:20:32.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.133+0000 7fded2d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fded406da20 0x7fded40c77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:32.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.133+0000 7fded2d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fded406da20 0x7fded40c77b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45254/0 (socket says 172.21.15.105:45254) 2024-04-15T15:20:32.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.133+0000 7fded2d9d640 1 -- 172.21.15.105:0/1740175862 learned_addr learned my addr 172.21.15.105:0/1740175862 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:32.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.133+0000 7fded259c640 1 --2- 172.21.15.105:0/1740175862 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fded40d1970 0x7fded40c9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:32.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.134+0000 7fded359e640 1 --2- 172.21.15.105:0/1740175862 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fded415eab0 0x7fded41587b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:32.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.134+0000 7fded2d9d640 1 -- 172.21.15.105:0/1740175862 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fded40d1970 msgr2=0x7fded40c9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:32.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.134+0000 7fded2d9d640 1 --2- 172.21.15.105:0/1740175862 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fded40d1970 0x7fded40c9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:32.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.134+0000 7fded2d9d640 1 -- 172.21.15.105:0/1740175862 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fded415eab0 msgr2=0x7fded41587b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:32.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.134+0000 7fded2d9d640 1 --2- 172.21.15.105:0/1740175862 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fded415eab0 0x7fded41587b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:32.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.134+0000 7fded2d9d640 1 -- 172.21.15.105:0/1740175862 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fded40c9d50 con 0x7fded406da20 2024-04-15T15:20:32.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.134+0000 7fded359e640 1 --2- 172.21.15.105:0/1740175862 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fded415eab0 0x7fded41587b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:20:32.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.134+0000 7fded259c640 1 --2- 172.21.15.105:0/1740175862 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fded40d1970 0x7fded40c9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:20:32.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.134+0000 7fded2d9d640 1 --2- 172.21.15.105:0/1740175862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fded406da20 0x7fded40c77b0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fdec400ba50 tx=0x7fdec400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:32.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.134+0000 7fdecb7fe640 1 -- 172.21.15.105:0/1740175862 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec4019070 con 0x7fded406da20 2024-04-15T15:20:32.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.134+0000 7fdecb7fe640 1 -- 172.21.15.105:0/1740175862 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdec400cc20 con 0x7fded406da20 2024-04-15T15:20:32.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.134+0000 7fded9159640 1 -- 172.21.15.105:0/1740175862 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fded40ca030 con 0x7fded406da20 2024-04-15T15:20:32.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.134+0000 7fdecb7fe640 1 -- 172.21.15.105:0/1740175862 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec4014680 con 0x7fded406da20 2024-04-15T15:20:32.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.134+0000 7fded9159640 1 -- 172.21.15.105:0/1740175862 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fded40ca5f0 con 0x7fded406da20 2024-04-15T15:20:32.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.135+0000 7fded9159640 1 -- 172.21.15.105:0/1740175862 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fded4068a40 con 0x7fded406da20 2024-04-15T15:20:32.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.135+0000 7fdecb7fe640 1 -- 172.21.15.105:0/1740175862 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdec401c070 con 0x7fded406da20 2024-04-15T15:20:32.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.136+0000 7fdecb7fe640 1 --2- 172.21.15.105:0/1740175862 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fdeb406c8f0 0x7fdeb406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:32.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.136+0000 7fded259c640 1 --2- 172.21.15.105:0/1740175862 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fdeb406c8f0 0x7fdeb406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:32.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.136+0000 7fdecb7fe640 1 -- 172.21.15.105:0/1740175862 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fdec408b870 con 0x7fded406da20 2024-04-15T15:20:32.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.137+0000 7fded259c640 1 --2- 172.21.15.105:0/1740175862 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fdeb406c8f0 0x7fdeb406edb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fdebc036000 tx=0x7fdebc03c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:32.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.140+0000 7fdecb7fe640 1 -- 172.21.15.105:0/1740175862 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdec4059650 con 0x7fded406da20 2024-04-15T15:20:32.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.314+0000 7fded9159640 1 -- 172.21.15.105:0/1740175862 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fded40750c0 con 0x7fded406da20 2024-04-15T15:20:32.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.315+0000 7fdecb7fe640 1 -- 172.21.15.105:0/1740175862 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16717 (secure 0 0 0) 0x7fdec405d950 con 0x7fded406da20 2024-04-15T15:20:32.317 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:20:32.317 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":34,"fsid":"09e4d2a4-a324-4793-af77-644b25e520e1","created":"2024-04-15T15:19:01.524543+0000","modified":"2024-04-15T15:20:30.645565+0000","last_up_change":"2024-04-15T15:19:12.825815+0000","last_in_change":"2024-04-15T15:19:03.788126+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:19:13.748585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:20:26.983353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-15T15:20:27.961802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6f6292b-2d7e-4aa0-9d1c-6ddd0f97287c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6801","nonce":2873040054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6803","nonce":2873040054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6807","nonce":2873040054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6805","nonce":2873040054}]},"public_addr":"172.21.15.105:6801/2873040054","cluster_addr":"172.21.15.105:6803/2873040054","heartbeat_back_addr":"172.21.15.105:6807/2873040054","heartbeat_front_addr":"172.21.15.105:6805/2873040054","state":["exists","up"]},{"osd":1,"uuid":"8900966a-b393-41e3-8f93-e5a43bd23622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6818","nonce":2494994479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6821","nonce":2494994479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6828","nonce":2494994479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6823","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6825","nonce":2494994479}]},"public_addr":"172.21.15.105:6818/2494994479","cluster_addr":"172.21.15.105:6821/2494994479","heartbeat_back_addr":"172.21.15.105:6828/2494994479","heartbeat_front_addr":"172.21.15.105:6825/2494994479","state":["exists","up"]},{"osd":2,"uuid":"81a5dc7a-9325-4ff5-941d-a2d5fba21f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6809","nonce":3798408699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6811","nonce":3798408699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6815","nonce":3798408699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6813","nonce":3798408699}]},"public_addr":"172.21.15.105:6809/3798408699","cluster_addr":"172.21.15.105:6811/3798408699","heartbeat_back_addr":"172.21.15.105:6815/3798408699","heartbeat_front_addr":"172.21.15.105:6813/3798408699","state":["exists","up"]},{"osd":3,"uuid":"df766ed2-5c71-4805-96f4-3d4970edc960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6819","nonce":1920720186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6824","nonce":1920720186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6831","nonce":1920720186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6827","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6829","nonce":1920720186}]},"public_addr":"172.21.15.105:6819/1920720186","cluster_addr":"172.21.15.105:6824/1920720186","heartbeat_back_addr":"172.21.15.105:6831/1920720186","heartbeat_front_addr":"172.21.15.105:6829/1920720186","state":["exists","up"]},{"osd":4,"uuid":"e1039e03-28b8-49df-b951-c91930b93c9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6810","nonce":3516494456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6813","nonce":3516494456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6820","nonce":3516494456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6817","nonce":3516494456}]},"public_addr":"172.21.15.154:6810/3516494456","cluster_addr":"172.21.15.154:6813/3516494456","heartbeat_back_addr":"172.21.15.154:6820/3516494456","heartbeat_front_addr":"172.21.15.154:6817/3516494456","state":["exists","up"]},{"osd":5,"uuid":"a9f5eaf1-66ff-405a-a9e4-ec3340812a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6801","nonce":540367751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6803","nonce":540367751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6807","nonce":540367751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6805","nonce":540367751}]},"public_addr":"172.21.15.154:6801/540367751","cluster_addr":"172.21.15.154:6803/540367751","heartbeat_back_addr":"172.21.15.154:6807/540367751","heartbeat_front_addr":"172.21.15.154:6805/540367751","state":["exists","up"]},{"osd":6,"uuid":"267d02ea-675f-428f-aae8-e0375e4d4feb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6825","nonce":1550829005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6827","nonce":1550829005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6831","nonce":1550829005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6829","nonce":1550829005}]},"public_addr":"172.21.15.154:6825/1550829005","cluster_addr":"172.21.15.154:6827/1550829005","heartbeat_back_addr":"172.21.15.154:6831/1550829005","heartbeat_front_addr":"172.21.15.154:6829/1550829005","state":["exists","up"]},{"osd":7,"uuid":"759ac6cc-896a-423c-885f-9fe90d8dcb14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6809","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6811","nonce":3920089903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6816","nonce":3920089903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6823","nonce":3920089903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6819","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6821","nonce":3920089903}]},"public_addr":"172.21.15.154:6811/3920089903","cluster_addr":"172.21.15.154:6816/3920089903","heartbeat_back_addr":"172.21.15.154:6823/3920089903","heartbeat_front_addr":"172.21.15.154:6821/3920089903","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.949509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.984072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.907278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.947391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.975447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.998896+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:09.028739+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.894119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:20:32.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.317+0000 7fded9159640 1 -- 172.21.15.105:0/1740175862 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fdeb406c8f0 msgr2=0x7fdeb406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:32.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.317+0000 7fded9159640 1 --2- 172.21.15.105:0/1740175862 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fdeb406c8f0 0x7fdeb406edb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fdebc036000 tx=0x7fdebc03c040 comp rx=0 tx=0).stop 2024-04-15T15:20:32.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.317+0000 7fded9159640 1 -- 172.21.15.105:0/1740175862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fded406da20 msgr2=0x7fded40c77b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:32.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.317+0000 7fded9159640 1 --2- 172.21.15.105:0/1740175862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fded406da20 0x7fded40c77b0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fdec400ba50 tx=0x7fdec400bf10 comp rx=0 tx=0).stop 2024-04-15T15:20:32.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.317+0000 7fded9159640 1 -- 172.21.15.105:0/1740175862 shutdown_connections 2024-04-15T15:20:32.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.317+0000 7fded9159640 1 --2- 172.21.15.105:0/1740175862 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fded40d1970 0x7fded40c9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:32.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.317+0000 7fded9159640 1 --2- 172.21.15.105:0/1740175862 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fdeb406c8f0 0x7fdeb406edb0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:32.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.317+0000 7fded9159640 1 --2- 172.21.15.105:0/1740175862 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fded415eab0 0x7fded41587b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:32.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.317+0000 7fded9159640 1 --2- 172.21.15.105:0/1740175862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fded406da20 0x7fded40c77b0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:32.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.317+0000 7fded9159640 1 -- 172.21.15.105:0/1740175862 >> 172.21.15.105:0/1740175862 conn(0x7fded40b4040 msgr2=0x7fded40b12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:32.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.318+0000 7fded9159640 1 -- 172.21.15.105:0/1740175862 shutdown_connections 2024-04-15T15:20:32.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.318+0000 7fded9159640 1 -- 172.21.15.105:0/1740175862 wait complete. 2024-04-15T15:20:32.330 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-15T15:20:32.331 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-15T15:20:32.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.432+0000 7efd803c8640 1 Processor -- start 2024-04-15T15:20:32.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.432+0000 7efd803c8640 1 -- start start 2024-04-15T15:20:32.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.433+0000 7efd803c8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efd780b1280 0x7efd78167470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:32.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.433+0000 7efd803c8640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd780af4b0 con 0x7efd7815cb00 2024-04-15T15:20:32.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.433+0000 7efd803c8640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd780af790 con 0x7efd780d0120 2024-04-15T15:20:32.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.433+0000 7efd803c8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7efd780afa70 con 0x7efd780b1280 2024-04-15T15:20:32.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.433+0000 7efd7d963640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7efd7815cb00 0x7efd780cdae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:43692/0 (socket says 172.21.15.105:43692) 2024-04-15T15:20:32.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.433+0000 7efd7d963640 1 -- 172.21.15.105:0/4097501666 learned_addr learned my addr 172.21.15.105:0/4097501666 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:20:32.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.434+0000 7efd7e965640 1 --2- 172.21.15.105:0/4097501666 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efd780b1280 0x7efd78167470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:32.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.434+0000 7efd7d162640 1 -- 172.21.15.105:0/4097501666 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 547104855 0 0) 0x7efd780af4b0 con 0x7efd7815cb00 2024-04-15T15:20:32.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.434+0000 7efd7d162640 1 -- 172.21.15.105:0/4097501666 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efd64003520 con 0x7efd7815cb00 2024-04-15T15:20:32.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.434+0000 7efd7d162640 1 -- 172.21.15.105:0/4097501666 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3754060487 0 0) 0x7efd64003520 con 0x7efd7815cb00 2024-04-15T15:20:32.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.434+0000 7efd7d162640 1 -- 172.21.15.105:0/4097501666 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efd780b1280 msgr2=0x7efd78167470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:32.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.434+0000 7efd7d162640 1 --2- 172.21.15.105:0/4097501666 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efd780b1280 0x7efd78167470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:32.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.434+0000 7efd7d162640 1 -- 172.21.15.105:0/4097501666 >> v1:172.21.15.154:6789/0 conn(0x7efd780d0120 legacy=0x7efd780d0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:32.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.434+0000 7efd7d162640 1 -- 172.21.15.105:0/4097501666 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd780afd50 con 0x7efd7815cb00 2024-04-15T15:20:32.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.434+0000 7efd7d162640 1 -- 172.21.15.105:0/4097501666 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7efd68001670 con 0x7efd7815cb00 2024-04-15T15:20:32.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.434+0000 7efd7d162640 1 -- 172.21.15.105:0/4097501666 >> v1:172.21.15.105:6789/0 conn(0x7efd7815cb00 legacy=0x7efd780cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:32.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.434+0000 7efd7d162640 1 --2- 172.21.15.105:0/4097501666 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efd780b1280 0x7efd640039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:32.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.434+0000 7efd7d162640 1 --2- 172.21.15.105:0/4097501666 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd64003fb0 0x7efd64004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:32.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.434+0000 7efd7d162640 1 --2- 172.21.15.105:0/4097501666 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efd64004d40 0x7efd640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:32.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.434+0000 7efd7d162640 1 -- 172.21.15.105:0/4097501666 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7efd780afa70 con 0x7efd64004d40 2024-04-15T15:20:32.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.434+0000 7efd7d162640 1 -- 172.21.15.105:0/4097501666 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7efd6400bb00 con 0x7efd64003fb0 2024-04-15T15:20:32.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.434+0000 7efd7d162640 1 -- 172.21.15.105:0/4097501666 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7efd6400bd00 con 0x7efd780b1280 2024-04-15T15:20:32.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.434+0000 7efd7e965640 1 --2- 172.21.15.105:0/4097501666 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efd64004d40 0x7efd640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:32.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.434+0000 7efd7d963640 1 --2- 172.21.15.105:0/4097501666 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd64003fb0 0x7efd64004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:32.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.434+0000 7efd7e164640 1 --2- 172.21.15.105:0/4097501666 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efd780b1280 0x7efd640039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:32.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.434+0000 7efd7e965640 1 -- 172.21.15.105:0/4097501666 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efd780b1280 msgr2=0x7efd640039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:32.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.434+0000 7efd7e965640 1 --2- 172.21.15.105:0/4097501666 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efd780b1280 0x7efd640039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:32.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.434+0000 7efd7e965640 1 -- 172.21.15.105:0/4097501666 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd64003fb0 msgr2=0x7efd64004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:32.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.434+0000 7efd7e965640 1 --2- 172.21.15.105:0/4097501666 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd64003fb0 0x7efd64004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:32.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.434+0000 7efd7e965640 1 -- 172.21.15.105:0/4097501666 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd64003520 con 0x7efd64004d40 2024-04-15T15:20:32.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.434+0000 7efd7d963640 1 --2- 172.21.15.105:0/4097501666 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd64003fb0 0x7efd64004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:20:32.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.435+0000 7efd7e965640 1 --2- 172.21.15.105:0/4097501666 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efd64004d40 0x7efd640051c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7efd74009a90 tx=0x7efd64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfa57bed34437efd server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:32.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.435+0000 7efd7d162640 1 -- 172.21.15.105:0/4097501666 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd74039070 con 0x7efd64004d40 2024-04-15T15:20:32.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.435+0000 7efd7d162640 1 -- 172.21.15.105:0/4097501666 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd7402c680 con 0x7efd64004d40 2024-04-15T15:20:32.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.435+0000 7efd7d162640 1 -- 172.21.15.105:0/4097501666 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd74009690 con 0x7efd64004d40 2024-04-15T15:20:32.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.436+0000 7efd803c8640 1 -- 172.21.15.105:0/4097501666 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efd64004d40 msgr2=0x7efd640051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:32.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.436+0000 7efd803c8640 1 --2- 172.21.15.105:0/4097501666 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efd64004d40 0x7efd640051c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7efd74009a90 tx=0x7efd64005830 comp rx=0 tx=0).stop 2024-04-15T15:20:32.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.436+0000 7efd7e965640 1 -- 172.21.15.105:0/4097501666 reap_dead start 2024-04-15T15:20:32.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.436+0000 7efd803c8640 1 -- 172.21.15.105:0/4097501666 shutdown_connections 2024-04-15T15:20:32.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.436+0000 7efd803c8640 1 -- 172.21.15.105:0/4097501666 >> 172.21.15.105:0/4097501666 conn(0x7efd780b4040 msgr2=0x7efd780b4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:32.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.436+0000 7efd803c8640 1 -- 172.21.15.105:0/4097501666 shutdown_connections 2024-04-15T15:20:32.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.436+0000 7efd803c8640 1 -- 172.21.15.105:0/4097501666 wait complete. 2024-04-15T15:20:32.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.437+0000 7efd803c8640 1 Processor -- start 2024-04-15T15:20:32.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.437+0000 7efd803c8640 1 -- start start 2024-04-15T15:20:32.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.438+0000 7efd803c8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efd64004d40 0x7efd780b7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:32.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.438+0000 7efd803c8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd780c9620 0x7efd780c9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:32.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.438+0000 7efd803c8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efd78141890 0x7efd78141d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:32.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.438+0000 7efd803c8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7efd780ce370 con 0x7efd78141890 2024-04-15T15:20:32.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.438+0000 7efd803c8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7efd780ce1f0 con 0x7efd780c9620 2024-04-15T15:20:32.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.438+0000 7efd803c8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7efd78165460 con 0x7efd64004d40 2024-04-15T15:20:32.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.438+0000 7efd7e965640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efd78141890 0x7efd78141d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:32.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.438+0000 7efd7e965640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efd78141890 0x7efd78141d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45274/0 (socket says 172.21.15.105:45274) 2024-04-15T15:20:32.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.438+0000 7efd7e965640 1 -- 172.21.15.105:0/951185518 learned_addr learned my addr 172.21.15.105:0/951185518 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:32.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.438+0000 7efd7e164640 1 --2- 172.21.15.105:0/951185518 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efd64004d40 0x7efd780b7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:32.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.438+0000 7efd7e965640 1 -- 172.21.15.105:0/951185518 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efd64004d40 msgr2=0x7efd780b7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:32.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.438+0000 7efd7d963640 1 --2- 172.21.15.105:0/951185518 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd780c9620 0x7efd780c9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:32.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.438+0000 7efd7e965640 1 --2- 172.21.15.105:0/951185518 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efd64004d40 0x7efd780b7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:32.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.438+0000 7efd7e965640 1 -- 172.21.15.105:0/951185518 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd780c9620 msgr2=0x7efd780c9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:32.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.438+0000 7efd7e965640 1 --2- 172.21.15.105:0/951185518 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd780c9620 0x7efd780c9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:32.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.438+0000 7efd7e965640 1 -- 172.21.15.105:0/951185518 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd780b65a0 con 0x7efd78141890 2024-04-15T15:20:32.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.439+0000 7efd7e965640 1 --2- 172.21.15.105:0/951185518 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efd78141890 0x7efd78141d10 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7efd74009bb0 tx=0x7efd74004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:32.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.439+0000 7efd6effd640 1 -- 172.21.15.105:0/951185518 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd74039030 con 0x7efd78141890 2024-04-15T15:20:32.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.439+0000 7efd6effd640 1 -- 172.21.15.105:0/951185518 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd74043070 con 0x7efd78141890 2024-04-15T15:20:32.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.439+0000 7efd6effd640 1 -- 172.21.15.105:0/951185518 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd74034ab0 con 0x7efd78141890 2024-04-15T15:20:32.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.439+0000 7efd803c8640 1 -- 172.21.15.105:0/951185518 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd780b6880 con 0x7efd78141890 2024-04-15T15:20:32.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.439+0000 7efd803c8640 1 -- 172.21.15.105:0/951185518 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd780b6e40 con 0x7efd78141890 2024-04-15T15:20:32.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.440+0000 7efd803c8640 1 -- 172.21.15.105:0/951185518 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd780b7b80 con 0x7efd78141890 2024-04-15T15:20:32.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.440+0000 7efd6effd640 1 -- 172.21.15.105:0/951185518 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efd74034c50 con 0x7efd78141890 2024-04-15T15:20:32.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.440+0000 7efd6effd640 1 --2- 172.21.15.105:0/951185518 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7efd5c06c920 0x7efd5c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:32.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.441+0000 7efd6effd640 1 -- 172.21.15.105:0/951185518 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7efd740ac8e0 con 0x7efd78141890 2024-04-15T15:20:32.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.441+0000 7efd7e164640 1 --2- 172.21.15.105:0/951185518 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7efd5c06c920 0x7efd5c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:32.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.441+0000 7efd7e164640 1 --2- 172.21.15.105:0/951185518 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7efd5c06c920 0x7efd5c06ede0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7efd70001c50 tx=0x7efd70009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:32.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.445+0000 7efd6effd640 1 -- 172.21.15.105:0/951185518 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efd780b7b80 con 0x7efd78141890 2024-04-15T15:20:32.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.618+0000 7efd803c8640 1 -- 172.21.15.105:0/951185518 --> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efd78068a40 con 0x7efd5c06c920 2024-04-15T15:20:32.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.624+0000 7efd6effd640 1 -- 172.21.15.105:0/951185518 <== mgr.4107 v2:172.21.15.105:6832/3506549322 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246632 (secure 0 0 0) 0x7efd78068a40 con 0x7efd5c06c920 2024-04-15T15:20:32.629 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:20:32.630 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-04-15T15:20:32.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.627+0000 7efd803c8640 1 -- 172.21.15.105:0/951185518 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7efd5c06c920 msgr2=0x7efd5c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:32.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.627+0000 7efd803c8640 1 --2- 172.21.15.105:0/951185518 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7efd5c06c920 0x7efd5c06ede0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7efd70001c50 tx=0x7efd70009040 comp rx=0 tx=0).stop 2024-04-15T15:20:32.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.627+0000 7efd803c8640 1 -- 172.21.15.105:0/951185518 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efd78141890 msgr2=0x7efd78141d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:32.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.627+0000 7efd803c8640 1 --2- 172.21.15.105:0/951185518 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efd78141890 0x7efd78141d10 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7efd74009bb0 tx=0x7efd74004730 comp rx=0 tx=0).stop 2024-04-15T15:20:32.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.627+0000 7efd803c8640 1 -- 172.21.15.105:0/951185518 shutdown_connections 2024-04-15T15:20:32.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.627+0000 7efd803c8640 1 --2- 172.21.15.105:0/951185518 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7efd5c06c920 0x7efd5c06ede0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:32.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.627+0000 7efd803c8640 1 --2- 172.21.15.105:0/951185518 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efd78141890 0x7efd78141d10 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:32.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.627+0000 7efd803c8640 1 --2- 172.21.15.105:0/951185518 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd780c9620 0x7efd780c9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:32.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.627+0000 7efd803c8640 1 --2- 172.21.15.105:0/951185518 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efd64004d40 0x7efd780b7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:32.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.627+0000 7efd803c8640 1 -- 172.21.15.105:0/951185518 >> 172.21.15.105:0/951185518 conn(0x7efd780b4040 msgr2=0x7efd780ce770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:32.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.627+0000 7efd803c8640 1 -- 172.21.15.105:0/951185518 shutdown_connections 2024-04-15T15:20:32.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.627+0000 7efd803c8640 1 -- 172.21.15.105:0/951185518 wait complete. 2024-04-15T15:20:32.640 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":66,"stamp":"2024-04-15T15:20:31.745917+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220056,"kb_used_data":3892,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754664,"statfs":{"total":773094113280,"available":772868775936,"internally_reserved":0,"allocated":3985408,"data_stored":2247651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.106189"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.648653+0000","last_change":"2024-04-15T15:20:28.648653+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.648653+0000","last_undegraded":"2024-04-15T15:20:28.648653+0000","last_fullsized":"2024-04-15T15:20:28.648653+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.657541+0000","last_change":"2024-04-15T15:20:28.657541+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.657541+0000","last_undegraded":"2024-04-15T15:20:28.657541+0000","last_fullsized":"2024-04-15T15:20:28.657541+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.654113+0000","last_change":"2024-04-15T15:20:28.654113+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.654113+0000","last_undegraded":"2024-04-15T15:20:28.654113+0000","last_fullsized":"2024-04-15T15:20:28.654113+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.656529+0000","last_change":"2024-04-15T15:20:28.656529+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.656529+0000","last_undegraded":"2024-04-15T15:20:28.656529+0000","last_fullsized":"2024-04-15T15:20:28.656529+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.649435+0000","last_change":"2024-04-15T15:20:28.649435+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.649435+0000","last_undegraded":"2024-04-15T15:20:28.649435+0000","last_fullsized":"2024-04-15T15:20:28.649435+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.656782+0000","last_change":"2024-04-15T15:20:28.656782+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.656782+0000","last_undegraded":"2024-04-15T15:20:28.656782+0000","last_fullsized":"2024-04-15T15:20:28.656782+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.653845+0000","last_change":"2024-04-15T15:20:28.653845+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.653845+0000","last_undegraded":"2024-04-15T15:20:28.653845+0000","last_fullsized":"2024-04-15T15:20:28.653845+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.655853+0000","last_change":"2024-04-15T15:20:28.655853+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.655853+0000","last_undegraded":"2024-04-15T15:20:28.655853+0000","last_fullsized":"2024-04-15T15:20:28.655853+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.650952+0000","last_change":"2024-04-15T15:20:28.650952+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.650952+0000","last_undegraded":"2024-04-15T15:20:28.650952+0000","last_fullsized":"2024-04-15T15:20:28.650952+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.656338+0000","last_change":"2024-04-15T15:20:28.656338+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.656338+0000","last_undegraded":"2024-04-15T15:20:28.656338+0000","last_fullsized":"2024-04-15T15:20:28.656338+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.656460+0000","last_change":"2024-04-15T15:20:28.656460+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.656460+0000","last_undegraded":"2024-04-15T15:20:28.656460+0000","last_fullsized":"2024-04-15T15:20:28.656460+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.654814+0000","last_change":"2024-04-15T15:20:28.654814+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.654814+0000","last_undegraded":"2024-04-15T15:20:28.654814+0000","last_fullsized":"2024-04-15T15:20:28.654814+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.648869+0000","last_change":"2024-04-15T15:20:28.648869+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.648869+0000","last_undegraded":"2024-04-15T15:20:28.648869+0000","last_fullsized":"2024-04-15T15:20:28.648869+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.656407+0000","last_change":"2024-04-15T15:20:28.656407+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.656407+0000","last_undegraded":"2024-04-15T15:20:28.656407+0000","last_fullsized":"2024-04-15T15:20:28.656407+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.654309+0000","last_change":"2024-04-15T15:20:28.654309+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.654309+0000","last_undegraded":"2024-04-15T15:20:28.654309+0000","last_fullsized":"2024-04-15T15:20:28.654309+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.653768+0000","last_change":"2024-04-15T15:20:28.653768+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.653768+0000","last_undegraded":"2024-04-15T15:20:28.653768+0000","last_fullsized":"2024-04-15T15:20:28.653768+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.656765+0000","last_change":"2024-04-15T15:20:28.656765+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.656765+0000","last_undegraded":"2024-04-15T15:20:28.656765+0000","last_fullsized":"2024-04-15T15:20:28.656765+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.655906+0000","last_change":"2024-04-15T15:20:28.655906+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.655906+0000","last_undegraded":"2024-04-15T15:20:28.655906+0000","last_fullsized":"2024-04-15T15:20:28.655906+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.655380+0000","last_change":"2024-04-15T15:20:28.655380+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.655380+0000","last_undegraded":"2024-04-15T15:20:28.655380+0000","last_fullsized":"2024-04-15T15:20:28.655380+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.648667+0000","last_change":"2024-04-15T15:20:28.648667+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.648667+0000","last_undegraded":"2024-04-15T15:20:28.648667+0000","last_fullsized":"2024-04-15T15:20:28.648667+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.652939+0000","last_change":"2024-04-15T15:20:28.652939+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.652939+0000","last_undegraded":"2024-04-15T15:20:28.652939+0000","last_fullsized":"2024-04-15T15:20:28.652939+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.652347+0000","last_change":"2024-04-15T15:20:28.652347+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.652347+0000","last_undegraded":"2024-04-15T15:20:28.652347+0000","last_fullsized":"2024-04-15T15:20:28.652347+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.653711+0000","last_change":"2024-04-15T15:20:28.653711+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.653711+0000","last_undegraded":"2024-04-15T15:20:28.653711+0000","last_fullsized":"2024-04-15T15:20:28.653711+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.654389+0000","last_change":"2024-04-15T15:20:28.654389+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.654389+0000","last_undegraded":"2024-04-15T15:20:28.654389+0000","last_fullsized":"2024-04-15T15:20:28.654389+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.656880+0000","last_change":"2024-04-15T15:20:28.656880+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.656880+0000","last_undegraded":"2024-04-15T15:20:28.656880+0000","last_fullsized":"2024-04-15T15:20:28.656880+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.654703+0000","last_change":"2024-04-15T15:20:28.654703+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.654703+0000","last_undegraded":"2024-04-15T15:20:28.654703+0000","last_fullsized":"2024-04-15T15:20:28.654703+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.656635+0000","last_change":"2024-04-15T15:20:28.656635+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.656635+0000","last_undegraded":"2024-04-15T15:20:28.656635+0000","last_fullsized":"2024-04-15T15:20:28.656635+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.654372+0000","last_change":"2024-04-15T15:20:28.654372+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.654372+0000","last_undegraded":"2024-04-15T15:20:28.654372+0000","last_fullsized":"2024-04-15T15:20:28.654372+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.652231+0000","last_change":"2024-04-15T15:20:28.652231+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.652231+0000","last_undegraded":"2024-04-15T15:20:28.652231+0000","last_fullsized":"2024-04-15T15:20:28.652231+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.655536+0000","last_change":"2024-04-15T15:20:28.655536+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.655536+0000","last_undegraded":"2024-04-15T15:20:28.655536+0000","last_fullsized":"2024-04-15T15:20:28.655536+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.648479+0000","last_change":"2024-04-15T15:20:28.648479+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.648479+0000","last_undegraded":"2024-04-15T15:20:28.648479+0000","last_fullsized":"2024-04-15T15:20:28.648479+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.655450+0000","last_change":"2024-04-15T15:20:28.655450+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.655450+0000","last_undegraded":"2024-04-15T15:20:28.655450+0000","last_fullsized":"2024-04-15T15:20:28.655450+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.653482+0000","last_change":"2024-04-15T15:20:28.653482+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.653482+0000","last_undegraded":"2024-04-15T15:20:28.653482+0000","last_fullsized":"2024-04-15T15:20:28.653482+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.664265+0000","last_change":"2024-04-15T15:20:28.659981+0000","last_active":"2024-04-15T15:20:28.664265+0000","last_peered":"2024-04-15T15:20:28.664265+0000","last_clean":"2024-04-15T15:20:28.664265+0000","last_became_active":"2024-04-15T15:20:28.657535+0000","last_became_peered":"2024-04-15T15:20:28.657535+0000","last_unstale":"2024-04-15T15:20:28.664265+0000","last_undegraded":"2024-04-15T15:20:28.664265+0000","last_fullsized":"2024-04-15T15:20:28.664265+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:28:40.356127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.648158+0000","last_change":"2024-04-15T15:20:28.648158+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.648158+0000","last_undegraded":"2024-04-15T15:20:28.648158+0000","last_fullsized":"2024-04-15T15:20:28.648158+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.665310+0000","last_change":"2024-04-15T15:20:28.658521+0000","last_active":"2024-04-15T15:20:28.665310+0000","last_peered":"2024-04-15T15:20:28.665310+0000","last_clean":"2024-04-15T15:20:28.665310+0000","last_became_active":"2024-04-15T15:20:28.658266+0000","last_became_peered":"2024-04-15T15:20:28.658266+0000","last_unstale":"2024-04-15T15:20:28.665310+0000","last_undegraded":"2024-04-15T15:20:28.665310+0000","last_fullsized":"2024-04-15T15:20:28.665310+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:11:11.096685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.652058+0000","last_change":"2024-04-15T15:20:28.652058+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.652058+0000","last_undegraded":"2024-04-15T15:20:28.652058+0000","last_fullsized":"2024-04-15T15:20:28.652058+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.667305+0000","last_change":"2024-04-15T15:20:28.664326+0000","last_active":"2024-04-15T15:20:28.667305+0000","last_peered":"2024-04-15T15:20:28.667305+0000","last_clean":"2024-04-15T15:20:28.667305+0000","last_became_active":"2024-04-15T15:20:28.664077+0000","last_became_peered":"2024-04-15T15:20:28.664077+0000","last_unstale":"2024-04-15T15:20:28.667305+0000","last_undegraded":"2024-04-15T15:20:28.667305+0000","last_fullsized":"2024-04-15T15:20:28.667305+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:53:08.913774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.656218+0000","last_change":"2024-04-15T15:20:28.656218+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.656218+0000","last_undegraded":"2024-04-15T15:20:28.656218+0000","last_fullsized":"2024-04-15T15:20:28.656218+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.661574+0000","last_change":"2024-04-15T15:20:28.659705+0000","last_active":"2024-04-15T15:20:28.661574+0000","last_peered":"2024-04-15T15:20:28.661574+0000","last_clean":"2024-04-15T15:20:28.661574+0000","last_became_active":"2024-04-15T15:20:28.659433+0000","last_became_peered":"2024-04-15T15:20:28.659433+0000","last_unstale":"2024-04-15T15:20:28.661574+0000","last_undegraded":"2024-04-15T15:20:28.661574+0000","last_fullsized":"2024-04-15T15:20:28.661574+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:08:06.553533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.655155+0000","last_change":"2024-04-15T15:20:28.655155+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.655155+0000","last_undegraded":"2024-04-15T15:20:28.655155+0000","last_fullsized":"2024-04-15T15:20:28.655155+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.666747+0000","last_change":"2024-04-15T15:20:28.662442+0000","last_active":"2024-04-15T15:20:28.666747+0000","last_peered":"2024-04-15T15:20:28.666747+0000","last_clean":"2024-04-15T15:20:28.666747+0000","last_became_active":"2024-04-15T15:20:28.661241+0000","last_became_peered":"2024-04-15T15:20:28.661241+0000","last_unstale":"2024-04-15T15:20:28.666747+0000","last_undegraded":"2024-04-15T15:20:28.666747+0000","last_fullsized":"2024-04-15T15:20:28.666747+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:06:50.496578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.653975+0000","last_change":"2024-04-15T15:20:28.653975+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.653975+0000","last_undegraded":"2024-04-15T15:20:28.653975+0000","last_fullsized":"2024-04-15T15:20:28.653975+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.666634+0000","last_change":"2024-04-15T15:20:28.662045+0000","last_active":"2024-04-15T15:20:28.666634+0000","last_peered":"2024-04-15T15:20:28.666634+0000","last_clean":"2024-04-15T15:20:28.666634+0000","last_became_active":"2024-04-15T15:20:28.661769+0000","last_became_peered":"2024-04-15T15:20:28.661769+0000","last_unstale":"2024-04-15T15:20:28.666634+0000","last_undegraded":"2024-04-15T15:20:28.666634+0000","last_fullsized":"2024-04-15T15:20:28.666634+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:50:03.424628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.651506+0000","last_change":"2024-04-15T15:20:28.651506+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.651506+0000","last_undegraded":"2024-04-15T15:20:28.651506+0000","last_fullsized":"2024-04-15T15:20:28.651506+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.667091+0000","last_change":"2024-04-15T15:20:28.661631+0000","last_active":"2024-04-15T15:20:28.667091+0000","last_peered":"2024-04-15T15:20:28.667091+0000","last_clean":"2024-04-15T15:20:28.667091+0000","last_became_active":"2024-04-15T15:20:28.661392+0000","last_became_peered":"2024-04-15T15:20:28.661392+0000","last_unstale":"2024-04-15T15:20:28.667091+0000","last_undegraded":"2024-04-15T15:20:28.667091+0000","last_fullsized":"2024-04-15T15:20:28.667091+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:32:32.932055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.655695+0000","last_change":"2024-04-15T15:20:28.655695+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.655695+0000","last_undegraded":"2024-04-15T15:20:28.655695+0000","last_fullsized":"2024-04-15T15:20:28.655695+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.666575+0000","last_change":"2024-04-15T15:20:28.661675+0000","last_active":"2024-04-15T15:20:28.666575+0000","last_peered":"2024-04-15T15:20:28.666575+0000","last_clean":"2024-04-15T15:20:28.666575+0000","last_became_active":"2024-04-15T15:20:28.661368+0000","last_became_peered":"2024-04-15T15:20:28.661368+0000","last_unstale":"2024-04-15T15:20:28.666575+0000","last_undegraded":"2024-04-15T15:20:28.666575+0000","last_fullsized":"2024-04-15T15:20:28.666575+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:48:09.613555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.653028+0000","last_change":"2024-04-15T15:20:28.653028+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.653028+0000","last_undegraded":"2024-04-15T15:20:28.653028+0000","last_fullsized":"2024-04-15T15:20:28.653028+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.669471+0000","last_change":"2024-04-15T15:20:28.665030+0000","last_active":"2024-04-15T15:20:28.669471+0000","last_peered":"2024-04-15T15:20:28.669471+0000","last_clean":"2024-04-15T15:20:28.669471+0000","last_became_active":"2024-04-15T15:20:28.664719+0000","last_became_peered":"2024-04-15T15:20:28.664719+0000","last_unstale":"2024-04-15T15:20:28.669471+0000","last_undegraded":"2024-04-15T15:20:28.669471+0000","last_fullsized":"2024-04-15T15:20:28.669471+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:53:47.542553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.654274+0000","last_change":"2024-04-15T15:20:28.654274+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.654274+0000","last_undegraded":"2024-04-15T15:20:28.654274+0000","last_fullsized":"2024-04-15T15:20:28.654274+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.669518+0000","last_change":"2024-04-15T15:20:28.662386+0000","last_active":"2024-04-15T15:20:28.669518+0000","last_peered":"2024-04-15T15:20:28.669518+0000","last_clean":"2024-04-15T15:20:28.669518+0000","last_became_active":"2024-04-15T15:20:28.662151+0000","last_became_peered":"2024-04-15T15:20:28.662151+0000","last_unstale":"2024-04-15T15:20:28.669518+0000","last_undegraded":"2024-04-15T15:20:28.669518+0000","last_fullsized":"2024-04-15T15:20:28.669518+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T03:16:13.044979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.654303+0000","last_change":"2024-04-15T15:20:28.654303+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.654303+0000","last_undegraded":"2024-04-15T15:20:28.654303+0000","last_fullsized":"2024-04-15T15:20:28.654303+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.664277+0000","last_change":"2024-04-15T15:20:28.662968+0000","last_active":"2024-04-15T15:20:28.664277+0000","last_peered":"2024-04-15T15:20:28.664277+0000","last_clean":"2024-04-15T15:20:28.664277+0000","last_became_active":"2024-04-15T15:20:28.662552+0000","last_became_peered":"2024-04-15T15:20:28.662552+0000","last_unstale":"2024-04-15T15:20:28.664277+0000","last_undegraded":"2024-04-15T15:20:28.664277+0000","last_fullsized":"2024-04-15T15:20:28.664277+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:47:14.250326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.652685+0000","last_change":"2024-04-15T15:20:28.652685+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.652685+0000","last_undegraded":"2024-04-15T15:20:28.652685+0000","last_fullsized":"2024-04-15T15:20:28.652685+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.667302+0000","last_change":"2024-04-15T15:20:28.663559+0000","last_active":"2024-04-15T15:20:28.667302+0000","last_peered":"2024-04-15T15:20:28.667302+0000","last_clean":"2024-04-15T15:20:28.667302+0000","last_became_active":"2024-04-15T15:20:28.662946+0000","last_became_peered":"2024-04-15T15:20:28.662946+0000","last_unstale":"2024-04-15T15:20:28.667302+0000","last_undegraded":"2024-04-15T15:20:28.667302+0000","last_fullsized":"2024-04-15T15:20:28.667302+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:16:18.672475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.655605+0000","last_change":"2024-04-15T15:20:28.655605+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.655605+0000","last_undegraded":"2024-04-15T15:20:28.655605+0000","last_fullsized":"2024-04-15T15:20:28.655605+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.666725+0000","last_change":"2024-04-15T15:20:28.662309+0000","last_active":"2024-04-15T15:20:28.666725+0000","last_peered":"2024-04-15T15:20:28.666725+0000","last_clean":"2024-04-15T15:20:28.666725+0000","last_became_active":"2024-04-15T15:20:28.661434+0000","last_became_peered":"2024-04-15T15:20:28.661434+0000","last_unstale":"2024-04-15T15:20:28.666725+0000","last_undegraded":"2024-04-15T15:20:28.666725+0000","last_fullsized":"2024-04-15T15:20:28.666725+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:51:31.432162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.653771+0000","last_change":"2024-04-15T15:20:28.653771+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.653771+0000","last_undegraded":"2024-04-15T15:20:28.653771+0000","last_fullsized":"2024-04-15T15:20:28.653771+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.669589+0000","last_change":"2024-04-15T15:20:28.665508+0000","last_active":"2024-04-15T15:20:28.669589+0000","last_peered":"2024-04-15T15:20:28.669589+0000","last_clean":"2024-04-15T15:20:28.669589+0000","last_became_active":"2024-04-15T15:20:28.665235+0000","last_became_peered":"2024-04-15T15:20:28.665235+0000","last_unstale":"2024-04-15T15:20:28.669589+0000","last_undegraded":"2024-04-15T15:20:28.669589+0000","last_fullsized":"2024-04-15T15:20:28.669589+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T03:16:13.044979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.654876+0000","last_change":"2024-04-15T15:20:28.654876+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.654876+0000","last_undegraded":"2024-04-15T15:20:28.654876+0000","last_fullsized":"2024-04-15T15:20:28.654876+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.667342+0000","last_change":"2024-04-15T15:20:28.665050+0000","last_active":"2024-04-15T15:20:28.667342+0000","last_peered":"2024-04-15T15:20:28.667342+0000","last_clean":"2024-04-15T15:20:28.667342+0000","last_became_active":"2024-04-15T15:20:28.664787+0000","last_became_peered":"2024-04-15T15:20:28.664787+0000","last_unstale":"2024-04-15T15:20:28.667342+0000","last_undegraded":"2024-04-15T15:20:28.667342+0000","last_fullsized":"2024-04-15T15:20:28.667342+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:10:52.680906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.647786+0000","last_change":"2024-04-15T15:20:28.647786+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.647786+0000","last_undegraded":"2024-04-15T15:20:28.647786+0000","last_fullsized":"2024-04-15T15:20:28.647786+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.669504+0000","last_change":"2024-04-15T15:20:28.665337+0000","last_active":"2024-04-15T15:20:28.669504+0000","last_peered":"2024-04-15T15:20:28.669504+0000","last_clean":"2024-04-15T15:20:28.669504+0000","last_became_active":"2024-04-15T15:20:28.664441+0000","last_became_peered":"2024-04-15T15:20:28.664441+0000","last_unstale":"2024-04-15T15:20:28.669504+0000","last_undegraded":"2024-04-15T15:20:28.669504+0000","last_fullsized":"2024-04-15T15:20:28.669504+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:40:45.758256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.651475+0000","last_change":"2024-04-15T15:20:28.651475+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.651475+0000","last_undegraded":"2024-04-15T15:20:28.651475+0000","last_fullsized":"2024-04-15T15:20:28.651475+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.669608+0000","last_change":"2024-04-15T15:20:28.666179+0000","last_active":"2024-04-15T15:20:28.669608+0000","last_peered":"2024-04-15T15:20:28.669608+0000","last_clean":"2024-04-15T15:20:28.669608+0000","last_became_active":"2024-04-15T15:20:28.665927+0000","last_became_peered":"2024-04-15T15:20:28.665927+0000","last_unstale":"2024-04-15T15:20:28.669608+0000","last_undegraded":"2024-04-15T15:20:28.669608+0000","last_fullsized":"2024-04-15T15:20:28.669608+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:54:49.252832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.653694+0000","last_change":"2024-04-15T15:20:28.653694+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.653694+0000","last_undegraded":"2024-04-15T15:20:28.653694+0000","last_fullsized":"2024-04-15T15:20:28.653694+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.666642+0000","last_change":"2024-04-15T15:20:28.661761+0000","last_active":"2024-04-15T15:20:28.666642+0000","last_peered":"2024-04-15T15:20:28.666642+0000","last_clean":"2024-04-15T15:20:28.666642+0000","last_became_active":"2024-04-15T15:20:28.661249+0000","last_became_peered":"2024-04-15T15:20:28.661249+0000","last_unstale":"2024-04-15T15:20:28.666642+0000","last_undegraded":"2024-04-15T15:20:28.666642+0000","last_fullsized":"2024-04-15T15:20:28.666642+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:40:31.925030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.655279+0000","last_change":"2024-04-15T15:20:28.655279+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.655279+0000","last_undegraded":"2024-04-15T15:20:28.655279+0000","last_fullsized":"2024-04-15T15:20:28.655279+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.660927+0000","last_change":"2024-04-15T15:20:28.657992+0000","last_active":"2024-04-15T15:20:28.660927+0000","last_peered":"2024-04-15T15:20:28.660927+0000","last_clean":"2024-04-15T15:20:28.660927+0000","last_became_active":"2024-04-15T15:20:28.657669+0000","last_became_peered":"2024-04-15T15:20:28.657669+0000","last_unstale":"2024-04-15T15:20:28.660927+0000","last_undegraded":"2024-04-15T15:20:28.660927+0000","last_fullsized":"2024-04-15T15:20:28.660927+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:07:23.296102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.648232+0000","last_change":"2024-04-15T15:20:28.648232+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.648232+0000","last_undegraded":"2024-04-15T15:20:28.648232+0000","last_fullsized":"2024-04-15T15:20:28.648232+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.666683+0000","last_change":"2024-04-15T15:20:28.662320+0000","last_active":"2024-04-15T15:20:28.666683+0000","last_peered":"2024-04-15T15:20:28.666683+0000","last_clean":"2024-04-15T15:20:28.666683+0000","last_became_active":"2024-04-15T15:20:28.662030+0000","last_became_peered":"2024-04-15T15:20:28.662030+0000","last_unstale":"2024-04-15T15:20:28.666683+0000","last_undegraded":"2024-04-15T15:20:28.666683+0000","last_fullsized":"2024-04-15T15:20:28.666683+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:08:06.553533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.648304+0000","last_change":"2024-04-15T15:20:28.648304+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.648304+0000","last_undegraded":"2024-04-15T15:20:28.648304+0000","last_fullsized":"2024-04-15T15:20:28.648304+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.669454+0000","last_change":"2024-04-15T15:20:28.661886+0000","last_active":"2024-04-15T15:20:28.669454+0000","last_peered":"2024-04-15T15:20:28.669454+0000","last_clean":"2024-04-15T15:20:28.669454+0000","last_became_active":"2024-04-15T15:20:28.661595+0000","last_became_peered":"2024-04-15T15:20:28.661595+0000","last_unstale":"2024-04-15T15:20:28.669454+0000","last_undegraded":"2024-04-15T15:20:28.669454+0000","last_fullsized":"2024-04-15T15:20:28.669454+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:28:40.356127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.648412+0000","last_change":"2024-04-15T15:20:28.648412+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.648412+0000","last_undegraded":"2024-04-15T15:20:28.648412+0000","last_fullsized":"2024-04-15T15:20:28.648412+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.666609+0000","last_change":"2024-04-15T15:20:28.660511+0000","last_active":"2024-04-15T15:20:28.666609+0000","last_peered":"2024-04-15T15:20:28.666609+0000","last_clean":"2024-04-15T15:20:28.666609+0000","last_became_active":"2024-04-15T15:20:28.660182+0000","last_became_peered":"2024-04-15T15:20:28.660182+0000","last_unstale":"2024-04-15T15:20:28.666609+0000","last_undegraded":"2024-04-15T15:20:28.666609+0000","last_fullsized":"2024-04-15T15:20:28.666609+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T03:16:13.044979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.648510+0000","last_change":"2024-04-15T15:20:28.648510+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.648510+0000","last_undegraded":"2024-04-15T15:20:28.648510+0000","last_fullsized":"2024-04-15T15:20:28.648510+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.662438+0000","last_change":"2024-04-15T15:20:28.659967+0000","last_active":"2024-04-15T15:20:28.662438+0000","last_peered":"2024-04-15T15:20:28.662438+0000","last_clean":"2024-04-15T15:20:28.662438+0000","last_became_active":"2024-04-15T15:20:28.659679+0000","last_became_peered":"2024-04-15T15:20:28.659679+0000","last_unstale":"2024-04-15T15:20:28.662438+0000","last_undegraded":"2024-04-15T15:20:28.662438+0000","last_fullsized":"2024-04-15T15:20:28.662438+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:33:51.571295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.654328+0000","last_change":"2024-04-15T15:20:28.654328+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.654328+0000","last_undegraded":"2024-04-15T15:20:28.654328+0000","last_fullsized":"2024-04-15T15:20:28.654328+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.662744+0000","last_change":"2024-04-15T15:20:28.658381+0000","last_active":"2024-04-15T15:20:28.662744+0000","last_peered":"2024-04-15T15:20:28.662744+0000","last_clean":"2024-04-15T15:20:28.662744+0000","last_became_active":"2024-04-15T15:20:28.658066+0000","last_became_peered":"2024-04-15T15:20:28.658066+0000","last_unstale":"2024-04-15T15:20:28.662744+0000","last_undegraded":"2024-04-15T15:20:28.662744+0000","last_fullsized":"2024-04-15T15:20:28.662744+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:40:45.758256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.648564+0000","last_change":"2024-04-15T15:20:28.648564+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.648564+0000","last_undegraded":"2024-04-15T15:20:28.648564+0000","last_fullsized":"2024-04-15T15:20:28.648564+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.666958+0000","last_change":"2024-04-15T15:20:28.663868+0000","last_active":"2024-04-15T15:20:28.666958+0000","last_peered":"2024-04-15T15:20:28.666958+0000","last_clean":"2024-04-15T15:20:28.666958+0000","last_became_active":"2024-04-15T15:20:28.663619+0000","last_became_peered":"2024-04-15T15:20:28.663619+0000","last_unstale":"2024-04-15T15:20:28.666958+0000","last_undegraded":"2024-04-15T15:20:28.666958+0000","last_fullsized":"2024-04-15T15:20:28.666958+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:10:59.194609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.653245+0000","last_change":"2024-04-15T15:20:28.653245+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.653245+0000","last_undegraded":"2024-04-15T15:20:28.653245+0000","last_fullsized":"2024-04-15T15:20:28.653245+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.669532+0000","last_change":"2024-04-15T15:20:28.662929+0000","last_active":"2024-04-15T15:20:28.669532+0000","last_peered":"2024-04-15T15:20:28.669532+0000","last_clean":"2024-04-15T15:20:28.669532+0000","last_became_active":"2024-04-15T15:20:28.662666+0000","last_became_peered":"2024-04-15T15:20:28.662666+0000","last_unstale":"2024-04-15T15:20:28.669532+0000","last_undegraded":"2024-04-15T15:20:28.669532+0000","last_fullsized":"2024-04-15T15:20:28.669532+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:54:49.252832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.653064+0000","last_change":"2024-04-15T15:20:28.653064+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.653064+0000","last_undegraded":"2024-04-15T15:20:28.653064+0000","last_fullsized":"2024-04-15T15:20:28.653064+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.664841+0000","last_change":"2024-04-15T15:20:28.660911+0000","last_active":"2024-04-15T15:20:28.664841+0000","last_peered":"2024-04-15T15:20:28.664841+0000","last_clean":"2024-04-15T15:20:28.664841+0000","last_became_active":"2024-04-15T15:20:28.660620+0000","last_became_peered":"2024-04-15T15:20:28.660620+0000","last_unstale":"2024-04-15T15:20:28.664841+0000","last_undegraded":"2024-04-15T15:20:28.664841+0000","last_fullsized":"2024-04-15T15:20:28.664841+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:11:11.096685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.648049+0000","last_change":"2024-04-15T15:20:28.648049+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.648049+0000","last_undegraded":"2024-04-15T15:20:28.648049+0000","last_fullsized":"2024-04-15T15:20:28.648049+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.665340+0000","last_change":"2024-04-15T15:20:28.659430+0000","last_active":"2024-04-15T15:20:28.665340+0000","last_peered":"2024-04-15T15:20:28.665340+0000","last_clean":"2024-04-15T15:20:28.665340+0000","last_became_active":"2024-04-15T15:20:28.659182+0000","last_became_peered":"2024-04-15T15:20:28.659182+0000","last_unstale":"2024-04-15T15:20:28.665340+0000","last_undegraded":"2024-04-15T15:20:28.665340+0000","last_fullsized":"2024-04-15T15:20:28.665340+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:10:59.194609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.652596+0000","last_change":"2024-04-15T15:20:28.652596+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.652596+0000","last_undegraded":"2024-04-15T15:20:28.652596+0000","last_fullsized":"2024-04-15T15:20:28.652596+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.666993+0000","last_change":"2024-04-15T15:20:28.664511+0000","last_active":"2024-04-15T15:20:28.666993+0000","last_peered":"2024-04-15T15:20:28.666993+0000","last_clean":"2024-04-15T15:20:28.666993+0000","last_became_active":"2024-04-15T15:20:28.664269+0000","last_became_peered":"2024-04-15T15:20:28.664269+0000","last_unstale":"2024-04-15T15:20:28.666993+0000","last_undegraded":"2024-04-15T15:20:28.666993+0000","last_fullsized":"2024-04-15T15:20:28.666993+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:28:40.356127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.653762+0000","last_change":"2024-04-15T15:20:28.653762+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.653762+0000","last_undegraded":"2024-04-15T15:20:28.653762+0000","last_fullsized":"2024-04-15T15:20:28.653762+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.661322+0000","last_change":"2024-04-15T15:20:28.658052+0000","last_active":"2024-04-15T15:20:28.661322+0000","last_peered":"2024-04-15T15:20:28.661322+0000","last_clean":"2024-04-15T15:20:28.661322+0000","last_became_active":"2024-04-15T15:20:28.657686+0000","last_became_peered":"2024-04-15T15:20:28.657686+0000","last_unstale":"2024-04-15T15:20:28.661322+0000","last_undegraded":"2024-04-15T15:20:28.661322+0000","last_fullsized":"2024-04-15T15:20:28.661322+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:39:39.696390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.652530+0000","last_change":"2024-04-15T15:20:28.652530+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.652530+0000","last_undegraded":"2024-04-15T15:20:28.652530+0000","last_fullsized":"2024-04-15T15:20:28.652530+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.669517+0000","last_change":"2024-04-15T15:20:28.665696+0000","last_active":"2024-04-15T15:20:28.669517+0000","last_peered":"2024-04-15T15:20:28.669517+0000","last_clean":"2024-04-15T15:20:28.669517+0000","last_became_active":"2024-04-15T15:20:28.665180+0000","last_became_peered":"2024-04-15T15:20:28.665180+0000","last_unstale":"2024-04-15T15:20:28.669517+0000","last_undegraded":"2024-04-15T15:20:28.669517+0000","last_fullsized":"2024-04-15T15:20:28.669517+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:47:14.250326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.654753+0000","last_change":"2024-04-15T15:20:28.654753+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.654753+0000","last_undegraded":"2024-04-15T15:20:28.654753+0000","last_fullsized":"2024-04-15T15:20:28.654753+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.664212+0000","last_change":"2024-04-15T15:20:28.659582+0000","last_active":"2024-04-15T15:20:28.664212+0000","last_peered":"2024-04-15T15:20:28.664212+0000","last_clean":"2024-04-15T15:20:28.664212+0000","last_became_active":"2024-04-15T15:20:28.659266+0000","last_became_peered":"2024-04-15T15:20:28.659266+0000","last_unstale":"2024-04-15T15:20:28.664212+0000","last_undegraded":"2024-04-15T15:20:28.664212+0000","last_fullsized":"2024-04-15T15:20:28.664212+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:10:59.194609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":193,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.656297+0000","last_change":"2024-04-15T15:19:14.878269+0000","last_active":"2024-04-15T15:20:28.656297+0000","last_peered":"2024-04-15T15:20:28.656297+0000","last_clean":"2024-04-15T15:20:28.656297+0000","last_became_active":"2024-04-15T15:19:14.877367+0000","last_became_peered":"2024-04-15T15:19:14.877367+0000","last_unstale":"2024-04-15T15:20:28.656297+0000","last_undegraded":"2024-04-15T15:20:28.656297+0000","last_fullsized":"2024-04-15T15:20:28.656297+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:13.827511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:13.827511+0000","last_clean_scrub_stamp":"2024-04-15T15:19:13.827511+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:32:55.902561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574866,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":942080,"data_stored":722949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":360448,"data_stored":138848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":942080,"data_stored":722949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-15T15:20:32.645 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-15T15:20:32.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.781+0000 7f59cb6eb640 1 Processor -- start 2024-04-15T15:20:32.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.781+0000 7f59cb6eb640 1 -- start start 2024-04-15T15:20:32.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.782+0000 7f59cb6eb640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f59c415ff90 0x7f59c40af410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:32.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.782+0000 7f59cb6eb640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59c40af950 con 0x7f59c40cdfe0 2024-04-15T15:20:32.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.782+0000 7f59cb6eb640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59c40cd110 con 0x7f59c4161100 2024-04-15T15:20:32.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.782+0000 7f59cb6eb640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f59c41615c0 con 0x7f59c415ff90 2024-04-15T15:20:32.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.782+0000 7f59c9487640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f59c415ff90 0x7f59c40af410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:32.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.782+0000 7f59c9487640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f59c415ff90 0x7f59c40af410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45280/0 (socket says 172.21.15.105:45280) 2024-04-15T15:20:32.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.782+0000 7f59c9c88640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f59c40cdfe0 0x7f59c40d0500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:43702/0 (socket says 172.21.15.105:43702) 2024-04-15T15:20:32.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.782+0000 7f59c9487640 1 -- 172.21.15.105:0/4136139133 learned_addr learned my addr 172.21.15.105:0/4136139133 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:32.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.782+0000 7f59c9487640 1 -- 172.21.15.105:0/4136139133 >> v1:172.21.15.154:6789/0 conn(0x7f59c4161100 legacy=0x7f59c40cd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:32.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.782+0000 7f59c9487640 1 -- 172.21.15.105:0/4136139133 >> v1:172.21.15.105:6789/0 conn(0x7f59c40cdfe0 legacy=0x7f59c40d0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:32.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.782+0000 7f59c9487640 1 -- 172.21.15.105:0/4136139133 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59c40ce990 con 0x7f59c415ff90 2024-04-15T15:20:32.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.783+0000 7f59c9487640 1 --2- 172.21.15.105:0/4136139133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59c415ff90 0x7f59c40af410 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f59c0009870 tx=0x7f59c002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=573ab9cd440f3dfa server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:32.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.783+0000 7f59bbfff640 1 -- 172.21.15.105:0/4136139133 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c0034070 con 0x7f59c415ff90 2024-04-15T15:20:32.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.783+0000 7f59bbfff640 1 -- 172.21.15.105:0/4136139133 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59c0030860 con 0x7f59c415ff90 2024-04-15T15:20:32.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.783+0000 7f59bbfff640 1 -- 172.21.15.105:0/4136139133 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c0030c30 con 0x7f59c415ff90 2024-04-15T15:20:32.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.784+0000 7f59cb6eb640 1 -- 172.21.15.105:0/4136139133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59c415ff90 msgr2=0x7f59c40af410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:32.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.784+0000 7f59cb6eb640 1 --2- 172.21.15.105:0/4136139133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59c415ff90 0x7f59c40af410 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f59c0009870 tx=0x7f59c002b9a0 comp rx=0 tx=0).stop 2024-04-15T15:20:32.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.784+0000 7f59cb6eb640 1 -- 172.21.15.105:0/4136139133 shutdown_connections 2024-04-15T15:20:32.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.784+0000 7f59cb6eb640 1 --2- 172.21.15.105:0/4136139133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59c415ff90 0x7f59c40af410 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:32.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.784+0000 7f59cb6eb640 1 -- 172.21.15.105:0/4136139133 >> 172.21.15.105:0/4136139133 conn(0x7f59c40d2600 msgr2=0x7f59c40d2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:32.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.784+0000 7f59cb6eb640 1 -- 172.21.15.105:0/4136139133 shutdown_connections 2024-04-15T15:20:32.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.784+0000 7f59cb6eb640 1 -- 172.21.15.105:0/4136139133 wait complete. 2024-04-15T15:20:32.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.785+0000 7f59cb6eb640 1 Processor -- start 2024-04-15T15:20:32.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.785+0000 7f59cb6eb640 1 -- start start 2024-04-15T15:20:32.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.786+0000 7f59cb6eb640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59c40cdfe0 0x7f59c414bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:32.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.786+0000 7f59cb6eb640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f59c415ff90 0x7f59c40b5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:32.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.786+0000 7f59cb6eb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59c4161100 0x7f59c40b7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:32.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.786+0000 7f59cb6eb640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f59c40ce450 con 0x7f59c4161100 2024-04-15T15:20:32.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.786+0000 7f59cb6eb640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f59c40cd3b0 con 0x7f59c40cdfe0 2024-04-15T15:20:32.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.786+0000 7f59cb6eb640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f59c4160370 con 0x7f59c415ff90 2024-04-15T15:20:32.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.786+0000 7f59c9c88640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59c4161100 0x7f59c40b7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:32.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.786+0000 7f59c9c88640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59c4161100 0x7f59c40b7370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45282/0 (socket says 172.21.15.105:45282) 2024-04-15T15:20:32.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.786+0000 7f59c9c88640 1 -- 172.21.15.105:0/3393743908 learned_addr learned my addr 172.21.15.105:0/3393743908 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:32.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.786+0000 7f59c9487640 1 --2- 172.21.15.105:0/3393743908 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59c40cdfe0 0x7f59c414bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:32.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.786+0000 7f59c8c86640 1 --2- 172.21.15.105:0/3393743908 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f59c415ff90 0x7f59c40b5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:32.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.786+0000 7f59c9c88640 1 -- 172.21.15.105:0/3393743908 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f59c415ff90 msgr2=0x7f59c40b5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:32.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.786+0000 7f59c9c88640 1 --2- 172.21.15.105:0/3393743908 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f59c415ff90 0x7f59c40b5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:32.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.786+0000 7f59c9c88640 1 -- 172.21.15.105:0/3393743908 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59c40cdfe0 msgr2=0x7f59c414bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:32.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.786+0000 7f59c9c88640 1 --2- 172.21.15.105:0/3393743908 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59c40cdfe0 0x7f59c414bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:32.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.786+0000 7f59c9c88640 1 -- 172.21.15.105:0/3393743908 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59c40b6d80 con 0x7f59c4161100 2024-04-15T15:20:32.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.786+0000 7f59c8c86640 1 --2- 172.21.15.105:0/3393743908 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f59c415ff90 0x7f59c40b5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:20:32.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.786+0000 7f59c9c88640 1 --2- 172.21.15.105:0/3393743908 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59c4161100 0x7f59c40b7370 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f59bc00c760 tx=0x7f59bc00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:32.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.786+0000 7f59b9ffb640 1 -- 172.21.15.105:0/3393743908 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59bc017070 con 0x7f59c4161100 2024-04-15T15:20:32.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.786+0000 7f59b9ffb640 1 -- 172.21.15.105:0/3393743908 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59bc00a640 con 0x7f59c4161100 2024-04-15T15:20:32.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.786+0000 7f59cb6eb640 1 -- 172.21.15.105:0/3393743908 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59c40b5ce0 con 0x7f59c4161100 2024-04-15T15:20:32.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.786+0000 7f59b9ffb640 1 -- 172.21.15.105:0/3393743908 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59bc00a9f0 con 0x7f59c4161100 2024-04-15T15:20:32.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.787+0000 7f59cb6eb640 1 -- 172.21.15.105:0/3393743908 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59c40b62a0 con 0x7f59c4161100 2024-04-15T15:20:32.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.788+0000 7f59cb6eb640 1 -- 172.21.15.105:0/3393743908 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59c4068a40 con 0x7f59c4161100 2024-04-15T15:20:32.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.788+0000 7f59b9ffb640 1 -- 172.21.15.105:0/3393743908 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f59bc01a070 con 0x7f59c4161100 2024-04-15T15:20:32.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.789+0000 7f59b9ffb640 1 --2- 172.21.15.105:0/3393743908 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f59a806c920 0x7f59a806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:32.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.789+0000 7f59b9ffb640 1 -- 172.21.15.105:0/3393743908 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f59bc08af20 con 0x7f59c4161100 2024-04-15T15:20:32.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.789+0000 7f59c9487640 1 --2- 172.21.15.105:0/3393743908 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f59a806c920 0x7f59a806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:32.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.789+0000 7f59c9487640 1 --2- 172.21.15.105:0/3393743908 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f59a806c920 0x7f59a806ede0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f59c0036000 tx=0x7f59c0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:32.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.793+0000 7f59b9ffb640 1 -- 172.21.15.105:0/3393743908 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f59bc058d90 con 0x7f59c4161100 2024-04-15T15:20:32.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.965+0000 7f59cb6eb640 1 -- 172.21.15.105:0/3393743908 --> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f59c40b8580 con 0x7f59a806c920 2024-04-15T15:20:32.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.971+0000 7f59b9ffb640 1 -- 172.21.15.105:0/3393743908 <== mgr.4107 v2:172.21.15.105:6832/3506549322 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246632 (secure 0 0 0) 0x7f59c40b8580 con 0x7f59a806c920 2024-04-15T15:20:32.975 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:20:32.976 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-04-15T15:20:32.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.974+0000 7f59cb6eb640 1 -- 172.21.15.105:0/3393743908 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f59a806c920 msgr2=0x7f59a806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:32.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.974+0000 7f59cb6eb640 1 --2- 172.21.15.105:0/3393743908 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f59a806c920 0x7f59a806ede0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f59c0036000 tx=0x7f59c0008030 comp rx=0 tx=0).stop 2024-04-15T15:20:32.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.974+0000 7f59cb6eb640 1 -- 172.21.15.105:0/3393743908 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59c4161100 msgr2=0x7f59c40b7370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:32.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.974+0000 7f59cb6eb640 1 --2- 172.21.15.105:0/3393743908 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59c4161100 0x7f59c40b7370 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f59bc00c760 tx=0x7f59bc00cc20 comp rx=0 tx=0).stop 2024-04-15T15:20:32.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.974+0000 7f59cb6eb640 1 -- 172.21.15.105:0/3393743908 shutdown_connections 2024-04-15T15:20:32.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.974+0000 7f59cb6eb640 1 --2- 172.21.15.105:0/3393743908 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f59c415ff90 0x7f59c40b5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:32.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.974+0000 7f59cb6eb640 1 --2- 172.21.15.105:0/3393743908 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f59a806c920 0x7f59a806ede0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:32.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.974+0000 7f59cb6eb640 1 --2- 172.21.15.105:0/3393743908 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59c4161100 0x7f59c40b7370 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:32.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.974+0000 7f59cb6eb640 1 --2- 172.21.15.105:0/3393743908 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59c40cdfe0 0x7f59c414bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:32.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.974+0000 7f59cb6eb640 1 -- 172.21.15.105:0/3393743908 >> 172.21.15.105:0/3393743908 conn(0x7f59c40d2600 msgr2=0x7f59c40704e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:32.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.974+0000 7f59cb6eb640 1 -- 172.21.15.105:0/3393743908 shutdown_connections 2024-04-15T15:20:32.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:32.974+0000 7f59cb6eb640 1 -- 172.21.15.105:0/3393743908 wait complete. 2024-04-15T15:20:32.986 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":66,"stamp":"2024-04-15T15:20:31.745917+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220056,"kb_used_data":3892,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754664,"statfs":{"total":773094113280,"available":772868775936,"internally_reserved":0,"allocated":3985408,"data_stored":2247651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.106189"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.648653+0000","last_change":"2024-04-15T15:20:28.648653+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.648653+0000","last_undegraded":"2024-04-15T15:20:28.648653+0000","last_fullsized":"2024-04-15T15:20:28.648653+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.657541+0000","last_change":"2024-04-15T15:20:28.657541+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.657541+0000","last_undegraded":"2024-04-15T15:20:28.657541+0000","last_fullsized":"2024-04-15T15:20:28.657541+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.654113+0000","last_change":"2024-04-15T15:20:28.654113+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.654113+0000","last_undegraded":"2024-04-15T15:20:28.654113+0000","last_fullsized":"2024-04-15T15:20:28.654113+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.656529+0000","last_change":"2024-04-15T15:20:28.656529+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.656529+0000","last_undegraded":"2024-04-15T15:20:28.656529+0000","last_fullsized":"2024-04-15T15:20:28.656529+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.649435+0000","last_change":"2024-04-15T15:20:28.649435+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.649435+0000","last_undegraded":"2024-04-15T15:20:28.649435+0000","last_fullsized":"2024-04-15T15:20:28.649435+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.656782+0000","last_change":"2024-04-15T15:20:28.656782+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.656782+0000","last_undegraded":"2024-04-15T15:20:28.656782+0000","last_fullsized":"2024-04-15T15:20:28.656782+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.653845+0000","last_change":"2024-04-15T15:20:28.653845+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.653845+0000","last_undegraded":"2024-04-15T15:20:28.653845+0000","last_fullsized":"2024-04-15T15:20:28.653845+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.655853+0000","last_change":"2024-04-15T15:20:28.655853+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.655853+0000","last_undegraded":"2024-04-15T15:20:28.655853+0000","last_fullsized":"2024-04-15T15:20:28.655853+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.650952+0000","last_change":"2024-04-15T15:20:28.650952+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.650952+0000","last_undegraded":"2024-04-15T15:20:28.650952+0000","last_fullsized":"2024-04-15T15:20:28.650952+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.656338+0000","last_change":"2024-04-15T15:20:28.656338+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.656338+0000","last_undegraded":"2024-04-15T15:20:28.656338+0000","last_fullsized":"2024-04-15T15:20:28.656338+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.656460+0000","last_change":"2024-04-15T15:20:28.656460+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.656460+0000","last_undegraded":"2024-04-15T15:20:28.656460+0000","last_fullsized":"2024-04-15T15:20:28.656460+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.654814+0000","last_change":"2024-04-15T15:20:28.654814+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.654814+0000","last_undegraded":"2024-04-15T15:20:28.654814+0000","last_fullsized":"2024-04-15T15:20:28.654814+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.648869+0000","last_change":"2024-04-15T15:20:28.648869+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.648869+0000","last_undegraded":"2024-04-15T15:20:28.648869+0000","last_fullsized":"2024-04-15T15:20:28.648869+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.656407+0000","last_change":"2024-04-15T15:20:28.656407+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.656407+0000","last_undegraded":"2024-04-15T15:20:28.656407+0000","last_fullsized":"2024-04-15T15:20:28.656407+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.654309+0000","last_change":"2024-04-15T15:20:28.654309+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.654309+0000","last_undegraded":"2024-04-15T15:20:28.654309+0000","last_fullsized":"2024-04-15T15:20:28.654309+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.653768+0000","last_change":"2024-04-15T15:20:28.653768+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.653768+0000","last_undegraded":"2024-04-15T15:20:28.653768+0000","last_fullsized":"2024-04-15T15:20:28.653768+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.656765+0000","last_change":"2024-04-15T15:20:28.656765+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.656765+0000","last_undegraded":"2024-04-15T15:20:28.656765+0000","last_fullsized":"2024-04-15T15:20:28.656765+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.655906+0000","last_change":"2024-04-15T15:20:28.655906+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.655906+0000","last_undegraded":"2024-04-15T15:20:28.655906+0000","last_fullsized":"2024-04-15T15:20:28.655906+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.655380+0000","last_change":"2024-04-15T15:20:28.655380+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.655380+0000","last_undegraded":"2024-04-15T15:20:28.655380+0000","last_fullsized":"2024-04-15T15:20:28.655380+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.648667+0000","last_change":"2024-04-15T15:20:28.648667+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.648667+0000","last_undegraded":"2024-04-15T15:20:28.648667+0000","last_fullsized":"2024-04-15T15:20:28.648667+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.652939+0000","last_change":"2024-04-15T15:20:28.652939+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.652939+0000","last_undegraded":"2024-04-15T15:20:28.652939+0000","last_fullsized":"2024-04-15T15:20:28.652939+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.652347+0000","last_change":"2024-04-15T15:20:28.652347+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.652347+0000","last_undegraded":"2024-04-15T15:20:28.652347+0000","last_fullsized":"2024-04-15T15:20:28.652347+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.653711+0000","last_change":"2024-04-15T15:20:28.653711+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.653711+0000","last_undegraded":"2024-04-15T15:20:28.653711+0000","last_fullsized":"2024-04-15T15:20:28.653711+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.654389+0000","last_change":"2024-04-15T15:20:28.654389+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.654389+0000","last_undegraded":"2024-04-15T15:20:28.654389+0000","last_fullsized":"2024-04-15T15:20:28.654389+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.656880+0000","last_change":"2024-04-15T15:20:28.656880+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.656880+0000","last_undegraded":"2024-04-15T15:20:28.656880+0000","last_fullsized":"2024-04-15T15:20:28.656880+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.654703+0000","last_change":"2024-04-15T15:20:28.654703+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.654703+0000","last_undegraded":"2024-04-15T15:20:28.654703+0000","last_fullsized":"2024-04-15T15:20:28.654703+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.656635+0000","last_change":"2024-04-15T15:20:28.656635+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.656635+0000","last_undegraded":"2024-04-15T15:20:28.656635+0000","last_fullsized":"2024-04-15T15:20:28.656635+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.654372+0000","last_change":"2024-04-15T15:20:28.654372+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.654372+0000","last_undegraded":"2024-04-15T15:20:28.654372+0000","last_fullsized":"2024-04-15T15:20:28.654372+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.652231+0000","last_change":"2024-04-15T15:20:28.652231+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.652231+0000","last_undegraded":"2024-04-15T15:20:28.652231+0000","last_fullsized":"2024-04-15T15:20:28.652231+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.655536+0000","last_change":"2024-04-15T15:20:28.655536+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.655536+0000","last_undegraded":"2024-04-15T15:20:28.655536+0000","last_fullsized":"2024-04-15T15:20:28.655536+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.648479+0000","last_change":"2024-04-15T15:20:28.648479+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.648479+0000","last_undegraded":"2024-04-15T15:20:28.648479+0000","last_fullsized":"2024-04-15T15:20:28.648479+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.655450+0000","last_change":"2024-04-15T15:20:28.655450+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.655450+0000","last_undegraded":"2024-04-15T15:20:28.655450+0000","last_fullsized":"2024-04-15T15:20:28.655450+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.653482+0000","last_change":"2024-04-15T15:20:28.653482+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.653482+0000","last_undegraded":"2024-04-15T15:20:28.653482+0000","last_fullsized":"2024-04-15T15:20:28.653482+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.664265+0000","last_change":"2024-04-15T15:20:28.659981+0000","last_active":"2024-04-15T15:20:28.664265+0000","last_peered":"2024-04-15T15:20:28.664265+0000","last_clean":"2024-04-15T15:20:28.664265+0000","last_became_active":"2024-04-15T15:20:28.657535+0000","last_became_peered":"2024-04-15T15:20:28.657535+0000","last_unstale":"2024-04-15T15:20:28.664265+0000","last_undegraded":"2024-04-15T15:20:28.664265+0000","last_fullsized":"2024-04-15T15:20:28.664265+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:28:40.356127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.648158+0000","last_change":"2024-04-15T15:20:28.648158+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.648158+0000","last_undegraded":"2024-04-15T15:20:28.648158+0000","last_fullsized":"2024-04-15T15:20:28.648158+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.665310+0000","last_change":"2024-04-15T15:20:28.658521+0000","last_active":"2024-04-15T15:20:28.665310+0000","last_peered":"2024-04-15T15:20:28.665310+0000","last_clean":"2024-04-15T15:20:28.665310+0000","last_became_active":"2024-04-15T15:20:28.658266+0000","last_became_peered":"2024-04-15T15:20:28.658266+0000","last_unstale":"2024-04-15T15:20:28.665310+0000","last_undegraded":"2024-04-15T15:20:28.665310+0000","last_fullsized":"2024-04-15T15:20:28.665310+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:11:11.096685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.652058+0000","last_change":"2024-04-15T15:20:28.652058+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.652058+0000","last_undegraded":"2024-04-15T15:20:28.652058+0000","last_fullsized":"2024-04-15T15:20:28.652058+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.667305+0000","last_change":"2024-04-15T15:20:28.664326+0000","last_active":"2024-04-15T15:20:28.667305+0000","last_peered":"2024-04-15T15:20:28.667305+0000","last_clean":"2024-04-15T15:20:28.667305+0000","last_became_active":"2024-04-15T15:20:28.664077+0000","last_became_peered":"2024-04-15T15:20:28.664077+0000","last_unstale":"2024-04-15T15:20:28.667305+0000","last_undegraded":"2024-04-15T15:20:28.667305+0000","last_fullsized":"2024-04-15T15:20:28.667305+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:53:08.913774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.656218+0000","last_change":"2024-04-15T15:20:28.656218+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.656218+0000","last_undegraded":"2024-04-15T15:20:28.656218+0000","last_fullsized":"2024-04-15T15:20:28.656218+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.661574+0000","last_change":"2024-04-15T15:20:28.659705+0000","last_active":"2024-04-15T15:20:28.661574+0000","last_peered":"2024-04-15T15:20:28.661574+0000","last_clean":"2024-04-15T15:20:28.661574+0000","last_became_active":"2024-04-15T15:20:28.659433+0000","last_became_peered":"2024-04-15T15:20:28.659433+0000","last_unstale":"2024-04-15T15:20:28.661574+0000","last_undegraded":"2024-04-15T15:20:28.661574+0000","last_fullsized":"2024-04-15T15:20:28.661574+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:08:06.553533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.655155+0000","last_change":"2024-04-15T15:20:28.655155+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.655155+0000","last_undegraded":"2024-04-15T15:20:28.655155+0000","last_fullsized":"2024-04-15T15:20:28.655155+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.666747+0000","last_change":"2024-04-15T15:20:28.662442+0000","last_active":"2024-04-15T15:20:28.666747+0000","last_peered":"2024-04-15T15:20:28.666747+0000","last_clean":"2024-04-15T15:20:28.666747+0000","last_became_active":"2024-04-15T15:20:28.661241+0000","last_became_peered":"2024-04-15T15:20:28.661241+0000","last_unstale":"2024-04-15T15:20:28.666747+0000","last_undegraded":"2024-04-15T15:20:28.666747+0000","last_fullsized":"2024-04-15T15:20:28.666747+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:06:50.496578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.653975+0000","last_change":"2024-04-15T15:20:28.653975+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.653975+0000","last_undegraded":"2024-04-15T15:20:28.653975+0000","last_fullsized":"2024-04-15T15:20:28.653975+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.666634+0000","last_change":"2024-04-15T15:20:28.662045+0000","last_active":"2024-04-15T15:20:28.666634+0000","last_peered":"2024-04-15T15:20:28.666634+0000","last_clean":"2024-04-15T15:20:28.666634+0000","last_became_active":"2024-04-15T15:20:28.661769+0000","last_became_peered":"2024-04-15T15:20:28.661769+0000","last_unstale":"2024-04-15T15:20:28.666634+0000","last_undegraded":"2024-04-15T15:20:28.666634+0000","last_fullsized":"2024-04-15T15:20:28.666634+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:50:03.424628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.651506+0000","last_change":"2024-04-15T15:20:28.651506+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.651506+0000","last_undegraded":"2024-04-15T15:20:28.651506+0000","last_fullsized":"2024-04-15T15:20:28.651506+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.667091+0000","last_change":"2024-04-15T15:20:28.661631+0000","last_active":"2024-04-15T15:20:28.667091+0000","last_peered":"2024-04-15T15:20:28.667091+0000","last_clean":"2024-04-15T15:20:28.667091+0000","last_became_active":"2024-04-15T15:20:28.661392+0000","last_became_peered":"2024-04-15T15:20:28.661392+0000","last_unstale":"2024-04-15T15:20:28.667091+0000","last_undegraded":"2024-04-15T15:20:28.667091+0000","last_fullsized":"2024-04-15T15:20:28.667091+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:32:32.932055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.655695+0000","last_change":"2024-04-15T15:20:28.655695+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.655695+0000","last_undegraded":"2024-04-15T15:20:28.655695+0000","last_fullsized":"2024-04-15T15:20:28.655695+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.666575+0000","last_change":"2024-04-15T15:20:28.661675+0000","last_active":"2024-04-15T15:20:28.666575+0000","last_peered":"2024-04-15T15:20:28.666575+0000","last_clean":"2024-04-15T15:20:28.666575+0000","last_became_active":"2024-04-15T15:20:28.661368+0000","last_became_peered":"2024-04-15T15:20:28.661368+0000","last_unstale":"2024-04-15T15:20:28.666575+0000","last_undegraded":"2024-04-15T15:20:28.666575+0000","last_fullsized":"2024-04-15T15:20:28.666575+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:48:09.613555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.653028+0000","last_change":"2024-04-15T15:20:28.653028+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.653028+0000","last_undegraded":"2024-04-15T15:20:28.653028+0000","last_fullsized":"2024-04-15T15:20:28.653028+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.669471+0000","last_change":"2024-04-15T15:20:28.665030+0000","last_active":"2024-04-15T15:20:28.669471+0000","last_peered":"2024-04-15T15:20:28.669471+0000","last_clean":"2024-04-15T15:20:28.669471+0000","last_became_active":"2024-04-15T15:20:28.664719+0000","last_became_peered":"2024-04-15T15:20:28.664719+0000","last_unstale":"2024-04-15T15:20:28.669471+0000","last_undegraded":"2024-04-15T15:20:28.669471+0000","last_fullsized":"2024-04-15T15:20:28.669471+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:53:47.542553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.654274+0000","last_change":"2024-04-15T15:20:28.654274+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.654274+0000","last_undegraded":"2024-04-15T15:20:28.654274+0000","last_fullsized":"2024-04-15T15:20:28.654274+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.669518+0000","last_change":"2024-04-15T15:20:28.662386+0000","last_active":"2024-04-15T15:20:28.669518+0000","last_peered":"2024-04-15T15:20:28.669518+0000","last_clean":"2024-04-15T15:20:28.669518+0000","last_became_active":"2024-04-15T15:20:28.662151+0000","last_became_peered":"2024-04-15T15:20:28.662151+0000","last_unstale":"2024-04-15T15:20:28.669518+0000","last_undegraded":"2024-04-15T15:20:28.669518+0000","last_fullsized":"2024-04-15T15:20:28.669518+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T03:16:13.044979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.654303+0000","last_change":"2024-04-15T15:20:28.654303+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.654303+0000","last_undegraded":"2024-04-15T15:20:28.654303+0000","last_fullsized":"2024-04-15T15:20:28.654303+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.664277+0000","last_change":"2024-04-15T15:20:28.662968+0000","last_active":"2024-04-15T15:20:28.664277+0000","last_peered":"2024-04-15T15:20:28.664277+0000","last_clean":"2024-04-15T15:20:28.664277+0000","last_became_active":"2024-04-15T15:20:28.662552+0000","last_became_peered":"2024-04-15T15:20:28.662552+0000","last_unstale":"2024-04-15T15:20:28.664277+0000","last_undegraded":"2024-04-15T15:20:28.664277+0000","last_fullsized":"2024-04-15T15:20:28.664277+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:47:14.250326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.652685+0000","last_change":"2024-04-15T15:20:28.652685+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.652685+0000","last_undegraded":"2024-04-15T15:20:28.652685+0000","last_fullsized":"2024-04-15T15:20:28.652685+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.667302+0000","last_change":"2024-04-15T15:20:28.663559+0000","last_active":"2024-04-15T15:20:28.667302+0000","last_peered":"2024-04-15T15:20:28.667302+0000","last_clean":"2024-04-15T15:20:28.667302+0000","last_became_active":"2024-04-15T15:20:28.662946+0000","last_became_peered":"2024-04-15T15:20:28.662946+0000","last_unstale":"2024-04-15T15:20:28.667302+0000","last_undegraded":"2024-04-15T15:20:28.667302+0000","last_fullsized":"2024-04-15T15:20:28.667302+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:16:18.672475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.655605+0000","last_change":"2024-04-15T15:20:28.655605+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.655605+0000","last_undegraded":"2024-04-15T15:20:28.655605+0000","last_fullsized":"2024-04-15T15:20:28.655605+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.666725+0000","last_change":"2024-04-15T15:20:28.662309+0000","last_active":"2024-04-15T15:20:28.666725+0000","last_peered":"2024-04-15T15:20:28.666725+0000","last_clean":"2024-04-15T15:20:28.666725+0000","last_became_active":"2024-04-15T15:20:28.661434+0000","last_became_peered":"2024-04-15T15:20:28.661434+0000","last_unstale":"2024-04-15T15:20:28.666725+0000","last_undegraded":"2024-04-15T15:20:28.666725+0000","last_fullsized":"2024-04-15T15:20:28.666725+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:51:31.432162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.653771+0000","last_change":"2024-04-15T15:20:28.653771+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.653771+0000","last_undegraded":"2024-04-15T15:20:28.653771+0000","last_fullsized":"2024-04-15T15:20:28.653771+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.669589+0000","last_change":"2024-04-15T15:20:28.665508+0000","last_active":"2024-04-15T15:20:28.669589+0000","last_peered":"2024-04-15T15:20:28.669589+0000","last_clean":"2024-04-15T15:20:28.669589+0000","last_became_active":"2024-04-15T15:20:28.665235+0000","last_became_peered":"2024-04-15T15:20:28.665235+0000","last_unstale":"2024-04-15T15:20:28.669589+0000","last_undegraded":"2024-04-15T15:20:28.669589+0000","last_fullsized":"2024-04-15T15:20:28.669589+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T03:16:13.044979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.654876+0000","last_change":"2024-04-15T15:20:28.654876+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.654876+0000","last_undegraded":"2024-04-15T15:20:28.654876+0000","last_fullsized":"2024-04-15T15:20:28.654876+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.667342+0000","last_change":"2024-04-15T15:20:28.665050+0000","last_active":"2024-04-15T15:20:28.667342+0000","last_peered":"2024-04-15T15:20:28.667342+0000","last_clean":"2024-04-15T15:20:28.667342+0000","last_became_active":"2024-04-15T15:20:28.664787+0000","last_became_peered":"2024-04-15T15:20:28.664787+0000","last_unstale":"2024-04-15T15:20:28.667342+0000","last_undegraded":"2024-04-15T15:20:28.667342+0000","last_fullsized":"2024-04-15T15:20:28.667342+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:10:52.680906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.647786+0000","last_change":"2024-04-15T15:20:28.647786+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.647786+0000","last_undegraded":"2024-04-15T15:20:28.647786+0000","last_fullsized":"2024-04-15T15:20:28.647786+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.669504+0000","last_change":"2024-04-15T15:20:28.665337+0000","last_active":"2024-04-15T15:20:28.669504+0000","last_peered":"2024-04-15T15:20:28.669504+0000","last_clean":"2024-04-15T15:20:28.669504+0000","last_became_active":"2024-04-15T15:20:28.664441+0000","last_became_peered":"2024-04-15T15:20:28.664441+0000","last_unstale":"2024-04-15T15:20:28.669504+0000","last_undegraded":"2024-04-15T15:20:28.669504+0000","last_fullsized":"2024-04-15T15:20:28.669504+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:40:45.758256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.651475+0000","last_change":"2024-04-15T15:20:28.651475+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.651475+0000","last_undegraded":"2024-04-15T15:20:28.651475+0000","last_fullsized":"2024-04-15T15:20:28.651475+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.669608+0000","last_change":"2024-04-15T15:20:28.666179+0000","last_active":"2024-04-15T15:20:28.669608+0000","last_peered":"2024-04-15T15:20:28.669608+0000","last_clean":"2024-04-15T15:20:28.669608+0000","last_became_active":"2024-04-15T15:20:28.665927+0000","last_became_peered":"2024-04-15T15:20:28.665927+0000","last_unstale":"2024-04-15T15:20:28.669608+0000","last_undegraded":"2024-04-15T15:20:28.669608+0000","last_fullsized":"2024-04-15T15:20:28.669608+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:54:49.252832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.653694+0000","last_change":"2024-04-15T15:20:28.653694+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.653694+0000","last_undegraded":"2024-04-15T15:20:28.653694+0000","last_fullsized":"2024-04-15T15:20:28.653694+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.666642+0000","last_change":"2024-04-15T15:20:28.661761+0000","last_active":"2024-04-15T15:20:28.666642+0000","last_peered":"2024-04-15T15:20:28.666642+0000","last_clean":"2024-04-15T15:20:28.666642+0000","last_became_active":"2024-04-15T15:20:28.661249+0000","last_became_peered":"2024-04-15T15:20:28.661249+0000","last_unstale":"2024-04-15T15:20:28.666642+0000","last_undegraded":"2024-04-15T15:20:28.666642+0000","last_fullsized":"2024-04-15T15:20:28.666642+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:40:31.925030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.655279+0000","last_change":"2024-04-15T15:20:28.655279+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.655279+0000","last_undegraded":"2024-04-15T15:20:28.655279+0000","last_fullsized":"2024-04-15T15:20:28.655279+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.660927+0000","last_change":"2024-04-15T15:20:28.657992+0000","last_active":"2024-04-15T15:20:28.660927+0000","last_peered":"2024-04-15T15:20:28.660927+0000","last_clean":"2024-04-15T15:20:28.660927+0000","last_became_active":"2024-04-15T15:20:28.657669+0000","last_became_peered":"2024-04-15T15:20:28.657669+0000","last_unstale":"2024-04-15T15:20:28.660927+0000","last_undegraded":"2024-04-15T15:20:28.660927+0000","last_fullsized":"2024-04-15T15:20:28.660927+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:07:23.296102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.648232+0000","last_change":"2024-04-15T15:20:28.648232+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.648232+0000","last_undegraded":"2024-04-15T15:20:28.648232+0000","last_fullsized":"2024-04-15T15:20:28.648232+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.666683+0000","last_change":"2024-04-15T15:20:28.662320+0000","last_active":"2024-04-15T15:20:28.666683+0000","last_peered":"2024-04-15T15:20:28.666683+0000","last_clean":"2024-04-15T15:20:28.666683+0000","last_became_active":"2024-04-15T15:20:28.662030+0000","last_became_peered":"2024-04-15T15:20:28.662030+0000","last_unstale":"2024-04-15T15:20:28.666683+0000","last_undegraded":"2024-04-15T15:20:28.666683+0000","last_fullsized":"2024-04-15T15:20:28.666683+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:08:06.553533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.648304+0000","last_change":"2024-04-15T15:20:28.648304+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.648304+0000","last_undegraded":"2024-04-15T15:20:28.648304+0000","last_fullsized":"2024-04-15T15:20:28.648304+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.669454+0000","last_change":"2024-04-15T15:20:28.661886+0000","last_active":"2024-04-15T15:20:28.669454+0000","last_peered":"2024-04-15T15:20:28.669454+0000","last_clean":"2024-04-15T15:20:28.669454+0000","last_became_active":"2024-04-15T15:20:28.661595+0000","last_became_peered":"2024-04-15T15:20:28.661595+0000","last_unstale":"2024-04-15T15:20:28.669454+0000","last_undegraded":"2024-04-15T15:20:28.669454+0000","last_fullsized":"2024-04-15T15:20:28.669454+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:28:40.356127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.648412+0000","last_change":"2024-04-15T15:20:28.648412+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.648412+0000","last_undegraded":"2024-04-15T15:20:28.648412+0000","last_fullsized":"2024-04-15T15:20:28.648412+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.666609+0000","last_change":"2024-04-15T15:20:28.660511+0000","last_active":"2024-04-15T15:20:28.666609+0000","last_peered":"2024-04-15T15:20:28.666609+0000","last_clean":"2024-04-15T15:20:28.666609+0000","last_became_active":"2024-04-15T15:20:28.660182+0000","last_became_peered":"2024-04-15T15:20:28.660182+0000","last_unstale":"2024-04-15T15:20:28.666609+0000","last_undegraded":"2024-04-15T15:20:28.666609+0000","last_fullsized":"2024-04-15T15:20:28.666609+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T03:16:13.044979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.648510+0000","last_change":"2024-04-15T15:20:28.648510+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.648510+0000","last_undegraded":"2024-04-15T15:20:28.648510+0000","last_fullsized":"2024-04-15T15:20:28.648510+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.662438+0000","last_change":"2024-04-15T15:20:28.659967+0000","last_active":"2024-04-15T15:20:28.662438+0000","last_peered":"2024-04-15T15:20:28.662438+0000","last_clean":"2024-04-15T15:20:28.662438+0000","last_became_active":"2024-04-15T15:20:28.659679+0000","last_became_peered":"2024-04-15T15:20:28.659679+0000","last_unstale":"2024-04-15T15:20:28.662438+0000","last_undegraded":"2024-04-15T15:20:28.662438+0000","last_fullsized":"2024-04-15T15:20:28.662438+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:33:51.571295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.654328+0000","last_change":"2024-04-15T15:20:28.654328+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.654328+0000","last_undegraded":"2024-04-15T15:20:28.654328+0000","last_fullsized":"2024-04-15T15:20:28.654328+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.662744+0000","last_change":"2024-04-15T15:20:28.658381+0000","last_active":"2024-04-15T15:20:28.662744+0000","last_peered":"2024-04-15T15:20:28.662744+0000","last_clean":"2024-04-15T15:20:28.662744+0000","last_became_active":"2024-04-15T15:20:28.658066+0000","last_became_peered":"2024-04-15T15:20:28.658066+0000","last_unstale":"2024-04-15T15:20:28.662744+0000","last_undegraded":"2024-04-15T15:20:28.662744+0000","last_fullsized":"2024-04-15T15:20:28.662744+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:40:45.758256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.648564+0000","last_change":"2024-04-15T15:20:28.648564+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.648564+0000","last_undegraded":"2024-04-15T15:20:28.648564+0000","last_fullsized":"2024-04-15T15:20:28.648564+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.666958+0000","last_change":"2024-04-15T15:20:28.663868+0000","last_active":"2024-04-15T15:20:28.666958+0000","last_peered":"2024-04-15T15:20:28.666958+0000","last_clean":"2024-04-15T15:20:28.666958+0000","last_became_active":"2024-04-15T15:20:28.663619+0000","last_became_peered":"2024-04-15T15:20:28.663619+0000","last_unstale":"2024-04-15T15:20:28.666958+0000","last_undegraded":"2024-04-15T15:20:28.666958+0000","last_fullsized":"2024-04-15T15:20:28.666958+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:10:59.194609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.653245+0000","last_change":"2024-04-15T15:20:28.653245+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.653245+0000","last_undegraded":"2024-04-15T15:20:28.653245+0000","last_fullsized":"2024-04-15T15:20:28.653245+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.669532+0000","last_change":"2024-04-15T15:20:28.662929+0000","last_active":"2024-04-15T15:20:28.669532+0000","last_peered":"2024-04-15T15:20:28.669532+0000","last_clean":"2024-04-15T15:20:28.669532+0000","last_became_active":"2024-04-15T15:20:28.662666+0000","last_became_peered":"2024-04-15T15:20:28.662666+0000","last_unstale":"2024-04-15T15:20:28.669532+0000","last_undegraded":"2024-04-15T15:20:28.669532+0000","last_fullsized":"2024-04-15T15:20:28.669532+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:54:49.252832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.653064+0000","last_change":"2024-04-15T15:20:28.653064+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.653064+0000","last_undegraded":"2024-04-15T15:20:28.653064+0000","last_fullsized":"2024-04-15T15:20:28.653064+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.664841+0000","last_change":"2024-04-15T15:20:28.660911+0000","last_active":"2024-04-15T15:20:28.664841+0000","last_peered":"2024-04-15T15:20:28.664841+0000","last_clean":"2024-04-15T15:20:28.664841+0000","last_became_active":"2024-04-15T15:20:28.660620+0000","last_became_peered":"2024-04-15T15:20:28.660620+0000","last_unstale":"2024-04-15T15:20:28.664841+0000","last_undegraded":"2024-04-15T15:20:28.664841+0000","last_fullsized":"2024-04-15T15:20:28.664841+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:11:11.096685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.648049+0000","last_change":"2024-04-15T15:20:28.648049+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.648049+0000","last_undegraded":"2024-04-15T15:20:28.648049+0000","last_fullsized":"2024-04-15T15:20:28.648049+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.665340+0000","last_change":"2024-04-15T15:20:28.659430+0000","last_active":"2024-04-15T15:20:28.665340+0000","last_peered":"2024-04-15T15:20:28.665340+0000","last_clean":"2024-04-15T15:20:28.665340+0000","last_became_active":"2024-04-15T15:20:28.659182+0000","last_became_peered":"2024-04-15T15:20:28.659182+0000","last_unstale":"2024-04-15T15:20:28.665340+0000","last_undegraded":"2024-04-15T15:20:28.665340+0000","last_fullsized":"2024-04-15T15:20:28.665340+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:10:59.194609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.652596+0000","last_change":"2024-04-15T15:20:28.652596+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.652596+0000","last_undegraded":"2024-04-15T15:20:28.652596+0000","last_fullsized":"2024-04-15T15:20:28.652596+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.666993+0000","last_change":"2024-04-15T15:20:28.664511+0000","last_active":"2024-04-15T15:20:28.666993+0000","last_peered":"2024-04-15T15:20:28.666993+0000","last_clean":"2024-04-15T15:20:28.666993+0000","last_became_active":"2024-04-15T15:20:28.664269+0000","last_became_peered":"2024-04-15T15:20:28.664269+0000","last_unstale":"2024-04-15T15:20:28.666993+0000","last_undegraded":"2024-04-15T15:20:28.666993+0000","last_fullsized":"2024-04-15T15:20:28.666993+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:28:40.356127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.653762+0000","last_change":"2024-04-15T15:20:28.653762+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.653762+0000","last_undegraded":"2024-04-15T15:20:28.653762+0000","last_fullsized":"2024-04-15T15:20:28.653762+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.661322+0000","last_change":"2024-04-15T15:20:28.658052+0000","last_active":"2024-04-15T15:20:28.661322+0000","last_peered":"2024-04-15T15:20:28.661322+0000","last_clean":"2024-04-15T15:20:28.661322+0000","last_became_active":"2024-04-15T15:20:28.657686+0000","last_became_peered":"2024-04-15T15:20:28.657686+0000","last_unstale":"2024-04-15T15:20:28.661322+0000","last_undegraded":"2024-04-15T15:20:28.661322+0000","last_fullsized":"2024-04-15T15:20:28.661322+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:39:39.696390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.652530+0000","last_change":"2024-04-15T15:20:28.652530+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.652530+0000","last_undegraded":"2024-04-15T15:20:28.652530+0000","last_fullsized":"2024-04-15T15:20:28.652530+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.669517+0000","last_change":"2024-04-15T15:20:28.665696+0000","last_active":"2024-04-15T15:20:28.669517+0000","last_peered":"2024-04-15T15:20:28.669517+0000","last_clean":"2024-04-15T15:20:28.669517+0000","last_became_active":"2024-04-15T15:20:28.665180+0000","last_became_peered":"2024-04-15T15:20:28.665180+0000","last_unstale":"2024-04-15T15:20:28.669517+0000","last_undegraded":"2024-04-15T15:20:28.669517+0000","last_fullsized":"2024-04-15T15:20:28.669517+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:47:14.250326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.654753+0000","last_change":"2024-04-15T15:20:28.654753+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.654753+0000","last_undegraded":"2024-04-15T15:20:28.654753+0000","last_fullsized":"2024-04-15T15:20:28.654753+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.664212+0000","last_change":"2024-04-15T15:20:28.659582+0000","last_active":"2024-04-15T15:20:28.664212+0000","last_peered":"2024-04-15T15:20:28.664212+0000","last_clean":"2024-04-15T15:20:28.664212+0000","last_became_active":"2024-04-15T15:20:28.659266+0000","last_became_peered":"2024-04-15T15:20:28.659266+0000","last_unstale":"2024-04-15T15:20:28.664212+0000","last_undegraded":"2024-04-15T15:20:28.664212+0000","last_fullsized":"2024-04-15T15:20:28.664212+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:10:59.194609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":193,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.656297+0000","last_change":"2024-04-15T15:19:14.878269+0000","last_active":"2024-04-15T15:20:28.656297+0000","last_peered":"2024-04-15T15:20:28.656297+0000","last_clean":"2024-04-15T15:20:28.656297+0000","last_became_active":"2024-04-15T15:19:14.877367+0000","last_became_peered":"2024-04-15T15:19:14.877367+0000","last_unstale":"2024-04-15T15:20:28.656297+0000","last_undegraded":"2024-04-15T15:20:28.656297+0000","last_fullsized":"2024-04-15T15:20:28.656297+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:13.827511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:13.827511+0000","last_clean_scrub_stamp":"2024-04-15T15:19:13.827511+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:32:55.902561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574866,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":942080,"data_stored":722949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":360448,"data_stored":138848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":942080,"data_stored":722949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-15T15:20:32.990 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-15T15:20:33.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.129+0000 7fbfc61f4640 1 Processor -- start 2024-04-15T15:20:33.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.129+0000 7fbfc61f4640 1 -- start start 2024-04-15T15:20:33.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.130+0000 7fbfc61f4640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbfc0071e60 0x7fbfc00722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:33.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.130+0000 7fbfc61f4640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbfc006df80 0x7fbfc006e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:33.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.130+0000 7fbfc61f4640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfc00ccfb0 con 0x7fbfc0145e00 2024-04-15T15:20:33.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.130+0000 7fbfc61f4640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fbfc0072800 con 0x7fbfc0071e60 2024-04-15T15:20:33.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.130+0000 7fbfc61f4640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fbfc00cdbc0 con 0x7fbfc006df80 2024-04-15T15:20:33.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.130+0000 7fbfbeffd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbfc0071e60 0x7fbfc00722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:33.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.130+0000 7fbfbeffd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbfc0071e60 0x7fbfc00722c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45290/0 (socket says 172.21.15.105:45290) 2024-04-15T15:20:33.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.130+0000 7fbfbeffd640 1 -- 172.21.15.105:0/2706767737 learned_addr learned my addr 172.21.15.105:0/2706767737 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:33.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.130+0000 7fbfbffff640 1 --2- 172.21.15.105:0/2706767737 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbfc006df80 0x7fbfc006e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:33.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.130+0000 7fbfbeffd640 1 -- 172.21.15.105:0/2706767737 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbfc006df80 msgr2=0x7fbfc006e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:33.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.130+0000 7fbfbeffd640 1 --2- 172.21.15.105:0/2706767737 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbfc006df80 0x7fbfc006e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:33.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.130+0000 7fbfbeffd640 1 -- 172.21.15.105:0/2706767737 >> v1:172.21.15.154:6789/0 conn(0x7fbfc0145e00 legacy=0x7fbfc00ce7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:33.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.130+0000 7fbfbeffd640 1 -- 172.21.15.105:0/2706767737 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfc00cd290 con 0x7fbfc0071e60 2024-04-15T15:20:33.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.130+0000 7fbfbffff640 1 --2- 172.21.15.105:0/2706767737 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbfc006df80 0x7fbfc006e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:33.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.131+0000 7fbfbeffd640 1 --2- 172.21.15.105:0/2706767737 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfc0071e60 0x7fbfc00722c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4009870 tx=0x7fbfb402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c687f928f7f336a server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:33.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.131+0000 7fbfbe7fc640 1 -- 172.21.15.105:0/2706767737 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb4035020 con 0x7fbfc0071e60 2024-04-15T15:20:33.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.131+0000 7fbfbe7fc640 1 -- 172.21.15.105:0/2706767737 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfb4031640 con 0x7fbfc0071e60 2024-04-15T15:20:33.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.131+0000 7fbfbe7fc640 1 -- 172.21.15.105:0/2706767737 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb402d040 con 0x7fbfc0071e60 2024-04-15T15:20:33.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.132+0000 7fbfc61f4640 1 -- 172.21.15.105:0/2706767737 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfc0071e60 msgr2=0x7fbfc00722c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:33.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.132+0000 7fbfc61f4640 1 --2- 172.21.15.105:0/2706767737 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfc0071e60 0x7fbfc00722c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4009870 tx=0x7fbfb402b8f0 comp rx=0 tx=0).stop 2024-04-15T15:20:33.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.132+0000 7fbfc61f4640 1 -- 172.21.15.105:0/2706767737 shutdown_connections 2024-04-15T15:20:33.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.132+0000 7fbfc61f4640 1 --2- 172.21.15.105:0/2706767737 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbfc006df80 0x7fbfc006e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:33.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.132+0000 7fbfc61f4640 1 --2- 172.21.15.105:0/2706767737 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfc0071e60 0x7fbfc00722c0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:33.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.132+0000 7fbfc61f4640 1 -- 172.21.15.105:0/2706767737 >> 172.21.15.105:0/2706767737 conn(0x7fbfc0067d50 msgr2=0x7fbfc006b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:33.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.132+0000 7fbfc61f4640 1 -- 172.21.15.105:0/2706767737 shutdown_connections 2024-04-15T15:20:33.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.132+0000 7fbfc61f4640 1 -- 172.21.15.105:0/2706767737 wait complete. 2024-04-15T15:20:33.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.133+0000 7fbfc61f4640 1 Processor -- start 2024-04-15T15:20:33.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.133+0000 7fbfc61f4640 1 -- start start 2024-04-15T15:20:33.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.134+0000 7fbfc61f4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbfc006df80 0x7fbfc013ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:33.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.134+0000 7fbfc61f4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfc0071e60 0x7fbfc015ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:33.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.134+0000 7fbfc61f4640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbfc0145e00 0x7fbfc00ac8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:33.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.134+0000 7fbfc61f4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc00cef00 con 0x7fbfc0071e60 2024-04-15T15:20:33.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.134+0000 7fbfc61f4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc00cd850 con 0x7fbfc006df80 2024-04-15T15:20:33.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.134+0000 7fbfc61f4640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fbfc00ce370 con 0x7fbfc0145e00 2024-04-15T15:20:33.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.134+0000 7fbfbeffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfc0071e60 0x7fbfc015ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:33.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.134+0000 7fbfbeffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfc0071e60 0x7fbfc015ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45302/0 (socket says 172.21.15.105:45302) 2024-04-15T15:20:33.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.134+0000 7fbfbeffd640 1 -- 172.21.15.105:0/3723239951 learned_addr learned my addr 172.21.15.105:0/3723239951 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:33.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.134+0000 7fbfbf7fe640 1 --2- 172.21.15.105:0/3723239951 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbfc006df80 0x7fbfc013ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:33.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.134+0000 7fbfbffff640 1 --2- 172.21.15.105:0/3723239951 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbfc0145e00 0x7fbfc00ac8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:33.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.134+0000 7fbfbeffd640 1 -- 172.21.15.105:0/3723239951 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbfc0145e00 msgr2=0x7fbfc00ac8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:33.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.134+0000 7fbfbeffd640 1 --2- 172.21.15.105:0/3723239951 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbfc0145e00 0x7fbfc00ac8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:33.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.134+0000 7fbfbeffd640 1 -- 172.21.15.105:0/3723239951 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbfc006df80 msgr2=0x7fbfc013ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:33.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.134+0000 7fbfbeffd640 1 --2- 172.21.15.105:0/3723239951 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbfc006df80 0x7fbfc013ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:33.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.134+0000 7fbfbeffd640 1 -- 172.21.15.105:0/3723239951 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfc013b060 con 0x7fbfc0071e60 2024-04-15T15:20:33.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.134+0000 7fbfbf7fe640 1 --2- 172.21.15.105:0/3723239951 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbfc006df80 0x7fbfc013ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:20:33.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.134+0000 7fbfbeffd640 1 --2- 172.21.15.105:0/3723239951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfc0071e60 0x7fbfc015ace0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fbfb402b930 tx=0x7fbfb40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:33.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.134+0000 7fbfa7fff640 1 -- 172.21.15.105:0/3723239951 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb4040070 con 0x7fbfc0071e60 2024-04-15T15:20:33.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.134+0000 7fbfa7fff640 1 -- 172.21.15.105:0/3723239951 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfb4039bf0 con 0x7fbfc0071e60 2024-04-15T15:20:33.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.134+0000 7fbfc61f4640 1 -- 172.21.15.105:0/3723239951 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfc015b7a0 con 0x7fbfc0071e60 2024-04-15T15:20:33.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.134+0000 7fbfa7fff640 1 -- 172.21.15.105:0/3723239951 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb4035040 con 0x7fbfc0071e60 2024-04-15T15:20:33.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.135+0000 7fbfc61f4640 1 -- 172.21.15.105:0/3723239951 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfc00aa250 con 0x7fbfc0071e60 2024-04-15T15:20:33.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.135+0000 7fbfc61f4640 1 -- 172.21.15.105:0/3723239951 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfc00aa040 con 0x7fbfc0071e60 2024-04-15T15:20:33.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.136+0000 7fbfa7fff640 1 -- 172.21.15.105:0/3723239951 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbfb404b060 con 0x7fbfc0071e60 2024-04-15T15:20:33.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.136+0000 7fbfa7fff640 1 --2- 172.21.15.105:0/3723239951 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fbfac06c920 0x7fbfac06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:33.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.136+0000 7fbfa7fff640 1 -- 172.21.15.105:0/3723239951 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fbfb40ad690 con 0x7fbfc0071e60 2024-04-15T15:20:33.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.136+0000 7fbfbf7fe640 1 --2- 172.21.15.105:0/3723239951 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fbfac06c920 0x7fbfac06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:33.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.137+0000 7fbfbf7fe640 1 --2- 172.21.15.105:0/3723239951 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fbfac06c920 0x7fbfac06ede0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0006fd0 tx=0x7fbfb0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:33.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.141+0000 7fbfa7fff640 1 -- 172.21.15.105:0/3723239951 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbfb402f300 con 0x7fbfc0071e60 2024-04-15T15:20:33.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.311+0000 7fbfc61f4640 1 -- 172.21.15.105:0/3723239951 --> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbfc0068a40 con 0x7fbfac06c920 2024-04-15T15:20:33.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.317+0000 7fbfa7fff640 1 -- 172.21.15.105:0/3723239951 <== mgr.4107 v2:172.21.15.105:6832/3506549322 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246632 (secure 0 0 0) 0x7fbfc0068a40 con 0x7fbfac06c920 2024-04-15T15:20:33.321 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:20:33.322 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-04-15T15:20:33.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.320+0000 7fbfc61f4640 1 -- 172.21.15.105:0/3723239951 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fbfac06c920 msgr2=0x7fbfac06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:33.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.320+0000 7fbfc61f4640 1 --2- 172.21.15.105:0/3723239951 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fbfac06c920 0x7fbfac06ede0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0006fd0 tx=0x7fbfb0008040 comp rx=0 tx=0).stop 2024-04-15T15:20:33.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.320+0000 7fbfc61f4640 1 -- 172.21.15.105:0/3723239951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfc0071e60 msgr2=0x7fbfc015ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:33.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.320+0000 7fbfc61f4640 1 --2- 172.21.15.105:0/3723239951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfc0071e60 0x7fbfc015ace0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fbfb402b930 tx=0x7fbfb40046c0 comp rx=0 tx=0).stop 2024-04-15T15:20:33.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.320+0000 7fbfc61f4640 1 -- 172.21.15.105:0/3723239951 shutdown_connections 2024-04-15T15:20:33.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.320+0000 7fbfc61f4640 1 --2- 172.21.15.105:0/3723239951 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbfc0145e00 0x7fbfc00ac8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:33.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.320+0000 7fbfc61f4640 1 --2- 172.21.15.105:0/3723239951 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fbfac06c920 0x7fbfac06ede0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:33.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.320+0000 7fbfc61f4640 1 --2- 172.21.15.105:0/3723239951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfc0071e60 0x7fbfc015ace0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:33.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.320+0000 7fbfc61f4640 1 --2- 172.21.15.105:0/3723239951 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbfc006df80 0x7fbfc013ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:33.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.320+0000 7fbfc61f4640 1 -- 172.21.15.105:0/3723239951 >> 172.21.15.105:0/3723239951 conn(0x7fbfc0067d50 msgr2=0x7fbfc015f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:33.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.320+0000 7fbfc61f4640 1 -- 172.21.15.105:0/3723239951 shutdown_connections 2024-04-15T15:20:33.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.320+0000 7fbfc61f4640 1 -- 172.21.15.105:0/3723239951 wait complete. 2024-04-15T15:20:33.332 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":66,"stamp":"2024-04-15T15:20:31.745917+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220056,"kb_used_data":3892,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754664,"statfs":{"total":773094113280,"available":772868775936,"internally_reserved":0,"allocated":3985408,"data_stored":2247651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.106189"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.648653+0000","last_change":"2024-04-15T15:20:28.648653+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.648653+0000","last_undegraded":"2024-04-15T15:20:28.648653+0000","last_fullsized":"2024-04-15T15:20:28.648653+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.657541+0000","last_change":"2024-04-15T15:20:28.657541+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.657541+0000","last_undegraded":"2024-04-15T15:20:28.657541+0000","last_fullsized":"2024-04-15T15:20:28.657541+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.654113+0000","last_change":"2024-04-15T15:20:28.654113+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.654113+0000","last_undegraded":"2024-04-15T15:20:28.654113+0000","last_fullsized":"2024-04-15T15:20:28.654113+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.656529+0000","last_change":"2024-04-15T15:20:28.656529+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.656529+0000","last_undegraded":"2024-04-15T15:20:28.656529+0000","last_fullsized":"2024-04-15T15:20:28.656529+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.649435+0000","last_change":"2024-04-15T15:20:28.649435+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.649435+0000","last_undegraded":"2024-04-15T15:20:28.649435+0000","last_fullsized":"2024-04-15T15:20:28.649435+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.656782+0000","last_change":"2024-04-15T15:20:28.656782+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.656782+0000","last_undegraded":"2024-04-15T15:20:28.656782+0000","last_fullsized":"2024-04-15T15:20:28.656782+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.653845+0000","last_change":"2024-04-15T15:20:28.653845+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.653845+0000","last_undegraded":"2024-04-15T15:20:28.653845+0000","last_fullsized":"2024-04-15T15:20:28.653845+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.655853+0000","last_change":"2024-04-15T15:20:28.655853+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.655853+0000","last_undegraded":"2024-04-15T15:20:28.655853+0000","last_fullsized":"2024-04-15T15:20:28.655853+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.650952+0000","last_change":"2024-04-15T15:20:28.650952+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.650952+0000","last_undegraded":"2024-04-15T15:20:28.650952+0000","last_fullsized":"2024-04-15T15:20:28.650952+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.656338+0000","last_change":"2024-04-15T15:20:28.656338+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.656338+0000","last_undegraded":"2024-04-15T15:20:28.656338+0000","last_fullsized":"2024-04-15T15:20:28.656338+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.656460+0000","last_change":"2024-04-15T15:20:28.656460+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.656460+0000","last_undegraded":"2024-04-15T15:20:28.656460+0000","last_fullsized":"2024-04-15T15:20:28.656460+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.654814+0000","last_change":"2024-04-15T15:20:28.654814+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.654814+0000","last_undegraded":"2024-04-15T15:20:28.654814+0000","last_fullsized":"2024-04-15T15:20:28.654814+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.648869+0000","last_change":"2024-04-15T15:20:28.648869+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.648869+0000","last_undegraded":"2024-04-15T15:20:28.648869+0000","last_fullsized":"2024-04-15T15:20:28.648869+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.656407+0000","last_change":"2024-04-15T15:20:28.656407+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.656407+0000","last_undegraded":"2024-04-15T15:20:28.656407+0000","last_fullsized":"2024-04-15T15:20:28.656407+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.654309+0000","last_change":"2024-04-15T15:20:28.654309+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.654309+0000","last_undegraded":"2024-04-15T15:20:28.654309+0000","last_fullsized":"2024-04-15T15:20:28.654309+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.653768+0000","last_change":"2024-04-15T15:20:28.653768+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.653768+0000","last_undegraded":"2024-04-15T15:20:28.653768+0000","last_fullsized":"2024-04-15T15:20:28.653768+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.656765+0000","last_change":"2024-04-15T15:20:28.656765+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.656765+0000","last_undegraded":"2024-04-15T15:20:28.656765+0000","last_fullsized":"2024-04-15T15:20:28.656765+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.655906+0000","last_change":"2024-04-15T15:20:28.655906+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.655906+0000","last_undegraded":"2024-04-15T15:20:28.655906+0000","last_fullsized":"2024-04-15T15:20:28.655906+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.655380+0000","last_change":"2024-04-15T15:20:28.655380+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.655380+0000","last_undegraded":"2024-04-15T15:20:28.655380+0000","last_fullsized":"2024-04-15T15:20:28.655380+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.648667+0000","last_change":"2024-04-15T15:20:28.648667+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.648667+0000","last_undegraded":"2024-04-15T15:20:28.648667+0000","last_fullsized":"2024-04-15T15:20:28.648667+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.652939+0000","last_change":"2024-04-15T15:20:28.652939+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.652939+0000","last_undegraded":"2024-04-15T15:20:28.652939+0000","last_fullsized":"2024-04-15T15:20:28.652939+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.652347+0000","last_change":"2024-04-15T15:20:28.652347+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.652347+0000","last_undegraded":"2024-04-15T15:20:28.652347+0000","last_fullsized":"2024-04-15T15:20:28.652347+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.653711+0000","last_change":"2024-04-15T15:20:28.653711+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.653711+0000","last_undegraded":"2024-04-15T15:20:28.653711+0000","last_fullsized":"2024-04-15T15:20:28.653711+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.654389+0000","last_change":"2024-04-15T15:20:28.654389+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.654389+0000","last_undegraded":"2024-04-15T15:20:28.654389+0000","last_fullsized":"2024-04-15T15:20:28.654389+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.656880+0000","last_change":"2024-04-15T15:20:28.656880+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.656880+0000","last_undegraded":"2024-04-15T15:20:28.656880+0000","last_fullsized":"2024-04-15T15:20:28.656880+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.654703+0000","last_change":"2024-04-15T15:20:28.654703+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.654703+0000","last_undegraded":"2024-04-15T15:20:28.654703+0000","last_fullsized":"2024-04-15T15:20:28.654703+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.656635+0000","last_change":"2024-04-15T15:20:28.656635+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.656635+0000","last_undegraded":"2024-04-15T15:20:28.656635+0000","last_fullsized":"2024-04-15T15:20:28.656635+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.654372+0000","last_change":"2024-04-15T15:20:28.654372+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.654372+0000","last_undegraded":"2024-04-15T15:20:28.654372+0000","last_fullsized":"2024-04-15T15:20:28.654372+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.652231+0000","last_change":"2024-04-15T15:20:28.652231+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.652231+0000","last_undegraded":"2024-04-15T15:20:28.652231+0000","last_fullsized":"2024-04-15T15:20:28.652231+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.655536+0000","last_change":"2024-04-15T15:20:28.655536+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.655536+0000","last_undegraded":"2024-04-15T15:20:28.655536+0000","last_fullsized":"2024-04-15T15:20:28.655536+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.648479+0000","last_change":"2024-04-15T15:20:28.648479+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.648479+0000","last_undegraded":"2024-04-15T15:20:28.648479+0000","last_fullsized":"2024-04-15T15:20:28.648479+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.655450+0000","last_change":"2024-04-15T15:20:28.655450+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.655450+0000","last_undegraded":"2024-04-15T15:20:28.655450+0000","last_fullsized":"2024-04-15T15:20:28.655450+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.653482+0000","last_change":"2024-04-15T15:20:28.653482+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.653482+0000","last_undegraded":"2024-04-15T15:20:28.653482+0000","last_fullsized":"2024-04-15T15:20:28.653482+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.664265+0000","last_change":"2024-04-15T15:20:28.659981+0000","last_active":"2024-04-15T15:20:28.664265+0000","last_peered":"2024-04-15T15:20:28.664265+0000","last_clean":"2024-04-15T15:20:28.664265+0000","last_became_active":"2024-04-15T15:20:28.657535+0000","last_became_peered":"2024-04-15T15:20:28.657535+0000","last_unstale":"2024-04-15T15:20:28.664265+0000","last_undegraded":"2024-04-15T15:20:28.664265+0000","last_fullsized":"2024-04-15T15:20:28.664265+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:28:40.356127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.648158+0000","last_change":"2024-04-15T15:20:28.648158+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.648158+0000","last_undegraded":"2024-04-15T15:20:28.648158+0000","last_fullsized":"2024-04-15T15:20:28.648158+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.665310+0000","last_change":"2024-04-15T15:20:28.658521+0000","last_active":"2024-04-15T15:20:28.665310+0000","last_peered":"2024-04-15T15:20:28.665310+0000","last_clean":"2024-04-15T15:20:28.665310+0000","last_became_active":"2024-04-15T15:20:28.658266+0000","last_became_peered":"2024-04-15T15:20:28.658266+0000","last_unstale":"2024-04-15T15:20:28.665310+0000","last_undegraded":"2024-04-15T15:20:28.665310+0000","last_fullsized":"2024-04-15T15:20:28.665310+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:11:11.096685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.652058+0000","last_change":"2024-04-15T15:20:28.652058+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.652058+0000","last_undegraded":"2024-04-15T15:20:28.652058+0000","last_fullsized":"2024-04-15T15:20:28.652058+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.667305+0000","last_change":"2024-04-15T15:20:28.664326+0000","last_active":"2024-04-15T15:20:28.667305+0000","last_peered":"2024-04-15T15:20:28.667305+0000","last_clean":"2024-04-15T15:20:28.667305+0000","last_became_active":"2024-04-15T15:20:28.664077+0000","last_became_peered":"2024-04-15T15:20:28.664077+0000","last_unstale":"2024-04-15T15:20:28.667305+0000","last_undegraded":"2024-04-15T15:20:28.667305+0000","last_fullsized":"2024-04-15T15:20:28.667305+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:53:08.913774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.656218+0000","last_change":"2024-04-15T15:20:28.656218+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.656218+0000","last_undegraded":"2024-04-15T15:20:28.656218+0000","last_fullsized":"2024-04-15T15:20:28.656218+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.661574+0000","last_change":"2024-04-15T15:20:28.659705+0000","last_active":"2024-04-15T15:20:28.661574+0000","last_peered":"2024-04-15T15:20:28.661574+0000","last_clean":"2024-04-15T15:20:28.661574+0000","last_became_active":"2024-04-15T15:20:28.659433+0000","last_became_peered":"2024-04-15T15:20:28.659433+0000","last_unstale":"2024-04-15T15:20:28.661574+0000","last_undegraded":"2024-04-15T15:20:28.661574+0000","last_fullsized":"2024-04-15T15:20:28.661574+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:08:06.553533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.655155+0000","last_change":"2024-04-15T15:20:28.655155+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.655155+0000","last_undegraded":"2024-04-15T15:20:28.655155+0000","last_fullsized":"2024-04-15T15:20:28.655155+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.666747+0000","last_change":"2024-04-15T15:20:28.662442+0000","last_active":"2024-04-15T15:20:28.666747+0000","last_peered":"2024-04-15T15:20:28.666747+0000","last_clean":"2024-04-15T15:20:28.666747+0000","last_became_active":"2024-04-15T15:20:28.661241+0000","last_became_peered":"2024-04-15T15:20:28.661241+0000","last_unstale":"2024-04-15T15:20:28.666747+0000","last_undegraded":"2024-04-15T15:20:28.666747+0000","last_fullsized":"2024-04-15T15:20:28.666747+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:06:50.496578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.653975+0000","last_change":"2024-04-15T15:20:28.653975+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.653975+0000","last_undegraded":"2024-04-15T15:20:28.653975+0000","last_fullsized":"2024-04-15T15:20:28.653975+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.666634+0000","last_change":"2024-04-15T15:20:28.662045+0000","last_active":"2024-04-15T15:20:28.666634+0000","last_peered":"2024-04-15T15:20:28.666634+0000","last_clean":"2024-04-15T15:20:28.666634+0000","last_became_active":"2024-04-15T15:20:28.661769+0000","last_became_peered":"2024-04-15T15:20:28.661769+0000","last_unstale":"2024-04-15T15:20:28.666634+0000","last_undegraded":"2024-04-15T15:20:28.666634+0000","last_fullsized":"2024-04-15T15:20:28.666634+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:50:03.424628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.651506+0000","last_change":"2024-04-15T15:20:28.651506+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.651506+0000","last_undegraded":"2024-04-15T15:20:28.651506+0000","last_fullsized":"2024-04-15T15:20:28.651506+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.667091+0000","last_change":"2024-04-15T15:20:28.661631+0000","last_active":"2024-04-15T15:20:28.667091+0000","last_peered":"2024-04-15T15:20:28.667091+0000","last_clean":"2024-04-15T15:20:28.667091+0000","last_became_active":"2024-04-15T15:20:28.661392+0000","last_became_peered":"2024-04-15T15:20:28.661392+0000","last_unstale":"2024-04-15T15:20:28.667091+0000","last_undegraded":"2024-04-15T15:20:28.667091+0000","last_fullsized":"2024-04-15T15:20:28.667091+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:32:32.932055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.655695+0000","last_change":"2024-04-15T15:20:28.655695+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.655695+0000","last_undegraded":"2024-04-15T15:20:28.655695+0000","last_fullsized":"2024-04-15T15:20:28.655695+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.666575+0000","last_change":"2024-04-15T15:20:28.661675+0000","last_active":"2024-04-15T15:20:28.666575+0000","last_peered":"2024-04-15T15:20:28.666575+0000","last_clean":"2024-04-15T15:20:28.666575+0000","last_became_active":"2024-04-15T15:20:28.661368+0000","last_became_peered":"2024-04-15T15:20:28.661368+0000","last_unstale":"2024-04-15T15:20:28.666575+0000","last_undegraded":"2024-04-15T15:20:28.666575+0000","last_fullsized":"2024-04-15T15:20:28.666575+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:48:09.613555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.653028+0000","last_change":"2024-04-15T15:20:28.653028+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.653028+0000","last_undegraded":"2024-04-15T15:20:28.653028+0000","last_fullsized":"2024-04-15T15:20:28.653028+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.669471+0000","last_change":"2024-04-15T15:20:28.665030+0000","last_active":"2024-04-15T15:20:28.669471+0000","last_peered":"2024-04-15T15:20:28.669471+0000","last_clean":"2024-04-15T15:20:28.669471+0000","last_became_active":"2024-04-15T15:20:28.664719+0000","last_became_peered":"2024-04-15T15:20:28.664719+0000","last_unstale":"2024-04-15T15:20:28.669471+0000","last_undegraded":"2024-04-15T15:20:28.669471+0000","last_fullsized":"2024-04-15T15:20:28.669471+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:53:47.542553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.654274+0000","last_change":"2024-04-15T15:20:28.654274+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.654274+0000","last_undegraded":"2024-04-15T15:20:28.654274+0000","last_fullsized":"2024-04-15T15:20:28.654274+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.669518+0000","last_change":"2024-04-15T15:20:28.662386+0000","last_active":"2024-04-15T15:20:28.669518+0000","last_peered":"2024-04-15T15:20:28.669518+0000","last_clean":"2024-04-15T15:20:28.669518+0000","last_became_active":"2024-04-15T15:20:28.662151+0000","last_became_peered":"2024-04-15T15:20:28.662151+0000","last_unstale":"2024-04-15T15:20:28.669518+0000","last_undegraded":"2024-04-15T15:20:28.669518+0000","last_fullsized":"2024-04-15T15:20:28.669518+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T03:16:13.044979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.654303+0000","last_change":"2024-04-15T15:20:28.654303+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.654303+0000","last_undegraded":"2024-04-15T15:20:28.654303+0000","last_fullsized":"2024-04-15T15:20:28.654303+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.664277+0000","last_change":"2024-04-15T15:20:28.662968+0000","last_active":"2024-04-15T15:20:28.664277+0000","last_peered":"2024-04-15T15:20:28.664277+0000","last_clean":"2024-04-15T15:20:28.664277+0000","last_became_active":"2024-04-15T15:20:28.662552+0000","last_became_peered":"2024-04-15T15:20:28.662552+0000","last_unstale":"2024-04-15T15:20:28.664277+0000","last_undegraded":"2024-04-15T15:20:28.664277+0000","last_fullsized":"2024-04-15T15:20:28.664277+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:47:14.250326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.652685+0000","last_change":"2024-04-15T15:20:28.652685+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.652685+0000","last_undegraded":"2024-04-15T15:20:28.652685+0000","last_fullsized":"2024-04-15T15:20:28.652685+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.667302+0000","last_change":"2024-04-15T15:20:28.663559+0000","last_active":"2024-04-15T15:20:28.667302+0000","last_peered":"2024-04-15T15:20:28.667302+0000","last_clean":"2024-04-15T15:20:28.667302+0000","last_became_active":"2024-04-15T15:20:28.662946+0000","last_became_peered":"2024-04-15T15:20:28.662946+0000","last_unstale":"2024-04-15T15:20:28.667302+0000","last_undegraded":"2024-04-15T15:20:28.667302+0000","last_fullsized":"2024-04-15T15:20:28.667302+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:16:18.672475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.655605+0000","last_change":"2024-04-15T15:20:28.655605+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.655605+0000","last_undegraded":"2024-04-15T15:20:28.655605+0000","last_fullsized":"2024-04-15T15:20:28.655605+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.666725+0000","last_change":"2024-04-15T15:20:28.662309+0000","last_active":"2024-04-15T15:20:28.666725+0000","last_peered":"2024-04-15T15:20:28.666725+0000","last_clean":"2024-04-15T15:20:28.666725+0000","last_became_active":"2024-04-15T15:20:28.661434+0000","last_became_peered":"2024-04-15T15:20:28.661434+0000","last_unstale":"2024-04-15T15:20:28.666725+0000","last_undegraded":"2024-04-15T15:20:28.666725+0000","last_fullsized":"2024-04-15T15:20:28.666725+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:51:31.432162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.653771+0000","last_change":"2024-04-15T15:20:28.653771+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.653771+0000","last_undegraded":"2024-04-15T15:20:28.653771+0000","last_fullsized":"2024-04-15T15:20:28.653771+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.669589+0000","last_change":"2024-04-15T15:20:28.665508+0000","last_active":"2024-04-15T15:20:28.669589+0000","last_peered":"2024-04-15T15:20:28.669589+0000","last_clean":"2024-04-15T15:20:28.669589+0000","last_became_active":"2024-04-15T15:20:28.665235+0000","last_became_peered":"2024-04-15T15:20:28.665235+0000","last_unstale":"2024-04-15T15:20:28.669589+0000","last_undegraded":"2024-04-15T15:20:28.669589+0000","last_fullsized":"2024-04-15T15:20:28.669589+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T03:16:13.044979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.654876+0000","last_change":"2024-04-15T15:20:28.654876+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.654876+0000","last_undegraded":"2024-04-15T15:20:28.654876+0000","last_fullsized":"2024-04-15T15:20:28.654876+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.667342+0000","last_change":"2024-04-15T15:20:28.665050+0000","last_active":"2024-04-15T15:20:28.667342+0000","last_peered":"2024-04-15T15:20:28.667342+0000","last_clean":"2024-04-15T15:20:28.667342+0000","last_became_active":"2024-04-15T15:20:28.664787+0000","last_became_peered":"2024-04-15T15:20:28.664787+0000","last_unstale":"2024-04-15T15:20:28.667342+0000","last_undegraded":"2024-04-15T15:20:28.667342+0000","last_fullsized":"2024-04-15T15:20:28.667342+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:10:52.680906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.647786+0000","last_change":"2024-04-15T15:20:28.647786+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.647786+0000","last_undegraded":"2024-04-15T15:20:28.647786+0000","last_fullsized":"2024-04-15T15:20:28.647786+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.669504+0000","last_change":"2024-04-15T15:20:28.665337+0000","last_active":"2024-04-15T15:20:28.669504+0000","last_peered":"2024-04-15T15:20:28.669504+0000","last_clean":"2024-04-15T15:20:28.669504+0000","last_became_active":"2024-04-15T15:20:28.664441+0000","last_became_peered":"2024-04-15T15:20:28.664441+0000","last_unstale":"2024-04-15T15:20:28.669504+0000","last_undegraded":"2024-04-15T15:20:28.669504+0000","last_fullsized":"2024-04-15T15:20:28.669504+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:40:45.758256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.651475+0000","last_change":"2024-04-15T15:20:28.651475+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.651475+0000","last_undegraded":"2024-04-15T15:20:28.651475+0000","last_fullsized":"2024-04-15T15:20:28.651475+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.669608+0000","last_change":"2024-04-15T15:20:28.666179+0000","last_active":"2024-04-15T15:20:28.669608+0000","last_peered":"2024-04-15T15:20:28.669608+0000","last_clean":"2024-04-15T15:20:28.669608+0000","last_became_active":"2024-04-15T15:20:28.665927+0000","last_became_peered":"2024-04-15T15:20:28.665927+0000","last_unstale":"2024-04-15T15:20:28.669608+0000","last_undegraded":"2024-04-15T15:20:28.669608+0000","last_fullsized":"2024-04-15T15:20:28.669608+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:54:49.252832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.653694+0000","last_change":"2024-04-15T15:20:28.653694+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.653694+0000","last_undegraded":"2024-04-15T15:20:28.653694+0000","last_fullsized":"2024-04-15T15:20:28.653694+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.666642+0000","last_change":"2024-04-15T15:20:28.661761+0000","last_active":"2024-04-15T15:20:28.666642+0000","last_peered":"2024-04-15T15:20:28.666642+0000","last_clean":"2024-04-15T15:20:28.666642+0000","last_became_active":"2024-04-15T15:20:28.661249+0000","last_became_peered":"2024-04-15T15:20:28.661249+0000","last_unstale":"2024-04-15T15:20:28.666642+0000","last_undegraded":"2024-04-15T15:20:28.666642+0000","last_fullsized":"2024-04-15T15:20:28.666642+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:40:31.925030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.655279+0000","last_change":"2024-04-15T15:20:28.655279+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.655279+0000","last_undegraded":"2024-04-15T15:20:28.655279+0000","last_fullsized":"2024-04-15T15:20:28.655279+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.660927+0000","last_change":"2024-04-15T15:20:28.657992+0000","last_active":"2024-04-15T15:20:28.660927+0000","last_peered":"2024-04-15T15:20:28.660927+0000","last_clean":"2024-04-15T15:20:28.660927+0000","last_became_active":"2024-04-15T15:20:28.657669+0000","last_became_peered":"2024-04-15T15:20:28.657669+0000","last_unstale":"2024-04-15T15:20:28.660927+0000","last_undegraded":"2024-04-15T15:20:28.660927+0000","last_fullsized":"2024-04-15T15:20:28.660927+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:07:23.296102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.648232+0000","last_change":"2024-04-15T15:20:28.648232+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.648232+0000","last_undegraded":"2024-04-15T15:20:28.648232+0000","last_fullsized":"2024-04-15T15:20:28.648232+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.666683+0000","last_change":"2024-04-15T15:20:28.662320+0000","last_active":"2024-04-15T15:20:28.666683+0000","last_peered":"2024-04-15T15:20:28.666683+0000","last_clean":"2024-04-15T15:20:28.666683+0000","last_became_active":"2024-04-15T15:20:28.662030+0000","last_became_peered":"2024-04-15T15:20:28.662030+0000","last_unstale":"2024-04-15T15:20:28.666683+0000","last_undegraded":"2024-04-15T15:20:28.666683+0000","last_fullsized":"2024-04-15T15:20:28.666683+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:08:06.553533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.648304+0000","last_change":"2024-04-15T15:20:28.648304+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.648304+0000","last_undegraded":"2024-04-15T15:20:28.648304+0000","last_fullsized":"2024-04-15T15:20:28.648304+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.669454+0000","last_change":"2024-04-15T15:20:28.661886+0000","last_active":"2024-04-15T15:20:28.669454+0000","last_peered":"2024-04-15T15:20:28.669454+0000","last_clean":"2024-04-15T15:20:28.669454+0000","last_became_active":"2024-04-15T15:20:28.661595+0000","last_became_peered":"2024-04-15T15:20:28.661595+0000","last_unstale":"2024-04-15T15:20:28.669454+0000","last_undegraded":"2024-04-15T15:20:28.669454+0000","last_fullsized":"2024-04-15T15:20:28.669454+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:28:40.356127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.648412+0000","last_change":"2024-04-15T15:20:28.648412+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.648412+0000","last_undegraded":"2024-04-15T15:20:28.648412+0000","last_fullsized":"2024-04-15T15:20:28.648412+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.666609+0000","last_change":"2024-04-15T15:20:28.660511+0000","last_active":"2024-04-15T15:20:28.666609+0000","last_peered":"2024-04-15T15:20:28.666609+0000","last_clean":"2024-04-15T15:20:28.666609+0000","last_became_active":"2024-04-15T15:20:28.660182+0000","last_became_peered":"2024-04-15T15:20:28.660182+0000","last_unstale":"2024-04-15T15:20:28.666609+0000","last_undegraded":"2024-04-15T15:20:28.666609+0000","last_fullsized":"2024-04-15T15:20:28.666609+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T03:16:13.044979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.648510+0000","last_change":"2024-04-15T15:20:28.648510+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.648510+0000","last_undegraded":"2024-04-15T15:20:28.648510+0000","last_fullsized":"2024-04-15T15:20:28.648510+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.662438+0000","last_change":"2024-04-15T15:20:28.659967+0000","last_active":"2024-04-15T15:20:28.662438+0000","last_peered":"2024-04-15T15:20:28.662438+0000","last_clean":"2024-04-15T15:20:28.662438+0000","last_became_active":"2024-04-15T15:20:28.659679+0000","last_became_peered":"2024-04-15T15:20:28.659679+0000","last_unstale":"2024-04-15T15:20:28.662438+0000","last_undegraded":"2024-04-15T15:20:28.662438+0000","last_fullsized":"2024-04-15T15:20:28.662438+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:33:51.571295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.654328+0000","last_change":"2024-04-15T15:20:28.654328+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.654328+0000","last_undegraded":"2024-04-15T15:20:28.654328+0000","last_fullsized":"2024-04-15T15:20:28.654328+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.662744+0000","last_change":"2024-04-15T15:20:28.658381+0000","last_active":"2024-04-15T15:20:28.662744+0000","last_peered":"2024-04-15T15:20:28.662744+0000","last_clean":"2024-04-15T15:20:28.662744+0000","last_became_active":"2024-04-15T15:20:28.658066+0000","last_became_peered":"2024-04-15T15:20:28.658066+0000","last_unstale":"2024-04-15T15:20:28.662744+0000","last_undegraded":"2024-04-15T15:20:28.662744+0000","last_fullsized":"2024-04-15T15:20:28.662744+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:40:45.758256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.648564+0000","last_change":"2024-04-15T15:20:28.648564+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.648564+0000","last_undegraded":"2024-04-15T15:20:28.648564+0000","last_fullsized":"2024-04-15T15:20:28.648564+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.666958+0000","last_change":"2024-04-15T15:20:28.663868+0000","last_active":"2024-04-15T15:20:28.666958+0000","last_peered":"2024-04-15T15:20:28.666958+0000","last_clean":"2024-04-15T15:20:28.666958+0000","last_became_active":"2024-04-15T15:20:28.663619+0000","last_became_peered":"2024-04-15T15:20:28.663619+0000","last_unstale":"2024-04-15T15:20:28.666958+0000","last_undegraded":"2024-04-15T15:20:28.666958+0000","last_fullsized":"2024-04-15T15:20:28.666958+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:10:59.194609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.653245+0000","last_change":"2024-04-15T15:20:28.653245+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.653245+0000","last_undegraded":"2024-04-15T15:20:28.653245+0000","last_fullsized":"2024-04-15T15:20:28.653245+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.669532+0000","last_change":"2024-04-15T15:20:28.662929+0000","last_active":"2024-04-15T15:20:28.669532+0000","last_peered":"2024-04-15T15:20:28.669532+0000","last_clean":"2024-04-15T15:20:28.669532+0000","last_became_active":"2024-04-15T15:20:28.662666+0000","last_became_peered":"2024-04-15T15:20:28.662666+0000","last_unstale":"2024-04-15T15:20:28.669532+0000","last_undegraded":"2024-04-15T15:20:28.669532+0000","last_fullsized":"2024-04-15T15:20:28.669532+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:54:49.252832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.653064+0000","last_change":"2024-04-15T15:20:28.653064+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.653064+0000","last_undegraded":"2024-04-15T15:20:28.653064+0000","last_fullsized":"2024-04-15T15:20:28.653064+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.664841+0000","last_change":"2024-04-15T15:20:28.660911+0000","last_active":"2024-04-15T15:20:28.664841+0000","last_peered":"2024-04-15T15:20:28.664841+0000","last_clean":"2024-04-15T15:20:28.664841+0000","last_became_active":"2024-04-15T15:20:28.660620+0000","last_became_peered":"2024-04-15T15:20:28.660620+0000","last_unstale":"2024-04-15T15:20:28.664841+0000","last_undegraded":"2024-04-15T15:20:28.664841+0000","last_fullsized":"2024-04-15T15:20:28.664841+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:11:11.096685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.648049+0000","last_change":"2024-04-15T15:20:28.648049+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.648049+0000","last_undegraded":"2024-04-15T15:20:28.648049+0000","last_fullsized":"2024-04-15T15:20:28.648049+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.665340+0000","last_change":"2024-04-15T15:20:28.659430+0000","last_active":"2024-04-15T15:20:28.665340+0000","last_peered":"2024-04-15T15:20:28.665340+0000","last_clean":"2024-04-15T15:20:28.665340+0000","last_became_active":"2024-04-15T15:20:28.659182+0000","last_became_peered":"2024-04-15T15:20:28.659182+0000","last_unstale":"2024-04-15T15:20:28.665340+0000","last_undegraded":"2024-04-15T15:20:28.665340+0000","last_fullsized":"2024-04-15T15:20:28.665340+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:10:59.194609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.652596+0000","last_change":"2024-04-15T15:20:28.652596+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.652596+0000","last_undegraded":"2024-04-15T15:20:28.652596+0000","last_fullsized":"2024-04-15T15:20:28.652596+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.666993+0000","last_change":"2024-04-15T15:20:28.664511+0000","last_active":"2024-04-15T15:20:28.666993+0000","last_peered":"2024-04-15T15:20:28.666993+0000","last_clean":"2024-04-15T15:20:28.666993+0000","last_became_active":"2024-04-15T15:20:28.664269+0000","last_became_peered":"2024-04-15T15:20:28.664269+0000","last_unstale":"2024-04-15T15:20:28.666993+0000","last_undegraded":"2024-04-15T15:20:28.666993+0000","last_fullsized":"2024-04-15T15:20:28.666993+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:28:40.356127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.653762+0000","last_change":"2024-04-15T15:20:28.653762+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.653762+0000","last_undegraded":"2024-04-15T15:20:28.653762+0000","last_fullsized":"2024-04-15T15:20:28.653762+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.661322+0000","last_change":"2024-04-15T15:20:28.658052+0000","last_active":"2024-04-15T15:20:28.661322+0000","last_peered":"2024-04-15T15:20:28.661322+0000","last_clean":"2024-04-15T15:20:28.661322+0000","last_became_active":"2024-04-15T15:20:28.657686+0000","last_became_peered":"2024-04-15T15:20:28.657686+0000","last_unstale":"2024-04-15T15:20:28.661322+0000","last_undegraded":"2024-04-15T15:20:28.661322+0000","last_fullsized":"2024-04-15T15:20:28.661322+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:39:39.696390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.652530+0000","last_change":"2024-04-15T15:20:28.652530+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.652530+0000","last_undegraded":"2024-04-15T15:20:28.652530+0000","last_fullsized":"2024-04-15T15:20:28.652530+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.669517+0000","last_change":"2024-04-15T15:20:28.665696+0000","last_active":"2024-04-15T15:20:28.669517+0000","last_peered":"2024-04-15T15:20:28.669517+0000","last_clean":"2024-04-15T15:20:28.669517+0000","last_became_active":"2024-04-15T15:20:28.665180+0000","last_became_peered":"2024-04-15T15:20:28.665180+0000","last_unstale":"2024-04-15T15:20:28.669517+0000","last_undegraded":"2024-04-15T15:20:28.669517+0000","last_fullsized":"2024-04-15T15:20:28.669517+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:47:14.250326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.654753+0000","last_change":"2024-04-15T15:20:28.654753+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.654753+0000","last_undegraded":"2024-04-15T15:20:28.654753+0000","last_fullsized":"2024-04-15T15:20:28.654753+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.664212+0000","last_change":"2024-04-15T15:20:28.659582+0000","last_active":"2024-04-15T15:20:28.664212+0000","last_peered":"2024-04-15T15:20:28.664212+0000","last_clean":"2024-04-15T15:20:28.664212+0000","last_became_active":"2024-04-15T15:20:28.659266+0000","last_became_peered":"2024-04-15T15:20:28.659266+0000","last_unstale":"2024-04-15T15:20:28.664212+0000","last_undegraded":"2024-04-15T15:20:28.664212+0000","last_fullsized":"2024-04-15T15:20:28.664212+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:10:59.194609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":193,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.656297+0000","last_change":"2024-04-15T15:19:14.878269+0000","last_active":"2024-04-15T15:20:28.656297+0000","last_peered":"2024-04-15T15:20:28.656297+0000","last_clean":"2024-04-15T15:20:28.656297+0000","last_became_active":"2024-04-15T15:19:14.877367+0000","last_became_peered":"2024-04-15T15:19:14.877367+0000","last_unstale":"2024-04-15T15:20:28.656297+0000","last_undegraded":"2024-04-15T15:20:28.656297+0000","last_fullsized":"2024-04-15T15:20:28.656297+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:13.827511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:13.827511+0000","last_clean_scrub_stamp":"2024-04-15T15:19:13.827511+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:32:55.902561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574866,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":942080,"data_stored":722949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":360448,"data_stored":138848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":942080,"data_stored":722949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-15T15:20:33.337 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3f is not active+clean 2024-04-15T15:20:33.337 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.648653+0000', 'last_change': '2024-04-15T15:20:28.648653+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.648653+0000', 'last_undegraded': '2024-04-15T15:20:28.648653+0000', 'last_fullsized': '2024-04-15T15:20:28.648653+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-15T15:20:33.338 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3e is not active+clean 2024-04-15T15:20:33.338 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.657541+0000', 'last_change': '2024-04-15T15:20:28.657541+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.657541+0000', 'last_undegraded': '2024-04-15T15:20:28.657541+0000', 'last_fullsized': '2024-04-15T15:20:28.657541+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-15T15:20:33.338 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3d is not active+clean 2024-04-15T15:20:33.338 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.654113+0000', 'last_change': '2024-04-15T15:20:28.654113+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.654113+0000', 'last_undegraded': '2024-04-15T15:20:28.654113+0000', 'last_fullsized': '2024-04-15T15:20:28.654113+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-15T15:20:33.338 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3c is not active+clean 2024-04-15T15:20:33.338 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.656529+0000', 'last_change': '2024-04-15T15:20:28.656529+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.656529+0000', 'last_undegraded': '2024-04-15T15:20:28.656529+0000', 'last_fullsized': '2024-04-15T15:20:28.656529+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-15T15:20:33.338 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3b is not active+clean 2024-04-15T15:20:33.338 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.649435+0000', 'last_change': '2024-04-15T15:20:28.649435+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.649435+0000', 'last_undegraded': '2024-04-15T15:20:28.649435+0000', 'last_fullsized': '2024-04-15T15:20:28.649435+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-15T15:20:33.338 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3a is not active+clean 2024-04-15T15:20:33.338 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.656782+0000', 'last_change': '2024-04-15T15:20:28.656782+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.656782+0000', 'last_undegraded': '2024-04-15T15:20:28.656782+0000', 'last_fullsized': '2024-04-15T15:20:28.656782+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-15T15:20:33.339 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.39 is not active+clean 2024-04-15T15:20:33.339 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.653845+0000', 'last_change': '2024-04-15T15:20:28.653845+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.653845+0000', 'last_undegraded': '2024-04-15T15:20:28.653845+0000', 'last_fullsized': '2024-04-15T15:20:28.653845+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-15T15:20:33.339 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.38 is not active+clean 2024-04-15T15:20:33.339 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.655853+0000', 'last_change': '2024-04-15T15:20:28.655853+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.655853+0000', 'last_undegraded': '2024-04-15T15:20:28.655853+0000', 'last_fullsized': '2024-04-15T15:20:28.655853+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-15T15:20:33.339 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.37 is not active+clean 2024-04-15T15:20:33.339 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.650952+0000', 'last_change': '2024-04-15T15:20:28.650952+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.650952+0000', 'last_undegraded': '2024-04-15T15:20:28.650952+0000', 'last_fullsized': '2024-04-15T15:20:28.650952+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-15T15:20:33.339 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.36 is not active+clean 2024-04-15T15:20:33.339 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.656338+0000', 'last_change': '2024-04-15T15:20:28.656338+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.656338+0000', 'last_undegraded': '2024-04-15T15:20:28.656338+0000', 'last_fullsized': '2024-04-15T15:20:28.656338+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-15T15:20:33.339 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.35 is not active+clean 2024-04-15T15:20:33.339 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.656460+0000', 'last_change': '2024-04-15T15:20:28.656460+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.656460+0000', 'last_undegraded': '2024-04-15T15:20:28.656460+0000', 'last_fullsized': '2024-04-15T15:20:28.656460+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-15T15:20:33.339 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.34 is not active+clean 2024-04-15T15:20:33.340 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.654814+0000', 'last_change': '2024-04-15T15:20:28.654814+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.654814+0000', 'last_undegraded': '2024-04-15T15:20:28.654814+0000', 'last_fullsized': '2024-04-15T15:20:28.654814+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-15T15:20:33.340 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.33 is not active+clean 2024-04-15T15:20:33.340 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.648869+0000', 'last_change': '2024-04-15T15:20:28.648869+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.648869+0000', 'last_undegraded': '2024-04-15T15:20:28.648869+0000', 'last_fullsized': '2024-04-15T15:20:28.648869+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-15T15:20:33.340 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.32 is not active+clean 2024-04-15T15:20:33.340 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.656407+0000', 'last_change': '2024-04-15T15:20:28.656407+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.656407+0000', 'last_undegraded': '2024-04-15T15:20:28.656407+0000', 'last_fullsized': '2024-04-15T15:20:28.656407+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-15T15:20:33.340 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.31 is not active+clean 2024-04-15T15:20:33.340 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.654309+0000', 'last_change': '2024-04-15T15:20:28.654309+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.654309+0000', 'last_undegraded': '2024-04-15T15:20:28.654309+0000', 'last_fullsized': '2024-04-15T15:20:28.654309+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-15T15:20:33.340 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.30 is not active+clean 2024-04-15T15:20:33.340 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.653768+0000', 'last_change': '2024-04-15T15:20:28.653768+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.653768+0000', 'last_undegraded': '2024-04-15T15:20:28.653768+0000', 'last_fullsized': '2024-04-15T15:20:28.653768+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-15T15:20:33.341 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2f is not active+clean 2024-04-15T15:20:33.341 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.656765+0000', 'last_change': '2024-04-15T15:20:28.656765+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.656765+0000', 'last_undegraded': '2024-04-15T15:20:28.656765+0000', 'last_fullsized': '2024-04-15T15:20:28.656765+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-15T15:20:33.341 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2e is not active+clean 2024-04-15T15:20:33.341 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.655906+0000', 'last_change': '2024-04-15T15:20:28.655906+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.655906+0000', 'last_undegraded': '2024-04-15T15:20:28.655906+0000', 'last_fullsized': '2024-04-15T15:20:28.655906+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-15T15:20:33.341 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2d is not active+clean 2024-04-15T15:20:33.341 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.655380+0000', 'last_change': '2024-04-15T15:20:28.655380+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.655380+0000', 'last_undegraded': '2024-04-15T15:20:28.655380+0000', 'last_fullsized': '2024-04-15T15:20:28.655380+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-15T15:20:33.341 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2c is not active+clean 2024-04-15T15:20:33.341 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.648667+0000', 'last_change': '2024-04-15T15:20:28.648667+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.648667+0000', 'last_undegraded': '2024-04-15T15:20:28.648667+0000', 'last_fullsized': '2024-04-15T15:20:28.648667+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-15T15:20:33.341 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2b is not active+clean 2024-04-15T15:20:33.342 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.652939+0000', 'last_change': '2024-04-15T15:20:28.652939+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.652939+0000', 'last_undegraded': '2024-04-15T15:20:28.652939+0000', 'last_fullsized': '2024-04-15T15:20:28.652939+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-15T15:20:33.342 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2a is not active+clean 2024-04-15T15:20:33.342 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.652347+0000', 'last_change': '2024-04-15T15:20:28.652347+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.652347+0000', 'last_undegraded': '2024-04-15T15:20:28.652347+0000', 'last_fullsized': '2024-04-15T15:20:28.652347+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-15T15:20:33.342 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.29 is not active+clean 2024-04-15T15:20:33.342 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.653711+0000', 'last_change': '2024-04-15T15:20:28.653711+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.653711+0000', 'last_undegraded': '2024-04-15T15:20:28.653711+0000', 'last_fullsized': '2024-04-15T15:20:28.653711+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-15T15:20:33.342 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.28 is not active+clean 2024-04-15T15:20:33.342 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.654389+0000', 'last_change': '2024-04-15T15:20:28.654389+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.654389+0000', 'last_undegraded': '2024-04-15T15:20:28.654389+0000', 'last_fullsized': '2024-04-15T15:20:28.654389+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-15T15:20:33.342 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.27 is not active+clean 2024-04-15T15:20:33.342 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.656880+0000', 'last_change': '2024-04-15T15:20:28.656880+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.656880+0000', 'last_undegraded': '2024-04-15T15:20:28.656880+0000', 'last_fullsized': '2024-04-15T15:20:28.656880+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-15T15:20:33.342 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.26 is not active+clean 2024-04-15T15:20:33.342 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.654703+0000', 'last_change': '2024-04-15T15:20:28.654703+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.654703+0000', 'last_undegraded': '2024-04-15T15:20:28.654703+0000', 'last_fullsized': '2024-04-15T15:20:28.654703+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-15T15:20:33.343 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.25 is not active+clean 2024-04-15T15:20:33.343 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.656635+0000', 'last_change': '2024-04-15T15:20:28.656635+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.656635+0000', 'last_undegraded': '2024-04-15T15:20:28.656635+0000', 'last_fullsized': '2024-04-15T15:20:28.656635+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-15T15:20:33.343 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.24 is not active+clean 2024-04-15T15:20:33.343 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.654372+0000', 'last_change': '2024-04-15T15:20:28.654372+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.654372+0000', 'last_undegraded': '2024-04-15T15:20:28.654372+0000', 'last_fullsized': '2024-04-15T15:20:28.654372+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-15T15:20:33.343 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.23 is not active+clean 2024-04-15T15:20:33.343 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.652231+0000', 'last_change': '2024-04-15T15:20:28.652231+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.652231+0000', 'last_undegraded': '2024-04-15T15:20:28.652231+0000', 'last_fullsized': '2024-04-15T15:20:28.652231+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-15T15:20:33.343 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.22 is not active+clean 2024-04-15T15:20:33.343 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.655536+0000', 'last_change': '2024-04-15T15:20:28.655536+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.655536+0000', 'last_undegraded': '2024-04-15T15:20:28.655536+0000', 'last_fullsized': '2024-04-15T15:20:28.655536+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-15T15:20:33.343 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.21 is not active+clean 2024-04-15T15:20:33.343 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.648479+0000', 'last_change': '2024-04-15T15:20:28.648479+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.648479+0000', 'last_undegraded': '2024-04-15T15:20:28.648479+0000', 'last_fullsized': '2024-04-15T15:20:28.648479+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-15T15:20:33.344 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.20 is not active+clean 2024-04-15T15:20:33.344 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.655450+0000', 'last_change': '2024-04-15T15:20:28.655450+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.655450+0000', 'last_undegraded': '2024-04-15T15:20:28.655450+0000', 'last_fullsized': '2024-04-15T15:20:28.655450+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-15T15:20:33.344 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1e is not active+clean 2024-04-15T15:20:33.344 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.653482+0000', 'last_change': '2024-04-15T15:20:28.653482+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.653482+0000', 'last_undegraded': '2024-04-15T15:20:28.653482+0000', 'last_fullsized': '2024-04-15T15:20:28.653482+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-15T15:20:33.344 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1f is not active+clean 2024-04-15T15:20:33.344 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.648158+0000', 'last_change': '2024-04-15T15:20:28.648158+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.648158+0000', 'last_undegraded': '2024-04-15T15:20:28.648158+0000', 'last_fullsized': '2024-04-15T15:20:28.648158+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-15T15:20:33.344 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1c is not active+clean 2024-04-15T15:20:33.344 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.652058+0000', 'last_change': '2024-04-15T15:20:28.652058+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.652058+0000', 'last_undegraded': '2024-04-15T15:20:28.652058+0000', 'last_fullsized': '2024-04-15T15:20:28.652058+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-15T15:20:33.344 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1d is not active+clean 2024-04-15T15:20:33.344 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.656218+0000', 'last_change': '2024-04-15T15:20:28.656218+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.656218+0000', 'last_undegraded': '2024-04-15T15:20:28.656218+0000', 'last_fullsized': '2024-04-15T15:20:28.656218+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-15T15:20:33.344 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1a is not active+clean 2024-04-15T15:20:33.344 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.655155+0000', 'last_change': '2024-04-15T15:20:28.655155+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.655155+0000', 'last_undegraded': '2024-04-15T15:20:28.655155+0000', 'last_fullsized': '2024-04-15T15:20:28.655155+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-15T15:20:33.345 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1b is not active+clean 2024-04-15T15:20:33.345 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.653975+0000', 'last_change': '2024-04-15T15:20:28.653975+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.653975+0000', 'last_undegraded': '2024-04-15T15:20:28.653975+0000', 'last_fullsized': '2024-04-15T15:20:28.653975+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-15T15:20:33.345 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.18 is not active+clean 2024-04-15T15:20:33.345 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.651506+0000', 'last_change': '2024-04-15T15:20:28.651506+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.651506+0000', 'last_undegraded': '2024-04-15T15:20:28.651506+0000', 'last_fullsized': '2024-04-15T15:20:28.651506+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-15T15:20:33.345 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.19 is not active+clean 2024-04-15T15:20:33.345 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.655695+0000', 'last_change': '2024-04-15T15:20:28.655695+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.655695+0000', 'last_undegraded': '2024-04-15T15:20:28.655695+0000', 'last_fullsized': '2024-04-15T15:20:28.655695+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-15T15:20:33.345 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.16 is not active+clean 2024-04-15T15:20:33.345 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.653028+0000', 'last_change': '2024-04-15T15:20:28.653028+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.653028+0000', 'last_undegraded': '2024-04-15T15:20:28.653028+0000', 'last_fullsized': '2024-04-15T15:20:28.653028+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-15T15:20:33.345 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.17 is not active+clean 2024-04-15T15:20:33.345 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.654274+0000', 'last_change': '2024-04-15T15:20:28.654274+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.654274+0000', 'last_undegraded': '2024-04-15T15:20:28.654274+0000', 'last_fullsized': '2024-04-15T15:20:28.654274+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-15T15:20:33.345 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.14 is not active+clean 2024-04-15T15:20:33.346 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.654303+0000', 'last_change': '2024-04-15T15:20:28.654303+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.654303+0000', 'last_undegraded': '2024-04-15T15:20:28.654303+0000', 'last_fullsized': '2024-04-15T15:20:28.654303+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-15T15:20:33.346 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.15 is not active+clean 2024-04-15T15:20:33.346 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.652685+0000', 'last_change': '2024-04-15T15:20:28.652685+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.652685+0000', 'last_undegraded': '2024-04-15T15:20:28.652685+0000', 'last_fullsized': '2024-04-15T15:20:28.652685+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-15T15:20:33.346 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.12 is not active+clean 2024-04-15T15:20:33.346 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.655605+0000', 'last_change': '2024-04-15T15:20:28.655605+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.655605+0000', 'last_undegraded': '2024-04-15T15:20:28.655605+0000', 'last_fullsized': '2024-04-15T15:20:28.655605+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-15T15:20:33.346 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.13 is not active+clean 2024-04-15T15:20:33.346 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.653771+0000', 'last_change': '2024-04-15T15:20:28.653771+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.653771+0000', 'last_undegraded': '2024-04-15T15:20:28.653771+0000', 'last_fullsized': '2024-04-15T15:20:28.653771+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-15T15:20:33.346 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.10 is not active+clean 2024-04-15T15:20:33.346 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.654876+0000', 'last_change': '2024-04-15T15:20:28.654876+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.654876+0000', 'last_undegraded': '2024-04-15T15:20:28.654876+0000', 'last_fullsized': '2024-04-15T15:20:28.654876+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-15T15:20:33.346 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.11 is not active+clean 2024-04-15T15:20:33.347 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.647786+0000', 'last_change': '2024-04-15T15:20:28.647786+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.647786+0000', 'last_undegraded': '2024-04-15T15:20:28.647786+0000', 'last_fullsized': '2024-04-15T15:20:28.647786+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-15T15:20:33.347 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.e is not active+clean 2024-04-15T15:20:33.347 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.651475+0000', 'last_change': '2024-04-15T15:20:28.651475+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.651475+0000', 'last_undegraded': '2024-04-15T15:20:28.651475+0000', 'last_fullsized': '2024-04-15T15:20:28.651475+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-15T15:20:33.347 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.f is not active+clean 2024-04-15T15:20:33.347 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.653694+0000', 'last_change': '2024-04-15T15:20:28.653694+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.653694+0000', 'last_undegraded': '2024-04-15T15:20:28.653694+0000', 'last_fullsized': '2024-04-15T15:20:28.653694+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-15T15:20:33.347 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.c is not active+clean 2024-04-15T15:20:33.347 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.655279+0000', 'last_change': '2024-04-15T15:20:28.655279+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.655279+0000', 'last_undegraded': '2024-04-15T15:20:28.655279+0000', 'last_fullsized': '2024-04-15T15:20:28.655279+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-15T15:20:33.347 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.d is not active+clean 2024-04-15T15:20:33.347 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.648232+0000', 'last_change': '2024-04-15T15:20:28.648232+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.648232+0000', 'last_undegraded': '2024-04-15T15:20:28.648232+0000', 'last_fullsized': '2024-04-15T15:20:28.648232+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-15T15:20:33.347 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.a is not active+clean 2024-04-15T15:20:33.347 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.648304+0000', 'last_change': '2024-04-15T15:20:28.648304+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.648304+0000', 'last_undegraded': '2024-04-15T15:20:28.648304+0000', 'last_fullsized': '2024-04-15T15:20:28.648304+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-15T15:20:33.348 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.b is not active+clean 2024-04-15T15:20:33.348 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.648412+0000', 'last_change': '2024-04-15T15:20:28.648412+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.648412+0000', 'last_undegraded': '2024-04-15T15:20:28.648412+0000', 'last_fullsized': '2024-04-15T15:20:28.648412+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-15T15:20:33.348 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.8 is not active+clean 2024-04-15T15:20:33.348 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.648510+0000', 'last_change': '2024-04-15T15:20:28.648510+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.648510+0000', 'last_undegraded': '2024-04-15T15:20:28.648510+0000', 'last_fullsized': '2024-04-15T15:20:28.648510+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-15T15:20:33.348 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.9 is not active+clean 2024-04-15T15:20:33.348 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.654328+0000', 'last_change': '2024-04-15T15:20:28.654328+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.654328+0000', 'last_undegraded': '2024-04-15T15:20:28.654328+0000', 'last_fullsized': '2024-04-15T15:20:28.654328+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-15T15:20:33.348 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.6 is not active+clean 2024-04-15T15:20:33.348 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.648564+0000', 'last_change': '2024-04-15T15:20:28.648564+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.648564+0000', 'last_undegraded': '2024-04-15T15:20:28.648564+0000', 'last_fullsized': '2024-04-15T15:20:28.648564+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-15T15:20:33.348 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.7 is not active+clean 2024-04-15T15:20:33.348 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.653245+0000', 'last_change': '2024-04-15T15:20:28.653245+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.653245+0000', 'last_undegraded': '2024-04-15T15:20:28.653245+0000', 'last_fullsized': '2024-04-15T15:20:28.653245+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-15T15:20:33.349 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.5 is not active+clean 2024-04-15T15:20:33.349 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.653064+0000', 'last_change': '2024-04-15T15:20:28.653064+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.653064+0000', 'last_undegraded': '2024-04-15T15:20:28.653064+0000', 'last_fullsized': '2024-04-15T15:20:28.653064+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-15T15:20:33.349 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2 is not active+clean 2024-04-15T15:20:33.349 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.648049+0000', 'last_change': '2024-04-15T15:20:28.648049+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.648049+0000', 'last_undegraded': '2024-04-15T15:20:28.648049+0000', 'last_fullsized': '2024-04-15T15:20:28.648049+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-15T15:20:33.349 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3 is not active+clean 2024-04-15T15:20:33.349 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.652596+0000', 'last_change': '2024-04-15T15:20:28.652596+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.652596+0000', 'last_undegraded': '2024-04-15T15:20:28.652596+0000', 'last_fullsized': '2024-04-15T15:20:28.652596+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-15T15:20:33.349 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.0 is not active+clean 2024-04-15T15:20:33.349 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.653762+0000', 'last_change': '2024-04-15T15:20:28.653762+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.653762+0000', 'last_undegraded': '2024-04-15T15:20:28.653762+0000', 'last_fullsized': '2024-04-15T15:20:28.653762+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-15T15:20:33.349 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1 is not active+clean 2024-04-15T15:20:33.349 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.652530+0000', 'last_change': '2024-04-15T15:20:28.652530+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.652530+0000', 'last_undegraded': '2024-04-15T15:20:28.652530+0000', 'last_fullsized': '2024-04-15T15:20:28.652530+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-15T15:20:33.349 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.4 is not active+clean 2024-04-15T15:20:33.350 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-04-15T15:20:28.654753+0000', 'last_change': '2024-04-15T15:20:28.654753+0000', 'last_active': '2024-04-15T15:20:28.638297+0000', 'last_peered': '2024-04-15T15:20:28.638297+0000', 'last_clean': '2024-04-15T15:20:28.638297+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-15T15:20:28.654753+0000', 'last_undegraded': '2024-04-15T15:20:28.654753+0000', 'last_fullsized': '2024-04-15T15:20:28.654753+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'last_clean_scrub_stamp': '2024-04-15T15:20:28.638297+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-15T15:20:33.350 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-15T15:20:33.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.473+0000 7f389a152640 1 Processor -- start 2024-04-15T15:20:33.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.473+0000 7f389a152640 1 -- start start 2024-04-15T15:20:33.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.474+0000 7f389a152640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f38940b78f0 0x7f38940d11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:33.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.474+0000 7f389a152640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f38940d16e0 0x7f38940d3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:33.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.474+0000 7f389a152640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38940d1b30 con 0x7f389415eab0 2024-04-15T15:20:33.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.474+0000 7f389a152640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f389415ee90 con 0x7f38940d16e0 2024-04-15T15:20:33.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.474+0000 7f389a152640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f3894162ff0 con 0x7f38940b78f0 2024-04-15T15:20:33.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.474+0000 7f3893fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f38940d16e0 0x7f38940d3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:33.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.474+0000 7f3893fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f38940d16e0 0x7f38940d3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45306/0 (socket says 172.21.15.105:45306) 2024-04-15T15:20:33.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.474+0000 7f3893fff640 1 -- 172.21.15.105:0/2768344066 learned_addr learned my addr 172.21.15.105:0/2768344066 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:33.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.474+0000 7f388affd640 1 --2- 172.21.15.105:0/2768344066 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f38940b78f0 0x7f38940d11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:33.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.474+0000 7f3893fff640 1 -- 172.21.15.105:0/2768344066 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f38940b78f0 msgr2=0x7f38940d11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:33.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.475+0000 7f3893fff640 1 --2- 172.21.15.105:0/2768344066 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f38940b78f0 0x7f38940d11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:33.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.475+0000 7f3893fff640 1 -- 172.21.15.105:0/2768344066 >> v1:172.21.15.154:6789/0 conn(0x7f389415eab0 legacy=0x7f3894162530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:33.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.475+0000 7f3893fff640 1 -- 172.21.15.105:0/2768344066 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38941632d0 con 0x7f38940d16e0 2024-04-15T15:20:33.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.475+0000 7f3893fff640 1 --2- 172.21.15.105:0/2768344066 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38940d16e0 0x7f38940d3530 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f388c009e50 tx=0x7f388c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=191a854ed3e2513d server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:33.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.475+0000 7f3892ffd640 1 -- 172.21.15.105:0/2768344066 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388c02ca70 con 0x7f38940d16e0 2024-04-15T15:20:33.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.475+0000 7f3892ffd640 1 -- 172.21.15.105:0/2768344066 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f388c02cc10 con 0x7f38940d16e0 2024-04-15T15:20:33.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.475+0000 7f3892ffd640 1 -- 172.21.15.105:0/2768344066 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388c0343f0 con 0x7f38940d16e0 2024-04-15T15:20:33.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.476+0000 7f389a152640 1 -- 172.21.15.105:0/2768344066 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38940d16e0 msgr2=0x7f38940d3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:33.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.476+0000 7f389a152640 1 --2- 172.21.15.105:0/2768344066 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38940d16e0 0x7f38940d3530 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f388c009e50 tx=0x7f388c02bed0 comp rx=0 tx=0).stop 2024-04-15T15:20:33.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.476+0000 7f389a152640 1 -- 172.21.15.105:0/2768344066 shutdown_connections 2024-04-15T15:20:33.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.476+0000 7f389a152640 1 --2- 172.21.15.105:0/2768344066 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38940d16e0 0x7f38940d3530 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:33.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.476+0000 7f389a152640 1 --2- 172.21.15.105:0/2768344066 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f38940b78f0 0x7f38940d11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:33.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.476+0000 7f389a152640 1 -- 172.21.15.105:0/2768344066 >> 172.21.15.105:0/2768344066 conn(0x7f3894067720 msgr2=0x7f3894067b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:33.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.476+0000 7f389a152640 1 -- 172.21.15.105:0/2768344066 shutdown_connections 2024-04-15T15:20:33.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.476+0000 7f389a152640 1 -- 172.21.15.105:0/2768344066 wait complete. 2024-04-15T15:20:33.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.477+0000 7f389a152640 1 Processor -- start 2024-04-15T15:20:33.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.477+0000 7f389a152640 1 -- start start 2024-04-15T15:20:33.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.478+0000 7f389a152640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38940b78f0 0x7f38940cadc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:33.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.478+0000 7f389a152640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f38940d16e0 0x7f38941f5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:33.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.478+0000 7f389a152640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f389415eab0 0x7f38940d4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:33.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.478+0000 7f389a152640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3894162c40 con 0x7f38940b78f0 2024-04-15T15:20:33.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.478+0000 7f389a152640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f38940d3010 con 0x7f38940d16e0 2024-04-15T15:20:33.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.478+0000 7f389a152640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f38941f4910 con 0x7f389415eab0 2024-04-15T15:20:33.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.478+0000 7f38937fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38940b78f0 0x7f38940cadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:33.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.478+0000 7f38937fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38940b78f0 0x7f38940cadc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45308/0 (socket says 172.21.15.105:45308) 2024-04-15T15:20:33.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.478+0000 7f38937fe640 1 -- 172.21.15.105:0/3537766872 learned_addr learned my addr 172.21.15.105:0/3537766872 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:33.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.478+0000 7f38937fe640 1 -- 172.21.15.105:0/3537766872 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f389415eab0 msgr2=0x7f38940d4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:33.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.478+0000 7f38937fe640 1 --2- 172.21.15.105:0/3537766872 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f389415eab0 0x7f38940d4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:33.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.478+0000 7f38937fe640 1 -- 172.21.15.105:0/3537766872 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f38940d16e0 msgr2=0x7f38941f5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:33.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.478+0000 7f38937fe640 1 --2- 172.21.15.105:0/3537766872 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f38940d16e0 0x7f38941f5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:33.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.478+0000 7f38937fe640 1 -- 172.21.15.105:0/3537766872 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38941f4b80 con 0x7f38940b78f0 2024-04-15T15:20:33.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.479+0000 7f38937fe640 1 --2- 172.21.15.105:0/3537766872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38940b78f0 0x7f38940cadc0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f388400b3f0 tx=0x7f388400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:33.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.479+0000 7f3890ff9640 1 -- 172.21.15.105:0/3537766872 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3884015020 con 0x7f38940b78f0 2024-04-15T15:20:33.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.479+0000 7f3890ff9640 1 -- 172.21.15.105:0/3537766872 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3884011830 con 0x7f38940b78f0 2024-04-15T15:20:33.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.479+0000 7f389a152640 1 -- 172.21.15.105:0/3537766872 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38940c9150 con 0x7f38940b78f0 2024-04-15T15:20:33.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.479+0000 7f3890ff9640 1 -- 172.21.15.105:0/3537766872 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3884011c20 con 0x7f38940b78f0 2024-04-15T15:20:33.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.479+0000 7f389a152640 1 -- 172.21.15.105:0/3537766872 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38940c8b90 con 0x7f38940b78f0 2024-04-15T15:20:33.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.480+0000 7f389a152640 1 -- 172.21.15.105:0/3537766872 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38941f4d50 con 0x7f38940b78f0 2024-04-15T15:20:33.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.480+0000 7f3890ff9640 1 -- 172.21.15.105:0/3537766872 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3884018020 con 0x7f38940b78f0 2024-04-15T15:20:33.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.481+0000 7f3890ff9640 1 --2- 172.21.15.105:0/3537766872 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f387406c8f0 0x7f387406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:33.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.481+0000 7f3890ff9640 1 -- 172.21.15.105:0/3537766872 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f388408b6f0 con 0x7f38940b78f0 2024-04-15T15:20:33.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.481+0000 7f388affd640 1 --2- 172.21.15.105:0/3537766872 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f387406c8f0 0x7f387406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:33.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.481+0000 7f388affd640 1 --2- 172.21.15.105:0/3537766872 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f387406c8f0 0x7f387406edb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f38940cb9b0 tx=0x7f387c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:33.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.485+0000 7f3890ff9640 1 -- 172.21.15.105:0/3537766872 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f388401f030 con 0x7f38940b78f0 2024-04-15T15:20:33.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.731+0000 7f389a152640 1 -- 172.21.15.105:0/3537766872 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f3894068a40 con 0x7f38940b78f0 2024-04-15T15:20:33.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.732+0000 7f3890ff9640 1 -- 172.21.15.105:0/3537766872 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1581 (secure 0 0 0) 0x7f38840172c0 con 0x7f38940b78f0 2024-04-15T15:20:33.734 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:20:33.734 INFO:teuthology.orchestra.run.smithi105.stdout:{"fsid":"09e4d2a4-a324-4793-af77-644b25e520e1","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":92,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":34,"num_osds":8,"num_up_osds":8,"osd_up_since":1713194352,"num_in_osds":8,"osd_in_since":1713194343,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":225337344,"bytes_avail":772868775936,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":14,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"a","status":"up:active","gid":5196}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-04-15T15:20:15.743377+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-15T15:20:33.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.734+0000 7f389a152640 1 -- 172.21.15.105:0/3537766872 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f387406c8f0 msgr2=0x7f387406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:33.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.734+0000 7f389a152640 1 --2- 172.21.15.105:0/3537766872 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f387406c8f0 0x7f387406edb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f38940cb9b0 tx=0x7f387c008040 comp rx=0 tx=0).stop 2024-04-15T15:20:33.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.734+0000 7f389a152640 1 -- 172.21.15.105:0/3537766872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38940b78f0 msgr2=0x7f38940cadc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:33.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.734+0000 7f389a152640 1 --2- 172.21.15.105:0/3537766872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38940b78f0 0x7f38940cadc0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f388400b3f0 tx=0x7f388400b8b0 comp rx=0 tx=0).stop 2024-04-15T15:20:33.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.734+0000 7f389a152640 1 -- 172.21.15.105:0/3537766872 shutdown_connections 2024-04-15T15:20:33.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.734+0000 7f389a152640 1 --2- 172.21.15.105:0/3537766872 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f389415eab0 0x7f38940d4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:33.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.734+0000 7f389a152640 1 --2- 172.21.15.105:0/3537766872 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f387406c8f0 0x7f387406edb0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:33.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.734+0000 7f389a152640 1 --2- 172.21.15.105:0/3537766872 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f38940d16e0 0x7f38941f5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:33.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.734+0000 7f389a152640 1 --2- 172.21.15.105:0/3537766872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38940b78f0 0x7f38940cadc0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:33.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.734+0000 7f389a152640 1 -- 172.21.15.105:0/3537766872 >> 172.21.15.105:0/3537766872 conn(0x7f3894067720 msgr2=0x7f3894159e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:33.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.734+0000 7f389a152640 1 -- 172.21.15.105:0/3537766872 shutdown_connections 2024-04-15T15:20:33.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.734+0000 7f389a152640 1 -- 172.21.15.105:0/3537766872 wait complete. 2024-04-15T15:20:33.747 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-15T15:20:33.747 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-15T15:20:33.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.849+0000 7f54c579d640 1 Processor -- start 2024-04-15T15:20:33.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.849+0000 7f54c579d640 1 -- start start 2024-04-15T15:20:33.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.850+0000 7f54c579d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f54c0144e70 0x7f54c00b27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:33.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.850+0000 7f54c579d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f54c00b4560 0x7f54c006b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:33.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.850+0000 7f54c579d640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54c00b5ef0 con 0x7f54c006bf00 2024-04-15T15:20:33.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.850+0000 7f54c579d640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f54c0145250 con 0x7f54c0144e70 2024-04-15T15:20:33.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.850+0000 7f54c579d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f54c00b61d0 con 0x7f54c00b4560 2024-04-15T15:20:33.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.850+0000 7f54bffff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f54c0144e70 0x7f54c00b27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:33.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.850+0000 7f54bffff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f54c0144e70 0x7f54c00b27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45314/0 (socket says 172.21.15.105:45314) 2024-04-15T15:20:33.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.850+0000 7f54c4f9c640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f54c006bf00 0x7f54c0162420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:43714/0 (socket says 172.21.15.105:43714) 2024-04-15T15:20:33.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.850+0000 7f54bffff640 1 -- 172.21.15.105:0/2710419501 learned_addr learned my addr 172.21.15.105:0/2710419501 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:33.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.850+0000 7f54bffff640 1 -- 172.21.15.105:0/2710419501 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f54c00b4560 msgr2=0x7f54c006b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:33.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.850+0000 7f54bffff640 1 --2- 172.21.15.105:0/2710419501 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f54c00b4560 0x7f54c006b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:33.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.850+0000 7f54bffff640 1 -- 172.21.15.105:0/2710419501 >> v1:172.21.15.105:6789/0 conn(0x7f54c006bf00 legacy=0x7f54c0162420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:33.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.851+0000 7f54bffff640 1 -- 172.21.15.105:0/2710419501 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54c00b64b0 con 0x7f54c0144e70 2024-04-15T15:20:33.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.851+0000 7f54bffff640 1 --2- 172.21.15.105:0/2710419501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54c0144e70 0x7f54c00b27c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f54b0009870 tx=0x7f54b002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=424ee9c88c727812 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:33.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.851+0000 7f54beffd640 1 -- 172.21.15.105:0/2710419501 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b0034070 con 0x7f54c0144e70 2024-04-15T15:20:33.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.851+0000 7f54beffd640 1 -- 172.21.15.105:0/2710419501 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54b0030840 con 0x7f54c0144e70 2024-04-15T15:20:33.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.851+0000 7f54beffd640 1 -- 172.21.15.105:0/2710419501 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b0030c10 con 0x7f54c0144e70 2024-04-15T15:20:33.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.852+0000 7f54c579d640 1 -- 172.21.15.105:0/2710419501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54c0144e70 msgr2=0x7f54c00b27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:33.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.852+0000 7f54c579d640 1 --2- 172.21.15.105:0/2710419501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54c0144e70 0x7f54c00b27c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f54b0009870 tx=0x7f54b002b980 comp rx=0 tx=0).stop 2024-04-15T15:20:33.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.852+0000 7f54c579d640 1 -- 172.21.15.105:0/2710419501 shutdown_connections 2024-04-15T15:20:33.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.852+0000 7f54c579d640 1 --2- 172.21.15.105:0/2710419501 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f54c00b4560 0x7f54c006b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:33.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.852+0000 7f54c579d640 1 --2- 172.21.15.105:0/2710419501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54c0144e70 0x7f54c00b27c0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:33.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.852+0000 7f54c579d640 1 -- 172.21.15.105:0/2710419501 >> 172.21.15.105:0/2710419501 conn(0x7f54c00b67e0 msgr2=0x7f54c00b6bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:33.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.852+0000 7f54c579d640 1 -- 172.21.15.105:0/2710419501 shutdown_connections 2024-04-15T15:20:33.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.852+0000 7f54c579d640 1 -- 172.21.15.105:0/2710419501 wait complete. 2024-04-15T15:20:33.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.853+0000 7f54c579d640 1 Processor -- start 2024-04-15T15:20:33.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.853+0000 7f54c579d640 1 -- start start 2024-04-15T15:20:33.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.854+0000 7f54c579d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54c006bf00 0x7f54c01f4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:33.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.854+0000 7f54c579d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f54c00b4560 0x7f54c0142d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:33.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.854+0000 7f54c579d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f54c0144e70 0x7f54c0139800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:33.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.854+0000 7f54c579d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f54c00b2d00 con 0x7f54c006bf00 2024-04-15T15:20:33.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.854+0000 7f54c579d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f54c01457e0 con 0x7f54c00b4560 2024-04-15T15:20:33.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.854+0000 7f54c579d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f54c0146d10 con 0x7f54c0144e70 2024-04-15T15:20:33.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.854+0000 7f54bffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54c006bf00 0x7f54c01f4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:33.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.854+0000 7f54bffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54c006bf00 0x7f54c01f4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45328/0 (socket says 172.21.15.105:45328) 2024-04-15T15:20:33.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.854+0000 7f54bffff640 1 -- 172.21.15.105:0/3824137881 learned_addr learned my addr 172.21.15.105:0/3824137881 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:33.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.854+0000 7f54bffff640 1 -- 172.21.15.105:0/3824137881 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f54c0144e70 msgr2=0x7f54c0139800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:33.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.854+0000 7f54bffff640 1 --2- 172.21.15.105:0/3824137881 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f54c0144e70 0x7f54c0139800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:33.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.854+0000 7f54bffff640 1 -- 172.21.15.105:0/3824137881 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f54c00b4560 msgr2=0x7f54c0142d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:33.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.854+0000 7f54bffff640 1 --2- 172.21.15.105:0/3824137881 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f54c00b4560 0x7f54c0142d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:33.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.854+0000 7f54bffff640 1 -- 172.21.15.105:0/3824137881 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54c015f840 con 0x7f54c006bf00 2024-04-15T15:20:33.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.855+0000 7f54bffff640 1 --2- 172.21.15.105:0/3824137881 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54c006bf00 0x7f54c01f4a00 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f54b0000c00 tx=0x7f54b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:33.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.855+0000 7f54bcff9640 1 -- 172.21.15.105:0/3824137881 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b0034040 con 0x7f54c006bf00 2024-04-15T15:20:33.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.855+0000 7f54bcff9640 1 -- 172.21.15.105:0/3824137881 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54b0039d90 con 0x7f54c006bf00 2024-04-15T15:20:33.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.855+0000 7f54c579d640 1 -- 172.21.15.105:0/3824137881 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54c015fb20 con 0x7f54c006bf00 2024-04-15T15:20:33.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.855+0000 7f54bcff9640 1 -- 172.21.15.105:0/3824137881 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b0043610 con 0x7f54c006bf00 2024-04-15T15:20:33.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.855+0000 7f54c579d640 1 -- 172.21.15.105:0/3824137881 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54c01600e0 con 0x7f54c006bf00 2024-04-15T15:20:33.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.856+0000 7f54c579d640 1 -- 172.21.15.105:0/3824137881 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54c0068a40 con 0x7f54c006bf00 2024-04-15T15:20:33.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.856+0000 7f54bcff9640 1 -- 172.21.15.105:0/3824137881 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f54b004a060 con 0x7f54c006bf00 2024-04-15T15:20:33.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.857+0000 7f54bcff9640 1 --2- 172.21.15.105:0/3824137881 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f54ac06c8f0 0x7f54ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:33.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.857+0000 7f54bcff9640 1 -- 172.21.15.105:0/3824137881 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f54b00add70 con 0x7f54c006bf00 2024-04-15T15:20:33.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.857+0000 7f54bf7fe640 1 --2- 172.21.15.105:0/3824137881 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f54ac06c8f0 0x7f54ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:33.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.857+0000 7f54bf7fe640 1 --2- 172.21.15.105:0/3824137881 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f54ac06c8f0 0x7f54ac06edb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f54c015df50 tx=0x7f54b4006c90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:33.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:33.861+0000 7f54bcff9640 1 -- 172.21.15.105:0/3824137881 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f54b003e070 con 0x7f54c006bf00 2024-04-15T15:20:34.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:34.032+0000 7f54c579d640 1 -- 172.21.15.105:0/3824137881 --> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f54c0144490 con 0x7f54ac06c8f0 2024-04-15T15:20:34.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:34.038+0000 7f54bcff9640 1 -- 172.21.15.105:0/3824137881 <== mgr.4107 v2:172.21.15.105:6832/3506549322 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246632 (secure 0 0 0) 0x7f54c0144490 con 0x7f54ac06c8f0 2024-04-15T15:20:34.042 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:20:34.043 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-04-15T15:20:34.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:34.040+0000 7f54c579d640 1 -- 172.21.15.105:0/3824137881 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f54ac06c8f0 msgr2=0x7f54ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:34.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:34.040+0000 7f54c579d640 1 --2- 172.21.15.105:0/3824137881 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f54ac06c8f0 0x7f54ac06edb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f54c015df50 tx=0x7f54b4006c90 comp rx=0 tx=0).stop 2024-04-15T15:20:34.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:34.040+0000 7f54c579d640 1 -- 172.21.15.105:0/3824137881 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54c006bf00 msgr2=0x7f54c01f4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:34.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:34.040+0000 7f54c579d640 1 --2- 172.21.15.105:0/3824137881 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54c006bf00 0x7f54c01f4a00 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f54b0000c00 tx=0x7f54b00046c0 comp rx=0 tx=0).stop 2024-04-15T15:20:34.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:34.040+0000 7f54c579d640 1 -- 172.21.15.105:0/3824137881 shutdown_connections 2024-04-15T15:20:34.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:34.040+0000 7f54c579d640 1 --2- 172.21.15.105:0/3824137881 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f54c0144e70 0x7f54c0139800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:34.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:34.040+0000 7f54c579d640 1 --2- 172.21.15.105:0/3824137881 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f54ac06c8f0 0x7f54ac06edb0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:34.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:34.040+0000 7f54c579d640 1 --2- 172.21.15.105:0/3824137881 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f54c00b4560 0x7f54c0142d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:34.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:34.040+0000 7f54c579d640 1 --2- 172.21.15.105:0/3824137881 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54c006bf00 0x7f54c01f4a00 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:34.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:34.040+0000 7f54c579d640 1 -- 172.21.15.105:0/3824137881 >> 172.21.15.105:0/3824137881 conn(0x7f54c00b67e0 msgr2=0x7f54c006fa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:34.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:34.041+0000 7f54c579d640 1 -- 172.21.15.105:0/3824137881 shutdown_connections 2024-04-15T15:20:34.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:34.041+0000 7f54c579d640 1 -- 172.21.15.105:0/3824137881 wait complete. 2024-04-15T15:20:34.053 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":67,"stamp":"2024-04-15T15:20:33.746207+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220056,"kb_used_data":3892,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754664,"statfs":{"total":773094113280,"available":772868775936,"internally_reserved":0,"allocated":3985408,"data_stored":2247651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000864"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.648653+0000","last_change":"2024-04-15T15:20:28.648653+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.648653+0000","last_undegraded":"2024-04-15T15:20:28.648653+0000","last_fullsized":"2024-04-15T15:20:28.648653+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.657541+0000","last_change":"2024-04-15T15:20:28.657541+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.657541+0000","last_undegraded":"2024-04-15T15:20:28.657541+0000","last_fullsized":"2024-04-15T15:20:28.657541+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.654113+0000","last_change":"2024-04-15T15:20:28.654113+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.654113+0000","last_undegraded":"2024-04-15T15:20:28.654113+0000","last_fullsized":"2024-04-15T15:20:28.654113+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.656529+0000","last_change":"2024-04-15T15:20:28.656529+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.656529+0000","last_undegraded":"2024-04-15T15:20:28.656529+0000","last_fullsized":"2024-04-15T15:20:28.656529+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.649435+0000","last_change":"2024-04-15T15:20:28.649435+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.649435+0000","last_undegraded":"2024-04-15T15:20:28.649435+0000","last_fullsized":"2024-04-15T15:20:28.649435+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.656782+0000","last_change":"2024-04-15T15:20:28.656782+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.656782+0000","last_undegraded":"2024-04-15T15:20:28.656782+0000","last_fullsized":"2024-04-15T15:20:28.656782+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.653845+0000","last_change":"2024-04-15T15:20:28.653845+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.653845+0000","last_undegraded":"2024-04-15T15:20:28.653845+0000","last_fullsized":"2024-04-15T15:20:28.653845+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.655853+0000","last_change":"2024-04-15T15:20:28.655853+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.655853+0000","last_undegraded":"2024-04-15T15:20:28.655853+0000","last_fullsized":"2024-04-15T15:20:28.655853+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.650952+0000","last_change":"2024-04-15T15:20:28.650952+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.650952+0000","last_undegraded":"2024-04-15T15:20:28.650952+0000","last_fullsized":"2024-04-15T15:20:28.650952+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.656338+0000","last_change":"2024-04-15T15:20:28.656338+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.656338+0000","last_undegraded":"2024-04-15T15:20:28.656338+0000","last_fullsized":"2024-04-15T15:20:28.656338+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.656460+0000","last_change":"2024-04-15T15:20:28.656460+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.656460+0000","last_undegraded":"2024-04-15T15:20:28.656460+0000","last_fullsized":"2024-04-15T15:20:28.656460+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.654814+0000","last_change":"2024-04-15T15:20:28.654814+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.654814+0000","last_undegraded":"2024-04-15T15:20:28.654814+0000","last_fullsized":"2024-04-15T15:20:28.654814+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.648869+0000","last_change":"2024-04-15T15:20:28.648869+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.648869+0000","last_undegraded":"2024-04-15T15:20:28.648869+0000","last_fullsized":"2024-04-15T15:20:28.648869+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.656407+0000","last_change":"2024-04-15T15:20:28.656407+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.656407+0000","last_undegraded":"2024-04-15T15:20:28.656407+0000","last_fullsized":"2024-04-15T15:20:28.656407+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.654309+0000","last_change":"2024-04-15T15:20:28.654309+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.654309+0000","last_undegraded":"2024-04-15T15:20:28.654309+0000","last_fullsized":"2024-04-15T15:20:28.654309+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.653768+0000","last_change":"2024-04-15T15:20:28.653768+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.653768+0000","last_undegraded":"2024-04-15T15:20:28.653768+0000","last_fullsized":"2024-04-15T15:20:28.653768+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.656765+0000","last_change":"2024-04-15T15:20:28.656765+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.656765+0000","last_undegraded":"2024-04-15T15:20:28.656765+0000","last_fullsized":"2024-04-15T15:20:28.656765+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.655906+0000","last_change":"2024-04-15T15:20:28.655906+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.655906+0000","last_undegraded":"2024-04-15T15:20:28.655906+0000","last_fullsized":"2024-04-15T15:20:28.655906+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.655380+0000","last_change":"2024-04-15T15:20:28.655380+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.655380+0000","last_undegraded":"2024-04-15T15:20:28.655380+0000","last_fullsized":"2024-04-15T15:20:28.655380+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.648667+0000","last_change":"2024-04-15T15:20:28.648667+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.648667+0000","last_undegraded":"2024-04-15T15:20:28.648667+0000","last_fullsized":"2024-04-15T15:20:28.648667+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.652939+0000","last_change":"2024-04-15T15:20:28.652939+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.652939+0000","last_undegraded":"2024-04-15T15:20:28.652939+0000","last_fullsized":"2024-04-15T15:20:28.652939+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.652347+0000","last_change":"2024-04-15T15:20:28.652347+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.652347+0000","last_undegraded":"2024-04-15T15:20:28.652347+0000","last_fullsized":"2024-04-15T15:20:28.652347+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.653711+0000","last_change":"2024-04-15T15:20:28.653711+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.653711+0000","last_undegraded":"2024-04-15T15:20:28.653711+0000","last_fullsized":"2024-04-15T15:20:28.653711+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.654389+0000","last_change":"2024-04-15T15:20:28.654389+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.654389+0000","last_undegraded":"2024-04-15T15:20:28.654389+0000","last_fullsized":"2024-04-15T15:20:28.654389+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.656880+0000","last_change":"2024-04-15T15:20:28.656880+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.656880+0000","last_undegraded":"2024-04-15T15:20:28.656880+0000","last_fullsized":"2024-04-15T15:20:28.656880+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.654703+0000","last_change":"2024-04-15T15:20:28.654703+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.654703+0000","last_undegraded":"2024-04-15T15:20:28.654703+0000","last_fullsized":"2024-04-15T15:20:28.654703+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.656635+0000","last_change":"2024-04-15T15:20:28.656635+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.656635+0000","last_undegraded":"2024-04-15T15:20:28.656635+0000","last_fullsized":"2024-04-15T15:20:28.656635+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.654372+0000","last_change":"2024-04-15T15:20:28.654372+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.654372+0000","last_undegraded":"2024-04-15T15:20:28.654372+0000","last_fullsized":"2024-04-15T15:20:28.654372+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.652231+0000","last_change":"2024-04-15T15:20:28.652231+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.652231+0000","last_undegraded":"2024-04-15T15:20:28.652231+0000","last_fullsized":"2024-04-15T15:20:28.652231+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.655536+0000","last_change":"2024-04-15T15:20:28.655536+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.655536+0000","last_undegraded":"2024-04-15T15:20:28.655536+0000","last_fullsized":"2024-04-15T15:20:28.655536+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.648479+0000","last_change":"2024-04-15T15:20:28.648479+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.648479+0000","last_undegraded":"2024-04-15T15:20:28.648479+0000","last_fullsized":"2024-04-15T15:20:28.648479+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.655450+0000","last_change":"2024-04-15T15:20:28.655450+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.655450+0000","last_undegraded":"2024-04-15T15:20:28.655450+0000","last_fullsized":"2024-04-15T15:20:28.655450+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.653482+0000","last_change":"2024-04-15T15:20:28.653482+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.653482+0000","last_undegraded":"2024-04-15T15:20:28.653482+0000","last_fullsized":"2024-04-15T15:20:28.653482+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.664265+0000","last_change":"2024-04-15T15:20:28.659981+0000","last_active":"2024-04-15T15:20:28.664265+0000","last_peered":"2024-04-15T15:20:28.664265+0000","last_clean":"2024-04-15T15:20:28.664265+0000","last_became_active":"2024-04-15T15:20:28.657535+0000","last_became_peered":"2024-04-15T15:20:28.657535+0000","last_unstale":"2024-04-15T15:20:28.664265+0000","last_undegraded":"2024-04-15T15:20:28.664265+0000","last_fullsized":"2024-04-15T15:20:28.664265+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:28:40.356127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.648158+0000","last_change":"2024-04-15T15:20:28.648158+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.648158+0000","last_undegraded":"2024-04-15T15:20:28.648158+0000","last_fullsized":"2024-04-15T15:20:28.648158+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.665310+0000","last_change":"2024-04-15T15:20:28.658521+0000","last_active":"2024-04-15T15:20:28.665310+0000","last_peered":"2024-04-15T15:20:28.665310+0000","last_clean":"2024-04-15T15:20:28.665310+0000","last_became_active":"2024-04-15T15:20:28.658266+0000","last_became_peered":"2024-04-15T15:20:28.658266+0000","last_unstale":"2024-04-15T15:20:28.665310+0000","last_undegraded":"2024-04-15T15:20:28.665310+0000","last_fullsized":"2024-04-15T15:20:28.665310+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:11:11.096685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.652058+0000","last_change":"2024-04-15T15:20:28.652058+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.652058+0000","last_undegraded":"2024-04-15T15:20:28.652058+0000","last_fullsized":"2024-04-15T15:20:28.652058+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.667305+0000","last_change":"2024-04-15T15:20:28.664326+0000","last_active":"2024-04-15T15:20:28.667305+0000","last_peered":"2024-04-15T15:20:28.667305+0000","last_clean":"2024-04-15T15:20:28.667305+0000","last_became_active":"2024-04-15T15:20:28.664077+0000","last_became_peered":"2024-04-15T15:20:28.664077+0000","last_unstale":"2024-04-15T15:20:28.667305+0000","last_undegraded":"2024-04-15T15:20:28.667305+0000","last_fullsized":"2024-04-15T15:20:28.667305+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:53:08.913774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.656218+0000","last_change":"2024-04-15T15:20:28.656218+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.656218+0000","last_undegraded":"2024-04-15T15:20:28.656218+0000","last_fullsized":"2024-04-15T15:20:28.656218+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.661574+0000","last_change":"2024-04-15T15:20:28.659705+0000","last_active":"2024-04-15T15:20:28.661574+0000","last_peered":"2024-04-15T15:20:28.661574+0000","last_clean":"2024-04-15T15:20:28.661574+0000","last_became_active":"2024-04-15T15:20:28.659433+0000","last_became_peered":"2024-04-15T15:20:28.659433+0000","last_unstale":"2024-04-15T15:20:28.661574+0000","last_undegraded":"2024-04-15T15:20:28.661574+0000","last_fullsized":"2024-04-15T15:20:28.661574+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:08:06.553533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.655155+0000","last_change":"2024-04-15T15:20:28.655155+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.655155+0000","last_undegraded":"2024-04-15T15:20:28.655155+0000","last_fullsized":"2024-04-15T15:20:28.655155+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.666747+0000","last_change":"2024-04-15T15:20:28.662442+0000","last_active":"2024-04-15T15:20:28.666747+0000","last_peered":"2024-04-15T15:20:28.666747+0000","last_clean":"2024-04-15T15:20:28.666747+0000","last_became_active":"2024-04-15T15:20:28.661241+0000","last_became_peered":"2024-04-15T15:20:28.661241+0000","last_unstale":"2024-04-15T15:20:28.666747+0000","last_undegraded":"2024-04-15T15:20:28.666747+0000","last_fullsized":"2024-04-15T15:20:28.666747+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:06:50.496578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.653975+0000","last_change":"2024-04-15T15:20:28.653975+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.653975+0000","last_undegraded":"2024-04-15T15:20:28.653975+0000","last_fullsized":"2024-04-15T15:20:28.653975+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.666634+0000","last_change":"2024-04-15T15:20:28.662045+0000","last_active":"2024-04-15T15:20:28.666634+0000","last_peered":"2024-04-15T15:20:28.666634+0000","last_clean":"2024-04-15T15:20:28.666634+0000","last_became_active":"2024-04-15T15:20:28.661769+0000","last_became_peered":"2024-04-15T15:20:28.661769+0000","last_unstale":"2024-04-15T15:20:28.666634+0000","last_undegraded":"2024-04-15T15:20:28.666634+0000","last_fullsized":"2024-04-15T15:20:28.666634+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:50:03.424628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.651506+0000","last_change":"2024-04-15T15:20:28.651506+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.651506+0000","last_undegraded":"2024-04-15T15:20:28.651506+0000","last_fullsized":"2024-04-15T15:20:28.651506+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.667091+0000","last_change":"2024-04-15T15:20:28.661631+0000","last_active":"2024-04-15T15:20:28.667091+0000","last_peered":"2024-04-15T15:20:28.667091+0000","last_clean":"2024-04-15T15:20:28.667091+0000","last_became_active":"2024-04-15T15:20:28.661392+0000","last_became_peered":"2024-04-15T15:20:28.661392+0000","last_unstale":"2024-04-15T15:20:28.667091+0000","last_undegraded":"2024-04-15T15:20:28.667091+0000","last_fullsized":"2024-04-15T15:20:28.667091+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:32:32.932055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.655695+0000","last_change":"2024-04-15T15:20:28.655695+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.655695+0000","last_undegraded":"2024-04-15T15:20:28.655695+0000","last_fullsized":"2024-04-15T15:20:28.655695+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.666575+0000","last_change":"2024-04-15T15:20:28.661675+0000","last_active":"2024-04-15T15:20:28.666575+0000","last_peered":"2024-04-15T15:20:28.666575+0000","last_clean":"2024-04-15T15:20:28.666575+0000","last_became_active":"2024-04-15T15:20:28.661368+0000","last_became_peered":"2024-04-15T15:20:28.661368+0000","last_unstale":"2024-04-15T15:20:28.666575+0000","last_undegraded":"2024-04-15T15:20:28.666575+0000","last_fullsized":"2024-04-15T15:20:28.666575+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:48:09.613555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.653028+0000","last_change":"2024-04-15T15:20:28.653028+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.653028+0000","last_undegraded":"2024-04-15T15:20:28.653028+0000","last_fullsized":"2024-04-15T15:20:28.653028+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.669471+0000","last_change":"2024-04-15T15:20:28.665030+0000","last_active":"2024-04-15T15:20:28.669471+0000","last_peered":"2024-04-15T15:20:28.669471+0000","last_clean":"2024-04-15T15:20:28.669471+0000","last_became_active":"2024-04-15T15:20:28.664719+0000","last_became_peered":"2024-04-15T15:20:28.664719+0000","last_unstale":"2024-04-15T15:20:28.669471+0000","last_undegraded":"2024-04-15T15:20:28.669471+0000","last_fullsized":"2024-04-15T15:20:28.669471+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:53:47.542553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.654274+0000","last_change":"2024-04-15T15:20:28.654274+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.654274+0000","last_undegraded":"2024-04-15T15:20:28.654274+0000","last_fullsized":"2024-04-15T15:20:28.654274+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.669518+0000","last_change":"2024-04-15T15:20:28.662386+0000","last_active":"2024-04-15T15:20:28.669518+0000","last_peered":"2024-04-15T15:20:28.669518+0000","last_clean":"2024-04-15T15:20:28.669518+0000","last_became_active":"2024-04-15T15:20:28.662151+0000","last_became_peered":"2024-04-15T15:20:28.662151+0000","last_unstale":"2024-04-15T15:20:28.669518+0000","last_undegraded":"2024-04-15T15:20:28.669518+0000","last_fullsized":"2024-04-15T15:20:28.669518+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T03:16:13.044979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.654303+0000","last_change":"2024-04-15T15:20:28.654303+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.654303+0000","last_undegraded":"2024-04-15T15:20:28.654303+0000","last_fullsized":"2024-04-15T15:20:28.654303+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.664277+0000","last_change":"2024-04-15T15:20:28.662968+0000","last_active":"2024-04-15T15:20:28.664277+0000","last_peered":"2024-04-15T15:20:28.664277+0000","last_clean":"2024-04-15T15:20:28.664277+0000","last_became_active":"2024-04-15T15:20:28.662552+0000","last_became_peered":"2024-04-15T15:20:28.662552+0000","last_unstale":"2024-04-15T15:20:28.664277+0000","last_undegraded":"2024-04-15T15:20:28.664277+0000","last_fullsized":"2024-04-15T15:20:28.664277+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:47:14.250326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.652685+0000","last_change":"2024-04-15T15:20:28.652685+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.652685+0000","last_undegraded":"2024-04-15T15:20:28.652685+0000","last_fullsized":"2024-04-15T15:20:28.652685+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.667302+0000","last_change":"2024-04-15T15:20:28.663559+0000","last_active":"2024-04-15T15:20:28.667302+0000","last_peered":"2024-04-15T15:20:28.667302+0000","last_clean":"2024-04-15T15:20:28.667302+0000","last_became_active":"2024-04-15T15:20:28.662946+0000","last_became_peered":"2024-04-15T15:20:28.662946+0000","last_unstale":"2024-04-15T15:20:28.667302+0000","last_undegraded":"2024-04-15T15:20:28.667302+0000","last_fullsized":"2024-04-15T15:20:28.667302+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:16:18.672475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.655605+0000","last_change":"2024-04-15T15:20:28.655605+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.655605+0000","last_undegraded":"2024-04-15T15:20:28.655605+0000","last_fullsized":"2024-04-15T15:20:28.655605+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.666725+0000","last_change":"2024-04-15T15:20:28.662309+0000","last_active":"2024-04-15T15:20:28.666725+0000","last_peered":"2024-04-15T15:20:28.666725+0000","last_clean":"2024-04-15T15:20:28.666725+0000","last_became_active":"2024-04-15T15:20:28.661434+0000","last_became_peered":"2024-04-15T15:20:28.661434+0000","last_unstale":"2024-04-15T15:20:28.666725+0000","last_undegraded":"2024-04-15T15:20:28.666725+0000","last_fullsized":"2024-04-15T15:20:28.666725+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:51:31.432162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.653771+0000","last_change":"2024-04-15T15:20:28.653771+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.653771+0000","last_undegraded":"2024-04-15T15:20:28.653771+0000","last_fullsized":"2024-04-15T15:20:28.653771+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.669589+0000","last_change":"2024-04-15T15:20:28.665508+0000","last_active":"2024-04-15T15:20:28.669589+0000","last_peered":"2024-04-15T15:20:28.669589+0000","last_clean":"2024-04-15T15:20:28.669589+0000","last_became_active":"2024-04-15T15:20:28.665235+0000","last_became_peered":"2024-04-15T15:20:28.665235+0000","last_unstale":"2024-04-15T15:20:28.669589+0000","last_undegraded":"2024-04-15T15:20:28.669589+0000","last_fullsized":"2024-04-15T15:20:28.669589+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T03:16:13.044979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.654876+0000","last_change":"2024-04-15T15:20:28.654876+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.654876+0000","last_undegraded":"2024-04-15T15:20:28.654876+0000","last_fullsized":"2024-04-15T15:20:28.654876+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.667342+0000","last_change":"2024-04-15T15:20:28.665050+0000","last_active":"2024-04-15T15:20:28.667342+0000","last_peered":"2024-04-15T15:20:28.667342+0000","last_clean":"2024-04-15T15:20:28.667342+0000","last_became_active":"2024-04-15T15:20:28.664787+0000","last_became_peered":"2024-04-15T15:20:28.664787+0000","last_unstale":"2024-04-15T15:20:28.667342+0000","last_undegraded":"2024-04-15T15:20:28.667342+0000","last_fullsized":"2024-04-15T15:20:28.667342+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:10:52.680906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.647786+0000","last_change":"2024-04-15T15:20:28.647786+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.647786+0000","last_undegraded":"2024-04-15T15:20:28.647786+0000","last_fullsized":"2024-04-15T15:20:28.647786+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.669504+0000","last_change":"2024-04-15T15:20:28.665337+0000","last_active":"2024-04-15T15:20:28.669504+0000","last_peered":"2024-04-15T15:20:28.669504+0000","last_clean":"2024-04-15T15:20:28.669504+0000","last_became_active":"2024-04-15T15:20:28.664441+0000","last_became_peered":"2024-04-15T15:20:28.664441+0000","last_unstale":"2024-04-15T15:20:28.669504+0000","last_undegraded":"2024-04-15T15:20:28.669504+0000","last_fullsized":"2024-04-15T15:20:28.669504+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:40:45.758256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.651475+0000","last_change":"2024-04-15T15:20:28.651475+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.651475+0000","last_undegraded":"2024-04-15T15:20:28.651475+0000","last_fullsized":"2024-04-15T15:20:28.651475+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.669608+0000","last_change":"2024-04-15T15:20:28.666179+0000","last_active":"2024-04-15T15:20:28.669608+0000","last_peered":"2024-04-15T15:20:28.669608+0000","last_clean":"2024-04-15T15:20:28.669608+0000","last_became_active":"2024-04-15T15:20:28.665927+0000","last_became_peered":"2024-04-15T15:20:28.665927+0000","last_unstale":"2024-04-15T15:20:28.669608+0000","last_undegraded":"2024-04-15T15:20:28.669608+0000","last_fullsized":"2024-04-15T15:20:28.669608+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:54:49.252832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.653694+0000","last_change":"2024-04-15T15:20:28.653694+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.653694+0000","last_undegraded":"2024-04-15T15:20:28.653694+0000","last_fullsized":"2024-04-15T15:20:28.653694+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.666642+0000","last_change":"2024-04-15T15:20:28.661761+0000","last_active":"2024-04-15T15:20:28.666642+0000","last_peered":"2024-04-15T15:20:28.666642+0000","last_clean":"2024-04-15T15:20:28.666642+0000","last_became_active":"2024-04-15T15:20:28.661249+0000","last_became_peered":"2024-04-15T15:20:28.661249+0000","last_unstale":"2024-04-15T15:20:28.666642+0000","last_undegraded":"2024-04-15T15:20:28.666642+0000","last_fullsized":"2024-04-15T15:20:28.666642+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:40:31.925030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.655279+0000","last_change":"2024-04-15T15:20:28.655279+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.655279+0000","last_undegraded":"2024-04-15T15:20:28.655279+0000","last_fullsized":"2024-04-15T15:20:28.655279+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.660927+0000","last_change":"2024-04-15T15:20:28.657992+0000","last_active":"2024-04-15T15:20:28.660927+0000","last_peered":"2024-04-15T15:20:28.660927+0000","last_clean":"2024-04-15T15:20:28.660927+0000","last_became_active":"2024-04-15T15:20:28.657669+0000","last_became_peered":"2024-04-15T15:20:28.657669+0000","last_unstale":"2024-04-15T15:20:28.660927+0000","last_undegraded":"2024-04-15T15:20:28.660927+0000","last_fullsized":"2024-04-15T15:20:28.660927+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:07:23.296102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.648232+0000","last_change":"2024-04-15T15:20:28.648232+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.648232+0000","last_undegraded":"2024-04-15T15:20:28.648232+0000","last_fullsized":"2024-04-15T15:20:28.648232+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.666683+0000","last_change":"2024-04-15T15:20:28.662320+0000","last_active":"2024-04-15T15:20:28.666683+0000","last_peered":"2024-04-15T15:20:28.666683+0000","last_clean":"2024-04-15T15:20:28.666683+0000","last_became_active":"2024-04-15T15:20:28.662030+0000","last_became_peered":"2024-04-15T15:20:28.662030+0000","last_unstale":"2024-04-15T15:20:28.666683+0000","last_undegraded":"2024-04-15T15:20:28.666683+0000","last_fullsized":"2024-04-15T15:20:28.666683+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:08:06.553533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.648304+0000","last_change":"2024-04-15T15:20:28.648304+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.648304+0000","last_undegraded":"2024-04-15T15:20:28.648304+0000","last_fullsized":"2024-04-15T15:20:28.648304+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.669454+0000","last_change":"2024-04-15T15:20:28.661886+0000","last_active":"2024-04-15T15:20:28.669454+0000","last_peered":"2024-04-15T15:20:28.669454+0000","last_clean":"2024-04-15T15:20:28.669454+0000","last_became_active":"2024-04-15T15:20:28.661595+0000","last_became_peered":"2024-04-15T15:20:28.661595+0000","last_unstale":"2024-04-15T15:20:28.669454+0000","last_undegraded":"2024-04-15T15:20:28.669454+0000","last_fullsized":"2024-04-15T15:20:28.669454+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:28:40.356127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.648412+0000","last_change":"2024-04-15T15:20:28.648412+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.648412+0000","last_undegraded":"2024-04-15T15:20:28.648412+0000","last_fullsized":"2024-04-15T15:20:28.648412+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.666609+0000","last_change":"2024-04-15T15:20:28.660511+0000","last_active":"2024-04-15T15:20:28.666609+0000","last_peered":"2024-04-15T15:20:28.666609+0000","last_clean":"2024-04-15T15:20:28.666609+0000","last_became_active":"2024-04-15T15:20:28.660182+0000","last_became_peered":"2024-04-15T15:20:28.660182+0000","last_unstale":"2024-04-15T15:20:28.666609+0000","last_undegraded":"2024-04-15T15:20:28.666609+0000","last_fullsized":"2024-04-15T15:20:28.666609+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T03:16:13.044979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.648510+0000","last_change":"2024-04-15T15:20:28.648510+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.648510+0000","last_undegraded":"2024-04-15T15:20:28.648510+0000","last_fullsized":"2024-04-15T15:20:28.648510+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.662438+0000","last_change":"2024-04-15T15:20:28.659967+0000","last_active":"2024-04-15T15:20:28.662438+0000","last_peered":"2024-04-15T15:20:28.662438+0000","last_clean":"2024-04-15T15:20:28.662438+0000","last_became_active":"2024-04-15T15:20:28.659679+0000","last_became_peered":"2024-04-15T15:20:28.659679+0000","last_unstale":"2024-04-15T15:20:28.662438+0000","last_undegraded":"2024-04-15T15:20:28.662438+0000","last_fullsized":"2024-04-15T15:20:28.662438+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:33:51.571295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.654328+0000","last_change":"2024-04-15T15:20:28.654328+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.654328+0000","last_undegraded":"2024-04-15T15:20:28.654328+0000","last_fullsized":"2024-04-15T15:20:28.654328+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.662744+0000","last_change":"2024-04-15T15:20:28.658381+0000","last_active":"2024-04-15T15:20:28.662744+0000","last_peered":"2024-04-15T15:20:28.662744+0000","last_clean":"2024-04-15T15:20:28.662744+0000","last_became_active":"2024-04-15T15:20:28.658066+0000","last_became_peered":"2024-04-15T15:20:28.658066+0000","last_unstale":"2024-04-15T15:20:28.662744+0000","last_undegraded":"2024-04-15T15:20:28.662744+0000","last_fullsized":"2024-04-15T15:20:28.662744+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:40:45.758256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.648564+0000","last_change":"2024-04-15T15:20:28.648564+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.648564+0000","last_undegraded":"2024-04-15T15:20:28.648564+0000","last_fullsized":"2024-04-15T15:20:28.648564+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.666958+0000","last_change":"2024-04-15T15:20:28.663868+0000","last_active":"2024-04-15T15:20:28.666958+0000","last_peered":"2024-04-15T15:20:28.666958+0000","last_clean":"2024-04-15T15:20:28.666958+0000","last_became_active":"2024-04-15T15:20:28.663619+0000","last_became_peered":"2024-04-15T15:20:28.663619+0000","last_unstale":"2024-04-15T15:20:28.666958+0000","last_undegraded":"2024-04-15T15:20:28.666958+0000","last_fullsized":"2024-04-15T15:20:28.666958+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:10:59.194609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.653245+0000","last_change":"2024-04-15T15:20:28.653245+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.653245+0000","last_undegraded":"2024-04-15T15:20:28.653245+0000","last_fullsized":"2024-04-15T15:20:28.653245+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.669532+0000","last_change":"2024-04-15T15:20:28.662929+0000","last_active":"2024-04-15T15:20:28.669532+0000","last_peered":"2024-04-15T15:20:28.669532+0000","last_clean":"2024-04-15T15:20:28.669532+0000","last_became_active":"2024-04-15T15:20:28.662666+0000","last_became_peered":"2024-04-15T15:20:28.662666+0000","last_unstale":"2024-04-15T15:20:28.669532+0000","last_undegraded":"2024-04-15T15:20:28.669532+0000","last_fullsized":"2024-04-15T15:20:28.669532+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:54:49.252832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.653064+0000","last_change":"2024-04-15T15:20:28.653064+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.653064+0000","last_undegraded":"2024-04-15T15:20:28.653064+0000","last_fullsized":"2024-04-15T15:20:28.653064+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.664841+0000","last_change":"2024-04-15T15:20:28.660911+0000","last_active":"2024-04-15T15:20:28.664841+0000","last_peered":"2024-04-15T15:20:28.664841+0000","last_clean":"2024-04-15T15:20:28.664841+0000","last_became_active":"2024-04-15T15:20:28.660620+0000","last_became_peered":"2024-04-15T15:20:28.660620+0000","last_unstale":"2024-04-15T15:20:28.664841+0000","last_undegraded":"2024-04-15T15:20:28.664841+0000","last_fullsized":"2024-04-15T15:20:28.664841+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:11:11.096685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.648049+0000","last_change":"2024-04-15T15:20:28.648049+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.648049+0000","last_undegraded":"2024-04-15T15:20:28.648049+0000","last_fullsized":"2024-04-15T15:20:28.648049+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.665340+0000","last_change":"2024-04-15T15:20:28.659430+0000","last_active":"2024-04-15T15:20:28.665340+0000","last_peered":"2024-04-15T15:20:28.665340+0000","last_clean":"2024-04-15T15:20:28.665340+0000","last_became_active":"2024-04-15T15:20:28.659182+0000","last_became_peered":"2024-04-15T15:20:28.659182+0000","last_unstale":"2024-04-15T15:20:28.665340+0000","last_undegraded":"2024-04-15T15:20:28.665340+0000","last_fullsized":"2024-04-15T15:20:28.665340+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:10:59.194609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.652596+0000","last_change":"2024-04-15T15:20:28.652596+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.652596+0000","last_undegraded":"2024-04-15T15:20:28.652596+0000","last_fullsized":"2024-04-15T15:20:28.652596+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.666993+0000","last_change":"2024-04-15T15:20:28.664511+0000","last_active":"2024-04-15T15:20:28.666993+0000","last_peered":"2024-04-15T15:20:28.666993+0000","last_clean":"2024-04-15T15:20:28.666993+0000","last_became_active":"2024-04-15T15:20:28.664269+0000","last_became_peered":"2024-04-15T15:20:28.664269+0000","last_unstale":"2024-04-15T15:20:28.666993+0000","last_undegraded":"2024-04-15T15:20:28.666993+0000","last_fullsized":"2024-04-15T15:20:28.666993+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:28:40.356127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.653762+0000","last_change":"2024-04-15T15:20:28.653762+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.653762+0000","last_undegraded":"2024-04-15T15:20:28.653762+0000","last_fullsized":"2024-04-15T15:20:28.653762+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.661322+0000","last_change":"2024-04-15T15:20:28.658052+0000","last_active":"2024-04-15T15:20:28.661322+0000","last_peered":"2024-04-15T15:20:28.661322+0000","last_clean":"2024-04-15T15:20:28.661322+0000","last_became_active":"2024-04-15T15:20:28.657686+0000","last_became_peered":"2024-04-15T15:20:28.657686+0000","last_unstale":"2024-04-15T15:20:28.661322+0000","last_undegraded":"2024-04-15T15:20:28.661322+0000","last_fullsized":"2024-04-15T15:20:28.661322+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:39:39.696390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.652530+0000","last_change":"2024-04-15T15:20:28.652530+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.652530+0000","last_undegraded":"2024-04-15T15:20:28.652530+0000","last_fullsized":"2024-04-15T15:20:28.652530+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.669517+0000","last_change":"2024-04-15T15:20:28.665696+0000","last_active":"2024-04-15T15:20:28.669517+0000","last_peered":"2024-04-15T15:20:28.669517+0000","last_clean":"2024-04-15T15:20:28.669517+0000","last_became_active":"2024-04-15T15:20:28.665180+0000","last_became_peered":"2024-04-15T15:20:28.665180+0000","last_unstale":"2024-04-15T15:20:28.669517+0000","last_undegraded":"2024-04-15T15:20:28.669517+0000","last_fullsized":"2024-04-15T15:20:28.669517+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:47:14.250326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-15T15:20:28.654753+0000","last_change":"2024-04-15T15:20:28.654753+0000","last_active":"2024-04-15T15:20:28.638297+0000","last_peered":"2024-04-15T15:20:28.638297+0000","last_clean":"2024-04-15T15:20:28.638297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-15T15:20:28.654753+0000","last_undegraded":"2024-04-15T15:20:28.654753+0000","last_fullsized":"2024-04-15T15:20:28.654753+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.664212+0000","last_change":"2024-04-15T15:20:28.659582+0000","last_active":"2024-04-15T15:20:28.664212+0000","last_peered":"2024-04-15T15:20:28.664212+0000","last_clean":"2024-04-15T15:20:28.664212+0000","last_became_active":"2024-04-15T15:20:28.659266+0000","last_became_peered":"2024-04-15T15:20:28.659266+0000","last_unstale":"2024-04-15T15:20:28.664212+0000","last_undegraded":"2024-04-15T15:20:28.664212+0000","last_fullsized":"2024-04-15T15:20:28.664212+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:10:59.194609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":193,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-15T15:20:28.656297+0000","last_change":"2024-04-15T15:19:14.878269+0000","last_active":"2024-04-15T15:20:28.656297+0000","last_peered":"2024-04-15T15:20:28.656297+0000","last_clean":"2024-04-15T15:20:28.656297+0000","last_became_active":"2024-04-15T15:19:14.877367+0000","last_became_peered":"2024-04-15T15:19:14.877367+0000","last_unstale":"2024-04-15T15:20:28.656297+0000","last_undegraded":"2024-04-15T15:20:28.656297+0000","last_fullsized":"2024-04-15T15:20:28.656297+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:13.827511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:13.827511+0000","last_clean_scrub_stamp":"2024-04-15T15:19:13.827511+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:32:55.902561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574866,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":942080,"data_stored":722949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":360448,"data_stored":138848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":942080,"data_stored":722949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-15T15:20:37.057 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-15T15:20:37.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.171+0000 7f6003021640 1 Processor -- start 2024-04-15T15:20:37.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.171+0000 7f6003021640 1 -- start start 2024-04-15T15:20:37.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.172+0000 7f6003021640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5ffc0722c0 0x7f5ffc06e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:37.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.172+0000 7f6003021640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ffc0ccfb0 con 0x7f5ffc071e60 2024-04-15T15:20:37.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.172+0000 7f6003021640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ffc0cd290 con 0x7f5ffc145e00 2024-04-15T15:20:37.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.172+0000 7f6003021640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f5ffc0cdbc0 con 0x7f5ffc0722c0 2024-04-15T15:20:37.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.172+0000 7f5ffbfff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f5ffc071e60 0x7f5ffc06df80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:43726/0 (socket says 172.21.15.105:43726) 2024-04-15T15:20:37.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.172+0000 7f5ffbfff640 1 -- 172.21.15.105:0/3907147182 learned_addr learned my addr 172.21.15.105:0/3907147182 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:20:37.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.173+0000 7f60015be640 1 --2- 172.21.15.105:0/3907147182 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5ffc0722c0 0x7f5ffc06e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:37.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.173+0000 7f5ffb7fe640 1 -- 172.21.15.105:0/3907147182 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2488884518 0 0) 0x7f5ffc0ccfb0 con 0x7f5ffc071e60 2024-04-15T15:20:37.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.173+0000 7f5ffb7fe640 1 -- 172.21.15.105:0/3907147182 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5fe4003520 con 0x7f5ffc071e60 2024-04-15T15:20:37.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.173+0000 7f5ffb7fe640 1 -- 172.21.15.105:0/3907147182 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1730407648 0 0) 0x7f5fe4003520 con 0x7f5ffc071e60 2024-04-15T15:20:37.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.173+0000 7f5ffb7fe640 1 -- 172.21.15.105:0/3907147182 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5ffc0722c0 msgr2=0x7f5ffc06e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:37.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.173+0000 7f5ffb7fe640 1 --2- 172.21.15.105:0/3907147182 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5ffc0722c0 0x7f5ffc06e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:37.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.173+0000 7f5ffb7fe640 1 -- 172.21.15.105:0/3907147182 >> v1:172.21.15.154:6789/0 conn(0x7f5ffc145e00 legacy=0x7f5ffc0ce7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:37.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.173+0000 7f5ffb7fe640 1 -- 172.21.15.105:0/3907147182 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ffc0cd570 con 0x7f5ffc071e60 2024-04-15T15:20:37.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.173+0000 7f5ffb7fe640 1 -- 172.21.15.105:0/3907147182 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7f5ff0001670 con 0x7f5ffc071e60 2024-04-15T15:20:37.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.173+0000 7f5ffb7fe640 1 -- 172.21.15.105:0/3907147182 >> v1:172.21.15.105:6789/0 conn(0x7f5ffc071e60 legacy=0x7f5ffc06df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:37.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.173+0000 7f5ffb7fe640 1 --2- 172.21.15.105:0/3907147182 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5ffc0722c0 0x7f5fe40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:37.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.173+0000 7f5ffb7fe640 1 --2- 172.21.15.105:0/3907147182 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5fe4003f40 0x7f5fe4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:37.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.173+0000 7f5ffb7fe640 1 --2- 172.21.15.105:0/3907147182 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fe4004d40 0x7f5fe40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:37.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.173+0000 7f5ffb7fe640 1 -- 172.21.15.105:0/3907147182 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5ffc0cdbc0 con 0x7f5fe4004d40 2024-04-15T15:20:37.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.173+0000 7f5ffb7fe640 1 -- 172.21.15.105:0/3907147182 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe400ba70 con 0x7f5fe4003f40 2024-04-15T15:20:37.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.173+0000 7f5ffb7fe640 1 -- 172.21.15.105:0/3907147182 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f5fe400bc40 con 0x7f5ffc0722c0 2024-04-15T15:20:37.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.173+0000 7f60015be640 1 --2- 172.21.15.105:0/3907147182 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fe4004d40 0x7f5fe40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:37.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.173+0000 7f6000dbd640 1 --2- 172.21.15.105:0/3907147182 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5ffc0722c0 0x7f5fe40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:37.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.173+0000 7f5ffbfff640 1 --2- 172.21.15.105:0/3907147182 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5fe4003f40 0x7f5fe4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:37.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.173+0000 7f60015be640 1 -- 172.21.15.105:0/3907147182 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5ffc0722c0 msgr2=0x7f5fe40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:37.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.173+0000 7f60015be640 1 --2- 172.21.15.105:0/3907147182 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5ffc0722c0 0x7f5fe40039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:37.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.173+0000 7f60015be640 1 -- 172.21.15.105:0/3907147182 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5fe4003f40 msgr2=0x7f5fe4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:37.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.173+0000 7f60015be640 1 --2- 172.21.15.105:0/3907147182 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5fe4003f40 0x7f5fe4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:37.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.173+0000 7f60015be640 1 -- 172.21.15.105:0/3907147182 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fe4003520 con 0x7f5fe4004d40 2024-04-15T15:20:37.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.174+0000 7f60015be640 1 --2- 172.21.15.105:0/3907147182 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fe4004d40 0x7f5fe40051c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f5ff4009fd0 tx=0x7f5fe4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dc60dc5744c6bc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:37.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.174+0000 7f5ffb7fe640 1 -- 172.21.15.105:0/3907147182 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff4039070 con 0x7f5fe4004d40 2024-04-15T15:20:37.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.174+0000 7f5ffb7fe640 1 -- 172.21.15.105:0/3907147182 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ff402cc90 con 0x7f5fe4004d40 2024-04-15T15:20:37.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.174+0000 7f5ffb7fe640 1 -- 172.21.15.105:0/3907147182 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff4009bf0 con 0x7f5fe4004d40 2024-04-15T15:20:37.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.175+0000 7f6003021640 1 -- 172.21.15.105:0/3907147182 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fe4004d40 msgr2=0x7f5fe40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:37.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.175+0000 7f6003021640 1 --2- 172.21.15.105:0/3907147182 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fe4004d40 0x7f5fe40051c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f5ff4009fd0 tx=0x7f5fe4005830 comp rx=0 tx=0).stop 2024-04-15T15:20:37.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.175+0000 7f60015be640 1 -- 172.21.15.105:0/3907147182 reap_dead start 2024-04-15T15:20:37.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.175+0000 7f6003021640 1 -- 172.21.15.105:0/3907147182 shutdown_connections 2024-04-15T15:20:37.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.175+0000 7f6003021640 1 -- 172.21.15.105:0/3907147182 >> 172.21.15.105:0/3907147182 conn(0x7f5ffc067d50 msgr2=0x7f5ffc06b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:37.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.175+0000 7f6003021640 1 -- 172.21.15.105:0/3907147182 shutdown_connections 2024-04-15T15:20:37.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.175+0000 7f6003021640 1 -- 172.21.15.105:0/3907147182 wait complete. 2024-04-15T15:20:37.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.176+0000 7f6003021640 1 Processor -- start 2024-04-15T15:20:37.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.176+0000 7f6003021640 1 -- start start 2024-04-15T15:20:37.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.177+0000 7f6003021640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ffc158000 0x7f5ffc1e9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:37.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.177+0000 7f6003021640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5ffc1e9cb0 0x7f5ffc141c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:37.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.177+0000 7f6003021640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ffc1583e0 0x7f5ffc140fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:37.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.177+0000 7f6003021640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5ffc0cef00 con 0x7f5ffc158000 2024-04-15T15:20:37.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.177+0000 7f6003021640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5ffc072710 con 0x7f5ffc1583e0 2024-04-15T15:20:37.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.177+0000 7f6003021640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f5ffc0ce370 con 0x7f5ffc1e9cb0 2024-04-15T15:20:37.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.177+0000 7f6000dbd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ffc158000 0x7f5ffc1e9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:37.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.177+0000 7f6000dbd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ffc158000 0x7f5ffc1e9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45346/0 (socket says 172.21.15.105:45346) 2024-04-15T15:20:37.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.177+0000 7f6000dbd640 1 -- 172.21.15.105:0/1733696808 learned_addr learned my addr 172.21.15.105:0/1733696808 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:37.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.177+0000 7f60015be640 1 --2- 172.21.15.105:0/1733696808 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ffc1583e0 0x7f5ffc140fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:37.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.177+0000 7f5ffbfff640 1 --2- 172.21.15.105:0/1733696808 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5ffc1e9cb0 0x7f5ffc141c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:37.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.177+0000 7f6000dbd640 1 -- 172.21.15.105:0/1733696808 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5ffc1e9cb0 msgr2=0x7f5ffc141c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:37.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.178+0000 7f6000dbd640 1 --2- 172.21.15.105:0/1733696808 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5ffc1e9cb0 0x7f5ffc141c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:37.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.178+0000 7f6000dbd640 1 -- 172.21.15.105:0/1733696808 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ffc1583e0 msgr2=0x7f5ffc140fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:37.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.178+0000 7f6000dbd640 1 --2- 172.21.15.105:0/1733696808 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ffc1583e0 0x7f5ffc140fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:37.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.178+0000 7f6000dbd640 1 -- 172.21.15.105:0/1733696808 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ffc1e86d0 con 0x7f5ffc158000 2024-04-15T15:20:37.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.178+0000 7f60015be640 1 --2- 172.21.15.105:0/1733696808 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ffc1583e0 0x7f5ffc140fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:37.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.178+0000 7f5ffbfff640 1 --2- 172.21.15.105:0/1733696808 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5ffc1e9cb0 0x7f5ffc141c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:37.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.178+0000 7f6000dbd640 1 --2- 172.21.15.105:0/1733696808 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ffc158000 0x7f5ffc1e9770 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f5fec00f7f0 tx=0x7f5fec00fcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:37.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.178+0000 7f5ff97fa640 1 -- 172.21.15.105:0/1733696808 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fec0052e0 con 0x7f5ffc158000 2024-04-15T15:20:37.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.178+0000 7f5ff97fa640 1 -- 172.21.15.105:0/1733696808 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fec018070 con 0x7f5ffc158000 2024-04-15T15:20:37.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.178+0000 7f5ff97fa640 1 -- 172.21.15.105:0/1733696808 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fec00d970 con 0x7f5ffc158000 2024-04-15T15:20:37.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.178+0000 7f6003021640 1 -- 172.21.15.105:0/1733696808 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ffc1e89b0 con 0x7f5ffc158000 2024-04-15T15:20:37.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.178+0000 7f6003021640 1 -- 172.21.15.105:0/1733696808 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ffc1e9250 con 0x7f5ffc158000 2024-04-15T15:20:37.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.179+0000 7f6003021640 1 -- 172.21.15.105:0/1733696808 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ffc1e8b80 con 0x7f5ffc158000 2024-04-15T15:20:37.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.179+0000 7f5ff97fa640 1 -- 172.21.15.105:0/1733696808 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5fec00db10 con 0x7f5ffc158000 2024-04-15T15:20:37.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.180+0000 7f5ff97fa640 1 --2- 172.21.15.105:0/1733696808 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f5fe806c8f0 0x7f5fe806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:37.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.180+0000 7f5ff97fa640 1 -- 172.21.15.105:0/1733696808 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f5fec08ba50 con 0x7f5ffc158000 2024-04-15T15:20:37.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.180+0000 7f5ffbfff640 1 --2- 172.21.15.105:0/1733696808 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f5fe806c8f0 0x7f5fe806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:37.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.180+0000 7f5ffbfff640 1 --2- 172.21.15.105:0/1733696808 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f5fe806c8f0 0x7f5fe806edb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc1f58f0 tx=0x7f5ff0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:37.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.185+0000 7f5ff97fa640 1 -- 172.21.15.105:0/1733696808 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5fec01a310 con 0x7f5ffc158000 2024-04-15T15:20:37.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.354+0000 7f6003021640 1 -- 172.21.15.105:0/1733696808 --> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5ffc068a40 con 0x7f5fe806c8f0 2024-04-15T15:20:37.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.360+0000 7f5ff97fa640 1 -- 172.21.15.105:0/1733696808 <== mgr.4107 v2:172.21.15.105:6832/3506549322 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251951 (secure 0 0 0) 0x7f5ffc068a40 con 0x7f5fe806c8f0 2024-04-15T15:20:37.364 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:20:37.365 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-04-15T15:20:37.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.363+0000 7f6003021640 1 -- 172.21.15.105:0/1733696808 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f5fe806c8f0 msgr2=0x7f5fe806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:37.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.363+0000 7f6003021640 1 --2- 172.21.15.105:0/1733696808 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f5fe806c8f0 0x7f5fe806edb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc1f58f0 tx=0x7f5ff0009040 comp rx=0 tx=0).stop 2024-04-15T15:20:37.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.363+0000 7f6003021640 1 -- 172.21.15.105:0/1733696808 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ffc158000 msgr2=0x7f5ffc1e9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:37.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.363+0000 7f6003021640 1 --2- 172.21.15.105:0/1733696808 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ffc158000 0x7f5ffc1e9770 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f5fec00f7f0 tx=0x7f5fec00fcb0 comp rx=0 tx=0).stop 2024-04-15T15:20:37.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.363+0000 7f6003021640 1 -- 172.21.15.105:0/1733696808 shutdown_connections 2024-04-15T15:20:37.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.363+0000 7f6003021640 1 --2- 172.21.15.105:0/1733696808 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5ffc1e9cb0 0x7f5ffc141c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:37.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.363+0000 7f6003021640 1 --2- 172.21.15.105:0/1733696808 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f5fe806c8f0 0x7f5fe806edb0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:37.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.363+0000 7f6003021640 1 --2- 172.21.15.105:0/1733696808 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ffc1583e0 0x7f5ffc140fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:37.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.363+0000 7f6003021640 1 --2- 172.21.15.105:0/1733696808 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ffc158000 0x7f5ffc1e9770 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:37.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.363+0000 7f6003021640 1 -- 172.21.15.105:0/1733696808 >> 172.21.15.105:0/1733696808 conn(0x7f5ffc067d50 msgr2=0x7f5ffc160030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:37.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.363+0000 7f6003021640 1 -- 172.21.15.105:0/1733696808 shutdown_connections 2024-04-15T15:20:37.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.363+0000 7f6003021640 1 -- 172.21.15.105:0/1733696808 wait complete. 2024-04-15T15:20:37.376 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":68,"stamp":"2024-04-15T15:20:35.746550+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":182,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220424,"kb_used_data":4232,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754296,"statfs":{"total":773094113280,"available":772868399104,"internally_reserved":0,"allocated":4333568,"data_stored":2400940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.100782"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:31.432288+0000","last_change":"2024-04-15T15:20:29.664613+0000","last_active":"2024-04-15T15:20:31.432288+0000","last_peered":"2024-04-15T15:20:31.432288+0000","last_clean":"2024-04-15T15:20:31.432288+0000","last_became_active":"2024-04-15T15:20:29.664121+0000","last_became_peered":"2024-04-15T15:20:29.664121+0000","last_unstale":"2024-04-15T15:20:31.432288+0000","last_undegraded":"2024-04-15T15:20:31.432288+0000","last_fullsized":"2024-04-15T15:20:31.432288+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:13:57.702462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:30.649309+0000","last_change":"2024-04-15T15:20:29.668093+0000","last_active":"2024-04-15T15:20:30.649309+0000","last_peered":"2024-04-15T15:20:30.649309+0000","last_clean":"2024-04-15T15:20:30.649309+0000","last_became_active":"2024-04-15T15:20:29.667798+0000","last_became_peered":"2024-04-15T15:20:29.667798+0000","last_unstale":"2024-04-15T15:20:30.649309+0000","last_undegraded":"2024-04-15T15:20:30.649309+0000","last_fullsized":"2024-04-15T15:20:30.649309+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:38:58.777641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:31.385558+0000","last_change":"2024-04-15T15:20:29.673510+0000","last_active":"2024-04-15T15:20:31.385558+0000","last_peered":"2024-04-15T15:20:31.385558+0000","last_clean":"2024-04-15T15:20:31.385558+0000","last_became_active":"2024-04-15T15:20:29.673191+0000","last_became_peered":"2024-04-15T15:20:29.673191+0000","last_unstale":"2024-04-15T15:20:31.385558+0000","last_undegraded":"2024-04-15T15:20:31.385558+0000","last_fullsized":"2024-04-15T15:20:31.385558+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:43:13.368696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:31.432026+0000","last_change":"2024-04-15T15:20:29.675352+0000","last_active":"2024-04-15T15:20:31.432026+0000","last_peered":"2024-04-15T15:20:31.432026+0000","last_clean":"2024-04-15T15:20:31.432026+0000","last_became_active":"2024-04-15T15:20:29.675113+0000","last_became_peered":"2024-04-15T15:20:29.675113+0000","last_unstale":"2024-04-15T15:20:31.432026+0000","last_undegraded":"2024-04-15T15:20:31.432026+0000","last_fullsized":"2024-04-15T15:20:31.432026+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:05:07.345787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:30.650033+0000","last_change":"2024-04-15T15:20:29.668300+0000","last_active":"2024-04-15T15:20:30.650033+0000","last_peered":"2024-04-15T15:20:30.650033+0000","last_clean":"2024-04-15T15:20:30.650033+0000","last_became_active":"2024-04-15T15:20:29.668057+0000","last_became_peered":"2024-04-15T15:20:29.668057+0000","last_unstale":"2024-04-15T15:20:30.650033+0000","last_undegraded":"2024-04-15T15:20:30.650033+0000","last_fullsized":"2024-04-15T15:20:30.650033+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:16:19.674738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:30.649061+0000","last_change":"2024-04-15T15:20:29.667030+0000","last_active":"2024-04-15T15:20:30.649061+0000","last_peered":"2024-04-15T15:20:30.649061+0000","last_clean":"2024-04-15T15:20:30.649061+0000","last_became_active":"2024-04-15T15:20:29.666738+0000","last_became_peered":"2024-04-15T15:20:29.666738+0000","last_unstale":"2024-04-15T15:20:30.649061+0000","last_undegraded":"2024-04-15T15:20:30.649061+0000","last_fullsized":"2024-04-15T15:20:30.649061+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:15:37.566764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:30.650124+0000","last_change":"2024-04-15T15:20:29.664641+0000","last_active":"2024-04-15T15:20:30.650124+0000","last_peered":"2024-04-15T15:20:30.650124+0000","last_clean":"2024-04-15T15:20:30.650124+0000","last_became_active":"2024-04-15T15:20:29.664389+0000","last_became_peered":"2024-04-15T15:20:29.664389+0000","last_unstale":"2024-04-15T15:20:30.650124+0000","last_undegraded":"2024-04-15T15:20:30.650124+0000","last_fullsized":"2024-04-15T15:20:30.650124+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:21:05.994904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:30.649748+0000","last_change":"2024-04-15T15:20:29.670268+0000","last_active":"2024-04-15T15:20:30.649748+0000","last_peered":"2024-04-15T15:20:30.649748+0000","last_clean":"2024-04-15T15:20:30.649748+0000","last_became_active":"2024-04-15T15:20:29.670009+0000","last_became_peered":"2024-04-15T15:20:29.670009+0000","last_unstale":"2024-04-15T15:20:30.649748+0000","last_undegraded":"2024-04-15T15:20:30.649748+0000","last_fullsized":"2024-04-15T15:20:30.649748+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:01:21.353275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:31.432710+0000","last_change":"2024-04-15T15:20:29.663107+0000","last_active":"2024-04-15T15:20:31.432710+0000","last_peered":"2024-04-15T15:20:31.432710+0000","last_clean":"2024-04-15T15:20:31.432710+0000","last_became_active":"2024-04-15T15:20:29.662813+0000","last_became_peered":"2024-04-15T15:20:29.662813+0000","last_unstale":"2024-04-15T15:20:31.432710+0000","last_undegraded":"2024-04-15T15:20:31.432710+0000","last_fullsized":"2024-04-15T15:20:31.432710+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:05:07.345787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:31.432220+0000","last_change":"2024-04-15T15:20:29.668774+0000","last_active":"2024-04-15T15:20:31.432220+0000","last_peered":"2024-04-15T15:20:31.432220+0000","last_clean":"2024-04-15T15:20:31.432220+0000","last_became_active":"2024-04-15T15:20:29.668525+0000","last_became_peered":"2024-04-15T15:20:29.668525+0000","last_unstale":"2024-04-15T15:20:31.432220+0000","last_undegraded":"2024-04-15T15:20:31.432220+0000","last_fullsized":"2024-04-15T15:20:31.432220+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:01:21.353275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:30.650034+0000","last_change":"2024-04-15T15:20:29.663887+0000","last_active":"2024-04-15T15:20:30.650034+0000","last_peered":"2024-04-15T15:20:30.650034+0000","last_clean":"2024-04-15T15:20:30.650034+0000","last_became_active":"2024-04-15T15:20:29.663633+0000","last_became_peered":"2024-04-15T15:20:29.663633+0000","last_unstale":"2024-04-15T15:20:30.650034+0000","last_undegraded":"2024-04-15T15:20:30.650034+0000","last_fullsized":"2024-04-15T15:20:30.650034+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:16:14.257551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:31.385408+0000","last_change":"2024-04-15T15:20:29.670874+0000","last_active":"2024-04-15T15:20:31.385408+0000","last_peered":"2024-04-15T15:20:31.385408+0000","last_clean":"2024-04-15T15:20:31.385408+0000","last_became_active":"2024-04-15T15:20:29.670571+0000","last_became_peered":"2024-04-15T15:20:29.670571+0000","last_unstale":"2024-04-15T15:20:31.385408+0000","last_undegraded":"2024-04-15T15:20:31.385408+0000","last_fullsized":"2024-04-15T15:20:31.385408+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:52:29.908065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:30.650074+0000","last_change":"2024-04-15T15:20:29.677522+0000","last_active":"2024-04-15T15:20:30.650074+0000","last_peered":"2024-04-15T15:20:30.650074+0000","last_clean":"2024-04-15T15:20:30.650074+0000","last_became_active":"2024-04-15T15:20:29.677255+0000","last_became_peered":"2024-04-15T15:20:29.677255+0000","last_unstale":"2024-04-15T15:20:30.650074+0000","last_undegraded":"2024-04-15T15:20:30.650074+0000","last_fullsized":"2024-04-15T15:20:30.650074+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:53:09.916038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:30.649195+0000","last_change":"2024-04-15T15:20:29.658482+0000","last_active":"2024-04-15T15:20:30.649195+0000","last_peered":"2024-04-15T15:20:30.649195+0000","last_clean":"2024-04-15T15:20:30.649195+0000","last_became_active":"2024-04-15T15:20:29.658246+0000","last_became_peered":"2024-04-15T15:20:29.658246+0000","last_unstale":"2024-04-15T15:20:30.649195+0000","last_undegraded":"2024-04-15T15:20:30.649195+0000","last_fullsized":"2024-04-15T15:20:30.649195+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:43:13.368696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:31.385235+0000","last_change":"2024-04-15T15:20:29.666559+0000","last_active":"2024-04-15T15:20:31.385235+0000","last_peered":"2024-04-15T15:20:31.385235+0000","last_clean":"2024-04-15T15:20:31.385235+0000","last_became_active":"2024-04-15T15:20:29.666179+0000","last_became_peered":"2024-04-15T15:20:29.666179+0000","last_unstale":"2024-04-15T15:20:31.385235+0000","last_undegraded":"2024-04-15T15:20:31.385235+0000","last_fullsized":"2024-04-15T15:20:31.385235+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:21:05.994904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:30.650489+0000","last_change":"2024-04-15T15:20:29.669952+0000","last_active":"2024-04-15T15:20:30.650489+0000","last_peered":"2024-04-15T15:20:30.650489+0000","last_clean":"2024-04-15T15:20:30.650489+0000","last_became_active":"2024-04-15T15:20:29.668479+0000","last_became_peered":"2024-04-15T15:20:29.668479+0000","last_unstale":"2024-04-15T15:20:30.650489+0000","last_undegraded":"2024-04-15T15:20:30.650489+0000","last_fullsized":"2024-04-15T15:20:30.650489+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:59:38.044606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:30.650366+0000","last_change":"2024-04-15T15:20:29.660845+0000","last_active":"2024-04-15T15:20:30.650366+0000","last_peered":"2024-04-15T15:20:30.650366+0000","last_clean":"2024-04-15T15:20:30.650366+0000","last_became_active":"2024-04-15T15:20:29.660605+0000","last_became_peered":"2024-04-15T15:20:29.660605+0000","last_unstale":"2024-04-15T15:20:30.650366+0000","last_undegraded":"2024-04-15T15:20:30.650366+0000","last_fullsized":"2024-04-15T15:20:30.650366+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:43:13.368696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:30.649358+0000","last_change":"2024-04-15T15:20:29.663877+0000","last_active":"2024-04-15T15:20:30.649358+0000","last_peered":"2024-04-15T15:20:30.649358+0000","last_clean":"2024-04-15T15:20:30.649358+0000","last_became_active":"2024-04-15T15:20:29.663623+0000","last_became_peered":"2024-04-15T15:20:29.663623+0000","last_unstale":"2024-04-15T15:20:30.649358+0000","last_undegraded":"2024-04-15T15:20:30.649358+0000","last_fullsized":"2024-04-15T15:20:30.649358+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:33:59.661423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:30.649436+0000","last_change":"2024-04-15T15:20:29.664625+0000","last_active":"2024-04-15T15:20:30.649436+0000","last_peered":"2024-04-15T15:20:30.649436+0000","last_clean":"2024-04-15T15:20:30.649436+0000","last_became_active":"2024-04-15T15:20:29.663566+0000","last_became_peered":"2024-04-15T15:20:29.663566+0000","last_unstale":"2024-04-15T15:20:30.649436+0000","last_undegraded":"2024-04-15T15:20:30.649436+0000","last_fullsized":"2024-04-15T15:20:30.649436+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:35:55.582857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:30.650141+0000","last_change":"2024-04-15T15:20:29.666394+0000","last_active":"2024-04-15T15:20:30.650141+0000","last_peered":"2024-04-15T15:20:30.650141+0000","last_clean":"2024-04-15T15:20:30.650141+0000","last_became_active":"2024-04-15T15:20:29.666122+0000","last_became_peered":"2024-04-15T15:20:29.666122+0000","last_unstale":"2024-04-15T15:20:30.650141+0000","last_undegraded":"2024-04-15T15:20:30.650141+0000","last_fullsized":"2024-04-15T15:20:30.650141+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:10:53.683169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:31.432234+0000","last_change":"2024-04-15T15:20:29.669169+0000","last_active":"2024-04-15T15:20:31.432234+0000","last_peered":"2024-04-15T15:20:31.432234+0000","last_clean":"2024-04-15T15:20:31.432234+0000","last_became_active":"2024-04-15T15:20:29.668863+0000","last_became_peered":"2024-04-15T15:20:29.668863+0000","last_unstale":"2024-04-15T15:20:31.432234+0000","last_undegraded":"2024-04-15T15:20:31.432234+0000","last_fullsized":"2024-04-15T15:20:31.432234+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:13:57.702462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:31.432128+0000","last_change":"2024-04-15T15:20:29.661425+0000","last_active":"2024-04-15T15:20:31.432128+0000","last_peered":"2024-04-15T15:20:31.432128+0000","last_clean":"2024-04-15T15:20:31.432128+0000","last_became_active":"2024-04-15T15:20:29.661042+0000","last_became_peered":"2024-04-15T15:20:29.661042+0000","last_unstale":"2024-04-15T15:20:31.432128+0000","last_undegraded":"2024-04-15T15:20:31.432128+0000","last_fullsized":"2024-04-15T15:20:31.432128+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:03:22.000782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:31.431550+0000","last_change":"2024-04-15T15:20:29.670897+0000","last_active":"2024-04-15T15:20:31.431550+0000","last_peered":"2024-04-15T15:20:31.431550+0000","last_clean":"2024-04-15T15:20:31.431550+0000","last_became_active":"2024-04-15T15:20:29.670657+0000","last_became_peered":"2024-04-15T15:20:29.670657+0000","last_unstale":"2024-04-15T15:20:31.431550+0000","last_undegraded":"2024-04-15T15:20:31.431550+0000","last_fullsized":"2024-04-15T15:20:31.431550+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:13:57.702462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:31.385187+0000","last_change":"2024-04-15T15:20:29.669844+0000","last_active":"2024-04-15T15:20:31.385187+0000","last_peered":"2024-04-15T15:20:31.385187+0000","last_clean":"2024-04-15T15:20:31.385187+0000","last_became_active":"2024-04-15T15:20:29.669544+0000","last_became_peered":"2024-04-15T15:20:29.669544+0000","last_unstale":"2024-04-15T15:20:31.385187+0000","last_undegraded":"2024-04-15T15:20:31.385187+0000","last_fullsized":"2024-04-15T15:20:31.385187+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:32:25.149403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:31.432350+0000","last_change":"2024-04-15T15:20:29.672613+0000","last_active":"2024-04-15T15:20:31.432350+0000","last_peered":"2024-04-15T15:20:31.432350+0000","last_clean":"2024-04-15T15:20:31.432350+0000","last_became_active":"2024-04-15T15:20:29.668476+0000","last_became_peered":"2024-04-15T15:20:29.668476+0000","last_unstale":"2024-04-15T15:20:31.432350+0000","last_undegraded":"2024-04-15T15:20:31.432350+0000","last_fullsized":"2024-04-15T15:20:31.432350+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:32:25.149403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:30.649437+0000","last_change":"2024-04-15T15:20:29.668309+0000","last_active":"2024-04-15T15:20:30.649437+0000","last_peered":"2024-04-15T15:20:30.649437+0000","last_clean":"2024-04-15T15:20:30.649437+0000","last_became_active":"2024-04-15T15:20:29.667087+0000","last_became_peered":"2024-04-15T15:20:29.667087+0000","last_unstale":"2024-04-15T15:20:30.649437+0000","last_undegraded":"2024-04-15T15:20:30.649437+0000","last_fullsized":"2024-04-15T15:20:30.649437+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:34:49.814061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:31.432231+0000","last_change":"2024-04-15T15:20:29.674724+0000","last_active":"2024-04-15T15:20:31.432231+0000","last_peered":"2024-04-15T15:20:31.432231+0000","last_clean":"2024-04-15T15:20:31.432231+0000","last_became_active":"2024-04-15T15:20:29.674454+0000","last_became_peered":"2024-04-15T15:20:29.674454+0000","last_unstale":"2024-04-15T15:20:31.432231+0000","last_undegraded":"2024-04-15T15:20:31.432231+0000","last_fullsized":"2024-04-15T15:20:31.432231+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:34:43.134380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:31.385499+0000","last_change":"2024-04-15T15:20:29.667699+0000","last_active":"2024-04-15T15:20:31.385499+0000","last_peered":"2024-04-15T15:20:31.385499+0000","last_clean":"2024-04-15T15:20:31.385499+0000","last_became_active":"2024-04-15T15:20:29.667424+0000","last_became_peered":"2024-04-15T15:20:29.667424+0000","last_unstale":"2024-04-15T15:20:31.385499+0000","last_undegraded":"2024-04-15T15:20:31.385499+0000","last_fullsized":"2024-04-15T15:20:31.385499+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:09:06.486051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:31.432353+0000","last_change":"2024-04-15T15:20:29.660440+0000","last_active":"2024-04-15T15:20:31.432353+0000","last_peered":"2024-04-15T15:20:31.432353+0000","last_clean":"2024-04-15T15:20:31.432353+0000","last_became_active":"2024-04-15T15:20:29.660171+0000","last_became_peered":"2024-04-15T15:20:29.660171+0000","last_unstale":"2024-04-15T15:20:31.432353+0000","last_undegraded":"2024-04-15T15:20:31.432353+0000","last_fullsized":"2024-04-15T15:20:31.432353+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:22:05.003943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:30.650256+0000","last_change":"2024-04-15T15:20:29.659708+0000","last_active":"2024-04-15T15:20:30.650256+0000","last_peered":"2024-04-15T15:20:30.650256+0000","last_clean":"2024-04-15T15:20:30.650256+0000","last_became_active":"2024-04-15T15:20:29.658188+0000","last_became_peered":"2024-04-15T15:20:29.658188+0000","last_unstale":"2024-04-15T15:20:30.650256+0000","last_undegraded":"2024-04-15T15:20:30.650256+0000","last_fullsized":"2024-04-15T15:20:30.650256+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:52:29.908065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:31.432230+0000","last_change":"2024-04-15T15:20:29.658531+0000","last_active":"2024-04-15T15:20:31.432230+0000","last_peered":"2024-04-15T15:20:31.432230+0000","last_clean":"2024-04-15T15:20:31.432230+0000","last_became_active":"2024-04-15T15:20:29.658259+0000","last_became_peered":"2024-04-15T15:20:29.658259+0000","last_unstale":"2024-04-15T15:20:31.432230+0000","last_undegraded":"2024-04-15T15:20:31.432230+0000","last_fullsized":"2024-04-15T15:20:31.432230+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:03:22.000782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:30.650270+0000","last_change":"2024-04-15T15:20:29.659913+0000","last_active":"2024-04-15T15:20:30.650270+0000","last_peered":"2024-04-15T15:20:30.650270+0000","last_clean":"2024-04-15T15:20:30.650270+0000","last_became_active":"2024-04-15T15:20:29.659548+0000","last_became_peered":"2024-04-15T15:20:29.659548+0000","last_unstale":"2024-04-15T15:20:30.650270+0000","last_undegraded":"2024-04-15T15:20:30.650270+0000","last_fullsized":"2024-04-15T15:20:30.650270+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:09:06.486051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:31.431633+0000","last_change":"2024-04-15T15:20:29.669740+0000","last_active":"2024-04-15T15:20:31.431633+0000","last_peered":"2024-04-15T15:20:31.431633+0000","last_clean":"2024-04-15T15:20:31.431633+0000","last_became_active":"2024-04-15T15:20:29.669477+0000","last_became_peered":"2024-04-15T15:20:29.669477+0000","last_unstale":"2024-04-15T15:20:31.431633+0000","last_undegraded":"2024-04-15T15:20:31.431633+0000","last_fullsized":"2024-04-15T15:20:31.431633+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:51:43.410546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":18,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:31.432544+0000","last_change":"2024-04-15T15:20:28.659981+0000","last_active":"2024-04-15T15:20:31.432544+0000","last_peered":"2024-04-15T15:20:31.432544+0000","last_clean":"2024-04-15T15:20:31.432544+0000","last_became_active":"2024-04-15T15:20:28.657535+0000","last_became_peered":"2024-04-15T15:20:28.657535+0000","last_unstale":"2024-04-15T15:20:31.432544+0000","last_undegraded":"2024-04-15T15:20:31.432544+0000","last_fullsized":"2024-04-15T15:20:31.432544+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:16:18.672475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:31.432447+0000","last_change":"2024-04-15T15:20:29.652864+0000","last_active":"2024-04-15T15:20:31.432447+0000","last_peered":"2024-04-15T15:20:31.432447+0000","last_clean":"2024-04-15T15:20:31.432447+0000","last_became_active":"2024-04-15T15:20:29.652579+0000","last_became_peered":"2024-04-15T15:20:29.652579+0000","last_unstale":"2024-04-15T15:20:31.432447+0000","last_undegraded":"2024-04-15T15:20:31.432447+0000","last_fullsized":"2024-04-15T15:20:31.432447+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:51:43.410546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:31.431718+0000","last_change":"2024-04-15T15:20:28.658521+0000","last_active":"2024-04-15T15:20:31.431718+0000","last_peered":"2024-04-15T15:20:31.431718+0000","last_clean":"2024-04-15T15:20:31.431718+0000","last_became_active":"2024-04-15T15:20:28.658266+0000","last_became_peered":"2024-04-15T15:20:28.658266+0000","last_unstale":"2024-04-15T15:20:31.431718+0000","last_undegraded":"2024-04-15T15:20:31.431718+0000","last_fullsized":"2024-04-15T15:20:31.431718+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:16:18.672475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:31.432654+0000","last_change":"2024-04-15T15:20:29.660414+0000","last_active":"2024-04-15T15:20:31.432654+0000","last_peered":"2024-04-15T15:20:31.432654+0000","last_clean":"2024-04-15T15:20:31.432654+0000","last_became_active":"2024-04-15T15:20:29.660126+0000","last_became_peered":"2024-04-15T15:20:29.660126+0000","last_unstale":"2024-04-15T15:20:31.432654+0000","last_undegraded":"2024-04-15T15:20:31.432654+0000","last_fullsized":"2024-04-15T15:20:31.432654+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:41:17.064300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:30.649529+0000","last_change":"2024-04-15T15:20:28.664326+0000","last_active":"2024-04-15T15:20:30.649529+0000","last_peered":"2024-04-15T15:20:30.649529+0000","last_clean":"2024-04-15T15:20:30.649529+0000","last_became_active":"2024-04-15T15:20:28.664077+0000","last_became_peered":"2024-04-15T15:20:28.664077+0000","last_unstale":"2024-04-15T15:20:30.649529+0000","last_undegraded":"2024-04-15T15:20:30.649529+0000","last_fullsized":"2024-04-15T15:20:30.649529+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:22:04.001680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:31.432243+0000","last_change":"2024-04-15T15:20:29.674577+0000","last_active":"2024-04-15T15:20:31.432243+0000","last_peered":"2024-04-15T15:20:31.432243+0000","last_clean":"2024-04-15T15:20:31.432243+0000","last_became_active":"2024-04-15T15:20:29.674393+0000","last_became_peered":"2024-04-15T15:20:29.674393+0000","last_unstale":"2024-04-15T15:20:31.432243+0000","last_undegraded":"2024-04-15T15:20:31.432243+0000","last_fullsized":"2024-04-15T15:20:31.432243+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:16:14.257551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:30.650142+0000","last_change":"2024-04-15T15:20:28.659705+0000","last_active":"2024-04-15T15:20:30.650142+0000","last_peered":"2024-04-15T15:20:30.650142+0000","last_clean":"2024-04-15T15:20:30.650142+0000","last_became_active":"2024-04-15T15:20:28.659433+0000","last_became_peered":"2024-04-15T15:20:28.659433+0000","last_unstale":"2024-04-15T15:20:30.650142+0000","last_undegraded":"2024-04-15T15:20:30.650142+0000","last_fullsized":"2024-04-15T15:20:30.650142+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:06:50.496578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:30.650125+0000","last_change":"2024-04-15T15:20:29.660313+0000","last_active":"2024-04-15T15:20:30.650125+0000","last_peered":"2024-04-15T15:20:30.650125+0000","last_clean":"2024-04-15T15:20:30.650125+0000","last_became_active":"2024-04-15T15:20:29.660063+0000","last_became_peered":"2024-04-15T15:20:29.660063+0000","last_unstale":"2024-04-15T15:20:30.650125+0000","last_undegraded":"2024-04-15T15:20:30.650125+0000","last_fullsized":"2024-04-15T15:20:30.650125+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:40:02.453672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:31.432335+0000","last_change":"2024-04-15T15:20:28.662442+0000","last_active":"2024-04-15T15:20:31.432335+0000","last_peered":"2024-04-15T15:20:31.432335+0000","last_clean":"2024-04-15T15:20:31.432335+0000","last_became_active":"2024-04-15T15:20:28.661241+0000","last_became_peered":"2024-04-15T15:20:28.661241+0000","last_unstale":"2024-04-15T15:20:31.432335+0000","last_undegraded":"2024-04-15T15:20:31.432335+0000","last_fullsized":"2024-04-15T15:20:31.432335+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:28:40.356127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:31.385396+0000","last_change":"2024-04-15T15:20:29.671351+0000","last_active":"2024-04-15T15:20:31.385396+0000","last_peered":"2024-04-15T15:20:31.385396+0000","last_clean":"2024-04-15T15:20:31.385396+0000","last_became_active":"2024-04-15T15:20:29.671072+0000","last_became_peered":"2024-04-15T15:20:29.671072+0000","last_unstale":"2024-04-15T15:20:31.385396+0000","last_undegraded":"2024-04-15T15:20:31.385396+0000","last_fullsized":"2024-04-15T15:20:31.385396+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:40:02.453672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:31.432135+0000","last_change":"2024-04-15T15:20:28.662045+0000","last_active":"2024-04-15T15:20:31.432135+0000","last_peered":"2024-04-15T15:20:31.432135+0000","last_clean":"2024-04-15T15:20:31.432135+0000","last_became_active":"2024-04-15T15:20:28.661769+0000","last_became_peered":"2024-04-15T15:20:28.661769+0000","last_unstale":"2024-04-15T15:20:31.432135+0000","last_undegraded":"2024-04-15T15:20:31.432135+0000","last_fullsized":"2024-04-15T15:20:31.432135+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:56:02.749126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:31.431957+0000","last_change":"2024-04-15T15:20:29.652781+0000","last_active":"2024-04-15T15:20:31.431957+0000","last_peered":"2024-04-15T15:20:31.431957+0000","last_clean":"2024-04-15T15:20:31.431957+0000","last_became_active":"2024-04-15T15:20:29.652399+0000","last_became_peered":"2024-04-15T15:20:29.652399+0000","last_unstale":"2024-04-15T15:20:31.431957+0000","last_undegraded":"2024-04-15T15:20:31.431957+0000","last_fullsized":"2024-04-15T15:20:31.431957+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:26:24.401164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:30.648883+0000","last_change":"2024-04-15T15:20:28.661631+0000","last_active":"2024-04-15T15:20:30.648883+0000","last_peered":"2024-04-15T15:20:30.648883+0000","last_clean":"2024-04-15T15:20:30.648883+0000","last_became_active":"2024-04-15T15:20:28.661392+0000","last_became_peered":"2024-04-15T15:20:28.661392+0000","last_unstale":"2024-04-15T15:20:30.648883+0000","last_undegraded":"2024-04-15T15:20:30.648883+0000","last_fullsized":"2024-04-15T15:20:30.648883+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:40:43.115181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:31.431550+0000","last_change":"2024-04-15T15:20:29.661924+0000","last_active":"2024-04-15T15:20:31.431550+0000","last_peered":"2024-04-15T15:20:31.431550+0000","last_clean":"2024-04-15T15:20:31.431550+0000","last_became_active":"2024-04-15T15:20:29.661460+0000","last_became_peered":"2024-04-15T15:20:29.661460+0000","last_unstale":"2024-04-15T15:20:31.431550+0000","last_undegraded":"2024-04-15T15:20:31.431550+0000","last_fullsized":"2024-04-15T15:20:31.431550+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:00:53.362550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:31.431935+0000","last_change":"2024-04-15T15:20:28.661675+0000","last_active":"2024-04-15T15:20:31.431935+0000","last_peered":"2024-04-15T15:20:31.431935+0000","last_clean":"2024-04-15T15:20:31.431935+0000","last_became_active":"2024-04-15T15:20:28.661368+0000","last_became_peered":"2024-04-15T15:20:28.661368+0000","last_unstale":"2024-04-15T15:20:31.431935+0000","last_undegraded":"2024-04-15T15:20:31.431935+0000","last_fullsized":"2024-04-15T15:20:31.431935+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:10:59.194609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:31.385536+0000","last_change":"2024-04-15T15:20:29.671140+0000","last_active":"2024-04-15T15:20:31.385536+0000","last_peered":"2024-04-15T15:20:31.385536+0000","last_clean":"2024-04-15T15:20:31.385536+0000","last_became_active":"2024-04-15T15:20:29.670787+0000","last_became_peered":"2024-04-15T15:20:29.670787+0000","last_unstale":"2024-04-15T15:20:31.385536+0000","last_undegraded":"2024-04-15T15:20:31.385536+0000","last_fullsized":"2024-04-15T15:20:31.385536+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:15:37.566764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:30.649367+0000","last_change":"2024-04-15T15:20:28.665030+0000","last_active":"2024-04-15T15:20:30.649367+0000","last_peered":"2024-04-15T15:20:30.649367+0000","last_clean":"2024-04-15T15:20:30.649367+0000","last_became_active":"2024-04-15T15:20:28.664719+0000","last_became_peered":"2024-04-15T15:20:28.664719+0000","last_unstale":"2024-04-15T15:20:30.649367+0000","last_undegraded":"2024-04-15T15:20:30.649367+0000","last_fullsized":"2024-04-15T15:20:30.649367+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:00:52.360287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:30.649275+0000","last_change":"2024-04-15T15:20:29.664040+0000","last_active":"2024-04-15T15:20:30.649275+0000","last_peered":"2024-04-15T15:20:30.649275+0000","last_clean":"2024-04-15T15:20:30.649275+0000","last_became_active":"2024-04-15T15:20:29.663764+0000","last_became_peered":"2024-04-15T15:20:29.663764+0000","last_unstale":"2024-04-15T15:20:30.649275+0000","last_undegraded":"2024-04-15T15:20:30.649275+0000","last_fullsized":"2024-04-15T15:20:30.649275+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:23:50.286836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:31.431820+0000","last_change":"2024-04-15T15:20:28.662386+0000","last_active":"2024-04-15T15:20:31.431820+0000","last_peered":"2024-04-15T15:20:31.431820+0000","last_clean":"2024-04-15T15:20:31.431820+0000","last_became_active":"2024-04-15T15:20:28.662151+0000","last_became_peered":"2024-04-15T15:20:28.662151+0000","last_unstale":"2024-04-15T15:20:31.431820+0000","last_undegraded":"2024-04-15T15:20:31.431820+0000","last_fullsized":"2024-04-15T15:20:31.431820+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:53:08.913774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:30.649455+0000","last_change":"2024-04-15T15:20:29.660970+0000","last_active":"2024-04-15T15:20:30.649455+0000","last_peered":"2024-04-15T15:20:30.649455+0000","last_clean":"2024-04-15T15:20:30.649455+0000","last_became_active":"2024-04-15T15:20:29.660688+0000","last_became_peered":"2024-04-15T15:20:29.660688+0000","last_unstale":"2024-04-15T15:20:30.649455+0000","last_undegraded":"2024-04-15T15:20:30.649455+0000","last_fullsized":"2024-04-15T15:20:30.649455+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:06:40.939164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:31.385673+0000","last_change":"2024-04-15T15:20:28.662968+0000","last_active":"2024-04-15T15:20:31.385673+0000","last_peered":"2024-04-15T15:20:31.385673+0000","last_clean":"2024-04-15T15:20:31.385673+0000","last_became_active":"2024-04-15T15:20:28.662552+0000","last_became_peered":"2024-04-15T15:20:28.662552+0000","last_unstale":"2024-04-15T15:20:31.385673+0000","last_undegraded":"2024-04-15T15:20:31.385673+0000","last_fullsized":"2024-04-15T15:20:31.385673+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:40:43.115181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:31.385637+0000","last_change":"2024-04-15T15:20:29.676691+0000","last_active":"2024-04-15T15:20:31.385637+0000","last_peered":"2024-04-15T15:20:31.385637+0000","last_clean":"2024-04-15T15:20:31.385637+0000","last_became_active":"2024-04-15T15:20:29.676425+0000","last_became_peered":"2024-04-15T15:20:29.676425+0000","last_unstale":"2024-04-15T15:20:31.385637+0000","last_undegraded":"2024-04-15T15:20:31.385637+0000","last_fullsized":"2024-04-15T15:20:31.385637+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:59:38.044606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":20,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:30.649294+0000","last_change":"2024-04-15T15:20:28.663559+0000","last_active":"2024-04-15T15:20:30.649294+0000","last_peered":"2024-04-15T15:20:30.649294+0000","last_clean":"2024-04-15T15:20:30.649294+0000","last_became_active":"2024-04-15T15:20:28.662946+0000","last_became_peered":"2024-04-15T15:20:28.662946+0000","last_unstale":"2024-04-15T15:20:30.649294+0000","last_undegraded":"2024-04-15T15:20:30.649294+0000","last_fullsized":"2024-04-15T15:20:30.649294+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:51:42.408282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:31.431749+0000","last_change":"2024-04-15T15:20:29.671973+0000","last_active":"2024-04-15T15:20:31.431749+0000","last_peered":"2024-04-15T15:20:31.431749+0000","last_clean":"2024-04-15T15:20:31.431749+0000","last_became_active":"2024-04-15T15:20:29.671672+0000","last_became_peered":"2024-04-15T15:20:29.671672+0000","last_unstale":"2024-04-15T15:20:31.431749+0000","last_undegraded":"2024-04-15T15:20:31.431749+0000","last_fullsized":"2024-04-15T15:20:31.431749+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:41:17.064300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:31.432443+0000","last_change":"2024-04-15T15:20:28.662309+0000","last_active":"2024-04-15T15:20:31.432443+0000","last_peered":"2024-04-15T15:20:31.432443+0000","last_clean":"2024-04-15T15:20:31.432443+0000","last_became_active":"2024-04-15T15:20:28.661434+0000","last_became_peered":"2024-04-15T15:20:28.661434+0000","last_unstale":"2024-04-15T15:20:31.432443+0000","last_undegraded":"2024-04-15T15:20:31.432443+0000","last_fullsized":"2024-04-15T15:20:31.432443+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:54:49.252832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:30.649222+0000","last_change":"2024-04-15T15:20:29.672235+0000","last_active":"2024-04-15T15:20:30.649222+0000","last_peered":"2024-04-15T15:20:30.649222+0000","last_clean":"2024-04-15T15:20:30.649222+0000","last_became_active":"2024-04-15T15:20:29.671901+0000","last_became_peered":"2024-04-15T15:20:29.671901+0000","last_unstale":"2024-04-15T15:20:30.649222+0000","last_undegraded":"2024-04-15T15:20:30.649222+0000","last_fullsized":"2024-04-15T15:20:30.649222+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:59:38.044606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:31.431916+0000","last_change":"2024-04-15T15:20:28.665508+0000","last_active":"2024-04-15T15:20:31.431916+0000","last_peered":"2024-04-15T15:20:31.431916+0000","last_clean":"2024-04-15T15:20:31.431916+0000","last_became_active":"2024-04-15T15:20:28.665235+0000","last_became_peered":"2024-04-15T15:20:28.665235+0000","last_unstale":"2024-04-15T15:20:31.431916+0000","last_undegraded":"2024-04-15T15:20:31.431916+0000","last_fullsized":"2024-04-15T15:20:31.431916+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:16:18.672475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:30.650307+0000","last_change":"2024-04-15T15:20:29.672095+0000","last_active":"2024-04-15T15:20:30.650307+0000","last_peered":"2024-04-15T15:20:30.650307+0000","last_clean":"2024-04-15T15:20:30.650307+0000","last_became_active":"2024-04-15T15:20:29.671812+0000","last_became_peered":"2024-04-15T15:20:29.671812+0000","last_unstale":"2024-04-15T15:20:30.650307+0000","last_undegraded":"2024-04-15T15:20:30.650307+0000","last_fullsized":"2024-04-15T15:20:30.650307+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:15:37.566764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:30.648933+0000","last_change":"2024-04-15T15:20:28.665050+0000","last_active":"2024-04-15T15:20:30.648933+0000","last_peered":"2024-04-15T15:20:30.648933+0000","last_clean":"2024-04-15T15:20:30.648933+0000","last_became_active":"2024-04-15T15:20:28.664787+0000","last_became_peered":"2024-04-15T15:20:28.664787+0000","last_unstale":"2024-04-15T15:20:30.648933+0000","last_undegraded":"2024-04-15T15:20:30.648933+0000","last_fullsized":"2024-04-15T15:20:30.648933+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:26:23.398900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:31.432436+0000","last_change":"2024-04-15T15:20:29.654128+0000","last_active":"2024-04-15T15:20:31.432436+0000","last_peered":"2024-04-15T15:20:31.432436+0000","last_clean":"2024-04-15T15:20:31.432436+0000","last_became_active":"2024-04-15T15:20:29.653864+0000","last_became_peered":"2024-04-15T15:20:29.653864+0000","last_unstale":"2024-04-15T15:20:31.432436+0000","last_undegraded":"2024-04-15T15:20:31.432436+0000","last_fullsized":"2024-04-15T15:20:31.432436+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:22:05.003943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":22,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:30.648904+0000","last_change":"2024-04-15T15:20:28.665337+0000","last_active":"2024-04-15T15:20:30.648904+0000","last_peered":"2024-04-15T15:20:30.648904+0000","last_clean":"2024-04-15T15:20:30.648904+0000","last_became_active":"2024-04-15T15:20:28.664441+0000","last_became_peered":"2024-04-15T15:20:28.664441+0000","last_unstale":"2024-04-15T15:20:30.648904+0000","last_undegraded":"2024-04-15T15:20:30.648904+0000","last_fullsized":"2024-04-15T15:20:30.648904+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:03:20.998519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:31.432459+0000","last_change":"2024-04-15T15:20:29.665872+0000","last_active":"2024-04-15T15:20:31.432459+0000","last_peered":"2024-04-15T15:20:31.432459+0000","last_clean":"2024-04-15T15:20:31.432459+0000","last_became_active":"2024-04-15T15:20:29.665544+0000","last_became_peered":"2024-04-15T15:20:29.665544+0000","last_unstale":"2024-04-15T15:20:31.432459+0000","last_undegraded":"2024-04-15T15:20:31.432459+0000","last_fullsized":"2024-04-15T15:20:31.432459+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:51:43.410546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:31.432385+0000","last_change":"2024-04-15T15:20:28.666179+0000","last_active":"2024-04-15T15:20:31.432385+0000","last_peered":"2024-04-15T15:20:31.432385+0000","last_clean":"2024-04-15T15:20:31.432385+0000","last_became_active":"2024-04-15T15:20:28.665927+0000","last_became_peered":"2024-04-15T15:20:28.665927+0000","last_unstale":"2024-04-15T15:20:31.432385+0000","last_undegraded":"2024-04-15T15:20:31.432385+0000","last_fullsized":"2024-04-15T15:20:31.432385+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:10:52.680906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:31.385696+0000","last_change":"2024-04-15T15:20:29.672651+0000","last_active":"2024-04-15T15:20:31.385696+0000","last_peered":"2024-04-15T15:20:31.385696+0000","last_clean":"2024-04-15T15:20:31.385696+0000","last_became_active":"2024-04-15T15:20:29.672406+0000","last_became_peered":"2024-04-15T15:20:29.672406+0000","last_unstale":"2024-04-15T15:20:31.385696+0000","last_undegraded":"2024-04-15T15:20:31.385696+0000","last_fullsized":"2024-04-15T15:20:31.385696+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:38:58.777641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:31.432418+0000","last_change":"2024-04-15T15:20:28.661761+0000","last_active":"2024-04-15T15:20:31.432418+0000","last_peered":"2024-04-15T15:20:31.432418+0000","last_clean":"2024-04-15T15:20:31.432418+0000","last_became_active":"2024-04-15T15:20:28.661249+0000","last_became_peered":"2024-04-15T15:20:28.661249+0000","last_unstale":"2024-04-15T15:20:31.432418+0000","last_undegraded":"2024-04-15T15:20:31.432418+0000","last_fullsized":"2024-04-15T15:20:31.432418+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:44:15.832156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:31.432161+0000","last_change":"2024-04-15T15:20:29.675318+0000","last_active":"2024-04-15T15:20:31.432161+0000","last_peered":"2024-04-15T15:20:31.432161+0000","last_clean":"2024-04-15T15:20:31.432161+0000","last_became_active":"2024-04-15T15:20:29.675063+0000","last_became_peered":"2024-04-15T15:20:29.675063+0000","last_unstale":"2024-04-15T15:20:31.432161+0000","last_undegraded":"2024-04-15T15:20:31.432161+0000","last_fullsized":"2024-04-15T15:20:31.432161+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:28:35.340051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:31.432847+0000","last_change":"2024-04-15T15:20:28.657992+0000","last_active":"2024-04-15T15:20:31.432847+0000","last_peered":"2024-04-15T15:20:31.432847+0000","last_clean":"2024-04-15T15:20:31.432847+0000","last_became_active":"2024-04-15T15:20:28.657669+0000","last_became_peered":"2024-04-15T15:20:28.657669+0000","last_unstale":"2024-04-15T15:20:31.432847+0000","last_undegraded":"2024-04-15T15:20:31.432847+0000","last_fullsized":"2024-04-15T15:20:31.432847+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:40:32.394676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:30.650196+0000","last_change":"2024-04-15T15:20:29.664787+0000","last_active":"2024-04-15T15:20:30.650196+0000","last_peered":"2024-04-15T15:20:30.650196+0000","last_clean":"2024-04-15T15:20:30.650196+0000","last_became_active":"2024-04-15T15:20:29.664514+0000","last_became_peered":"2024-04-15T15:20:29.664514+0000","last_unstale":"2024-04-15T15:20:30.650196+0000","last_undegraded":"2024-04-15T15:20:30.650196+0000","last_fullsized":"2024-04-15T15:20:30.650196+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:53:48.544816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:31.432712+0000","last_change":"2024-04-15T15:20:28.662320+0000","last_active":"2024-04-15T15:20:31.432712+0000","last_peered":"2024-04-15T15:20:31.432712+0000","last_clean":"2024-04-15T15:20:31.432712+0000","last_became_active":"2024-04-15T15:20:28.662030+0000","last_became_peered":"2024-04-15T15:20:28.662030+0000","last_unstale":"2024-04-15T15:20:31.432712+0000","last_undegraded":"2024-04-15T15:20:31.432712+0000","last_fullsized":"2024-04-15T15:20:31.432712+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:08:28.127181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:30.649957+0000","last_change":"2024-04-15T15:20:29.660216+0000","last_active":"2024-04-15T15:20:30.649957+0000","last_peered":"2024-04-15T15:20:30.649957+0000","last_clean":"2024-04-15T15:20:30.649957+0000","last_became_active":"2024-04-15T15:20:29.659902+0000","last_became_peered":"2024-04-15T15:20:29.659902+0000","last_unstale":"2024-04-15T15:20:30.649957+0000","last_undegraded":"2024-04-15T15:20:30.649957+0000","last_fullsized":"2024-04-15T15:20:30.649957+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:32:33.934318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:31.431750+0000","last_change":"2024-04-15T15:20:28.661886+0000","last_active":"2024-04-15T15:20:31.431750+0000","last_peered":"2024-04-15T15:20:31.431750+0000","last_clean":"2024-04-15T15:20:31.431750+0000","last_became_active":"2024-04-15T15:20:28.661595+0000","last_became_peered":"2024-04-15T15:20:28.661595+0000","last_unstale":"2024-04-15T15:20:31.431750+0000","last_undegraded":"2024-04-15T15:20:31.431750+0000","last_fullsized":"2024-04-15T15:20:31.431750+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:53:47.542553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:30.650192+0000","last_change":"2024-04-15T15:20:29.669349+0000","last_active":"2024-04-15T15:20:30.650192+0000","last_peered":"2024-04-15T15:20:30.650192+0000","last_clean":"2024-04-15T15:20:30.650192+0000","last_became_active":"2024-04-15T15:20:29.669026+0000","last_became_peered":"2024-04-15T15:20:29.669026+0000","last_unstale":"2024-04-15T15:20:30.650192+0000","last_undegraded":"2024-04-15T15:20:30.650192+0000","last_fullsized":"2024-04-15T15:20:30.650192+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:47:15.252589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:31.432102+0000","last_change":"2024-04-15T15:20:28.660511+0000","last_active":"2024-04-15T15:20:31.432102+0000","last_peered":"2024-04-15T15:20:31.432102+0000","last_clean":"2024-04-15T15:20:31.432102+0000","last_became_active":"2024-04-15T15:20:28.660182+0000","last_became_peered":"2024-04-15T15:20:28.660182+0000","last_unstale":"2024-04-15T15:20:31.432102+0000","last_undegraded":"2024-04-15T15:20:31.432102+0000","last_fullsized":"2024-04-15T15:20:31.432102+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:53:08.913774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:30.649869+0000","last_change":"2024-04-15T15:20:29.667145+0000","last_active":"2024-04-15T15:20:30.649869+0000","last_peered":"2024-04-15T15:20:30.649869+0000","last_clean":"2024-04-15T15:20:30.649869+0000","last_became_active":"2024-04-15T15:20:29.666884+0000","last_became_peered":"2024-04-15T15:20:29.666884+0000","last_unstale":"2024-04-15T15:20:30.649869+0000","last_undegraded":"2024-04-15T15:20:30.649869+0000","last_fullsized":"2024-04-15T15:20:30.649869+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:54:50.255095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:31.432558+0000","last_change":"2024-04-15T15:20:28.659967+0000","last_active":"2024-04-15T15:20:31.432558+0000","last_peered":"2024-04-15T15:20:31.432558+0000","last_clean":"2024-04-15T15:20:31.432558+0000","last_became_active":"2024-04-15T15:20:28.659679+0000","last_became_peered":"2024-04-15T15:20:28.659679+0000","last_unstale":"2024-04-15T15:20:31.432558+0000","last_undegraded":"2024-04-15T15:20:31.432558+0000","last_fullsized":"2024-04-15T15:20:31.432558+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:11:11.096685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:30.649828+0000","last_change":"2024-04-15T15:20:29.661001+0000","last_active":"2024-04-15T15:20:30.649828+0000","last_peered":"2024-04-15T15:20:30.649828+0000","last_clean":"2024-04-15T15:20:30.649828+0000","last_became_active":"2024-04-15T15:20:29.660567+0000","last_became_peered":"2024-04-15T15:20:29.660567+0000","last_unstale":"2024-04-15T15:20:30.649828+0000","last_undegraded":"2024-04-15T15:20:30.649828+0000","last_fullsized":"2024-04-15T15:20:30.649828+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:34:43.134380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:31.385113+0000","last_change":"2024-04-15T15:20:28.658381+0000","last_active":"2024-04-15T15:20:31.385113+0000","last_peered":"2024-04-15T15:20:31.385113+0000","last_clean":"2024-04-15T15:20:31.385113+0000","last_became_active":"2024-04-15T15:20:28.658066+0000","last_became_peered":"2024-04-15T15:20:28.658066+0000","last_unstale":"2024-04-15T15:20:31.385113+0000","last_undegraded":"2024-04-15T15:20:31.385113+0000","last_fullsized":"2024-04-15T15:20:31.385113+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:26:23.398900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:30.650210+0000","last_change":"2024-04-15T15:20:29.665377+0000","last_active":"2024-04-15T15:20:30.650210+0000","last_peered":"2024-04-15T15:20:30.650210+0000","last_clean":"2024-04-15T15:20:30.650210+0000","last_became_active":"2024-04-15T15:20:29.665129+0000","last_became_peered":"2024-04-15T15:20:29.665129+0000","last_unstale":"2024-04-15T15:20:30.650210+0000","last_undegraded":"2024-04-15T15:20:30.650210+0000","last_fullsized":"2024-04-15T15:20:30.650210+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:40:46.760520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:31.432436+0000","last_change":"2024-04-15T15:20:28.663868+0000","last_active":"2024-04-15T15:20:31.432436+0000","last_peered":"2024-04-15T15:20:31.432436+0000","last_clean":"2024-04-15T15:20:31.432436+0000","last_became_active":"2024-04-15T15:20:28.663619+0000","last_became_peered":"2024-04-15T15:20:28.663619+0000","last_unstale":"2024-04-15T15:20:31.432436+0000","last_undegraded":"2024-04-15T15:20:31.432436+0000","last_fullsized":"2024-04-15T15:20:31.432436+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:53:47.542553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:31.385687+0000","last_change":"2024-04-15T15:20:29.677317+0000","last_active":"2024-04-15T15:20:31.385687+0000","last_peered":"2024-04-15T15:20:31.385687+0000","last_clean":"2024-04-15T15:20:31.385687+0000","last_became_active":"2024-04-15T15:20:29.677128+0000","last_became_peered":"2024-04-15T15:20:29.677128+0000","last_unstale":"2024-04-15T15:20:31.385687+0000","last_undegraded":"2024-04-15T15:20:31.385687+0000","last_fullsized":"2024-04-15T15:20:31.385687+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:23:50.286836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"34'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:31.431935+0000","last_change":"2024-04-15T15:20:28.662929+0000","last_active":"2024-04-15T15:20:31.431935+0000","last_peered":"2024-04-15T15:20:31.431935+0000","last_clean":"2024-04-15T15:20:31.431935+0000","last_became_active":"2024-04-15T15:20:28.662666+0000","last_became_peered":"2024-04-15T15:20:28.662666+0000","last_unstale":"2024-04-15T15:20:31.431935+0000","last_undegraded":"2024-04-15T15:20:31.431935+0000","last_fullsized":"2024-04-15T15:20:31.431935+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:10:52.680906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:31.431593+0000","last_change":"2024-04-15T15:20:29.676760+0000","last_active":"2024-04-15T15:20:31.431593+0000","last_peered":"2024-04-15T15:20:31.431593+0000","last_clean":"2024-04-15T15:20:31.431593+0000","last_became_active":"2024-04-15T15:20:29.676486+0000","last_became_peered":"2024-04-15T15:20:29.676486+0000","last_unstale":"2024-04-15T15:20:31.431593+0000","last_undegraded":"2024-04-15T15:20:31.431593+0000","last_fullsized":"2024-04-15T15:20:31.431593+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:22:05.003943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:31.432174+0000","last_change":"2024-04-15T15:20:28.660911+0000","last_active":"2024-04-15T15:20:31.432174+0000","last_peered":"2024-04-15T15:20:31.432174+0000","last_clean":"2024-04-15T15:20:31.432174+0000","last_became_active":"2024-04-15T15:20:28.660620+0000","last_became_peered":"2024-04-15T15:20:28.660620+0000","last_unstale":"2024-04-15T15:20:31.432174+0000","last_undegraded":"2024-04-15T15:20:31.432174+0000","last_fullsized":"2024-04-15T15:20:31.432174+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:53:08.913774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:31.432120+0000","last_change":"2024-04-15T15:20:29.667405+0000","last_active":"2024-04-15T15:20:31.432120+0000","last_peered":"2024-04-15T15:20:31.432120+0000","last_clean":"2024-04-15T15:20:31.432120+0000","last_became_active":"2024-04-15T15:20:29.667157+0000","last_became_peered":"2024-04-15T15:20:29.667157+0000","last_unstale":"2024-04-15T15:20:31.432120+0000","last_undegraded":"2024-04-15T15:20:31.432120+0000","last_fullsized":"2024-04-15T15:20:31.432120+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:26:24.401164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:31.431850+0000","last_change":"2024-04-15T15:20:28.659430+0000","last_active":"2024-04-15T15:20:31.431850+0000","last_peered":"2024-04-15T15:20:31.431850+0000","last_clean":"2024-04-15T15:20:31.431850+0000","last_became_active":"2024-04-15T15:20:28.659182+0000","last_became_peered":"2024-04-15T15:20:28.659182+0000","last_unstale":"2024-04-15T15:20:31.431850+0000","last_undegraded":"2024-04-15T15:20:31.431850+0000","last_fullsized":"2024-04-15T15:20:31.431850+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:40:45.758256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:31.431702+0000","last_change":"2024-04-15T15:20:29.672794+0000","last_active":"2024-04-15T15:20:31.431702+0000","last_peered":"2024-04-15T15:20:31.431702+0000","last_clean":"2024-04-15T15:20:31.431702+0000","last_became_active":"2024-04-15T15:20:29.672491+0000","last_became_peered":"2024-04-15T15:20:29.672491+0000","last_unstale":"2024-04-15T15:20:31.431702+0000","last_undegraded":"2024-04-15T15:20:31.431702+0000","last_fullsized":"2024-04-15T15:20:31.431702+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:00:53.362550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:31.432040+0000","last_change":"2024-04-15T15:20:28.664511+0000","last_active":"2024-04-15T15:20:31.432040+0000","last_peered":"2024-04-15T15:20:31.432040+0000","last_clean":"2024-04-15T15:20:31.432040+0000","last_became_active":"2024-04-15T15:20:28.664269+0000","last_became_peered":"2024-04-15T15:20:28.664269+0000","last_unstale":"2024-04-15T15:20:31.432040+0000","last_undegraded":"2024-04-15T15:20:31.432040+0000","last_fullsized":"2024-04-15T15:20:31.432040+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:32:32.932055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:30.648899+0000","last_change":"2024-04-15T15:20:29.661909+0000","last_active":"2024-04-15T15:20:30.648899+0000","last_peered":"2024-04-15T15:20:30.648899+0000","last_clean":"2024-04-15T15:20:30.648899+0000","last_became_active":"2024-04-15T15:20:29.661374+0000","last_became_peered":"2024-04-15T15:20:29.661374+0000","last_unstale":"2024-04-15T15:20:30.648899+0000","last_undegraded":"2024-04-15T15:20:30.648899+0000","last_fullsized":"2024-04-15T15:20:30.648899+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:09:06.486051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:31.432696+0000","last_change":"2024-04-15T15:20:28.658052+0000","last_active":"2024-04-15T15:20:31.432696+0000","last_peered":"2024-04-15T15:20:31.432696+0000","last_clean":"2024-04-15T15:20:31.432696+0000","last_became_active":"2024-04-15T15:20:28.657686+0000","last_became_peered":"2024-04-15T15:20:28.657686+0000","last_unstale":"2024-04-15T15:20:31.432696+0000","last_undegraded":"2024-04-15T15:20:31.432696+0000","last_fullsized":"2024-04-15T15:20:31.432696+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T03:16:13.044979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:31.432562+0000","last_change":"2024-04-15T15:20:29.657918+0000","last_active":"2024-04-15T15:20:31.432562+0000","last_peered":"2024-04-15T15:20:31.432562+0000","last_clean":"2024-04-15T15:20:31.432562+0000","last_became_active":"2024-04-15T15:20:29.657612+0000","last_became_peered":"2024-04-15T15:20:29.657612+0000","last_unstale":"2024-04-15T15:20:31.432562+0000","last_undegraded":"2024-04-15T15:20:31.432562+0000","last_fullsized":"2024-04-15T15:20:31.432562+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:00:53.362550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:30.648998+0000","last_change":"2024-04-15T15:20:28.665696+0000","last_active":"2024-04-15T15:20:30.648998+0000","last_peered":"2024-04-15T15:20:30.648998+0000","last_clean":"2024-04-15T15:20:30.648998+0000","last_became_active":"2024-04-15T15:20:28.665180+0000","last_became_peered":"2024-04-15T15:20:28.665180+0000","last_unstale":"2024-04-15T15:20:30.648998+0000","last_undegraded":"2024-04-15T15:20:30.648998+0000","last_fullsized":"2024-04-15T15:20:30.648998+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:13:56.700199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:30.650012+0000","last_change":"2024-04-15T15:20:29.668726+0000","last_active":"2024-04-15T15:20:30.650012+0000","last_peered":"2024-04-15T15:20:30.650012+0000","last_clean":"2024-04-15T15:20:30.650012+0000","last_became_active":"2024-04-15T15:20:29.668461+0000","last_became_peered":"2024-04-15T15:20:29.668461+0000","last_unstale":"2024-04-15T15:20:30.650012+0000","last_undegraded":"2024-04-15T15:20:30.650012+0000","last_fullsized":"2024-04-15T15:20:30.650012+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:28.638297+0000","last_clean_scrub_stamp":"2024-04-15T15:20:28.638297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:32:25.149403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:31.431983+0000","last_change":"2024-04-15T15:20:28.659582+0000","last_active":"2024-04-15T15:20:31.431983+0000","last_peered":"2024-04-15T15:20:31.431983+0000","last_clean":"2024-04-15T15:20:31.431983+0000","last_became_active":"2024-04-15T15:20:28.659266+0000","last_became_peered":"2024-04-15T15:20:28.659266+0000","last_unstale":"2024-04-15T15:20:31.431983+0000","last_undegraded":"2024-04-15T15:20:31.431983+0000","last_fullsized":"2024-04-15T15:20:31.431983+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:20:27.636034+0000","last_clean_scrub_stamp":"2024-04-15T15:20:27.636034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:32:32.932055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":197,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:20:30.650438+0000","last_change":"2024-04-15T15:19:14.878269+0000","last_active":"2024-04-15T15:20:30.650438+0000","last_peered":"2024-04-15T15:20:30.650438+0000","last_clean":"2024-04-15T15:20:30.650438+0000","last_became_active":"2024-04-15T15:19:14.877367+0000","last_became_peered":"2024-04-15T15:19:14.877367+0000","last_unstale":"2024-04-15T15:20:30.650438+0000","last_undegraded":"2024-04-15T15:20:30.650438+0000","last_fullsized":"2024-04-15T15:20:30.650438+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:13.827511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:13.827511+0000","last_clean_scrub_stamp":"2024-04-15T15:19:13.827511+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:32:55.902561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27988,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343852,"statfs":{"total":96636764160,"available":96608104448,"internally_reserved":0,"allocated":983040,"data_stored":742381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":393216,"data_stored":152539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":393216,"data_stored":152595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":393216,"data_stored":152027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":401408,"data_stored":153557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":389120,"data_stored":151993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":401408,"data_stored":153557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343864,"statfs":{"total":96636764160,"available":96608116736,"internally_reserved":0,"allocated":978944,"data_stored":742291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-15T15:20:37.380 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-15T15:20:37.380 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-15T15:20:37.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.525+0000 7f6085382640 1 Processor -- start 2024-04-15T15:20:37.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.525+0000 7f6085382640 1 -- start start 2024-04-15T15:20:37.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.526+0000 7f6085382640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f60800d1b60 0x7f60800734b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:37.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.526+0000 7f6085382640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f608006c900 con 0x7f60800d2030 2024-04-15T15:20:37.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.526+0000 7f6085382640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6080146370 con 0x7f6080145f90 2024-04-15T15:20:37.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.526+0000 7f6085382640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f608006c3d0 con 0x7f60800d1b60 2024-04-15T15:20:37.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.526+0000 7f607f7fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f60800d2030 0x7f60800ceac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:43742/0 (socket says 172.21.15.105:43742) 2024-04-15T15:20:37.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.526+0000 7f607f7fe640 1 -- 172.21.15.105:0/1976418267 learned_addr learned my addr 172.21.15.105:0/1976418267 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:20:37.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.526+0000 7f607e7fc640 1 --2- 172.21.15.105:0/1976418267 >> v2:172.21.15.154:3300/0 conn(0x7f60800d1b60 0x7f60800734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:37.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.526+0000 7f607dffb640 1 -- 172.21.15.105:0/1976418267 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3045597571 0 0) 0x7f608006c900 con 0x7f60800d2030 2024-04-15T15:20:37.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.527+0000 7f607dffb640 1 -- 172.21.15.105:0/1976418267 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6068003520 con 0x7f60800d2030 2024-04-15T15:20:37.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.527+0000 7f607e7fc640 1 -- 172.21.15.105:0/1976418267 >> v1:172.21.15.154:6789/0 conn(0x7f6080145f90 legacy=0x7f60800d1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:37.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.527+0000 7f607e7fc640 1 -- 172.21.15.105:0/1976418267 >> v1:172.21.15.105:6789/0 conn(0x7f60800d2030 legacy=0x7f60800ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:37.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.527+0000 7f607e7fc640 1 -- 172.21.15.105:0/1976418267 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f608006c620 con 0x7f60800d1b60 2024-04-15T15:20:37.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.528+0000 7f607e7fc640 1 --2- 172.21.15.105:0/1976418267 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60800d1b60 0x7f60800734b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f60740097a0 tx=0x7f607402b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df366fe961044b72 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:37.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.528+0000 7f607dffb640 1 -- 172.21.15.105:0/1976418267 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6074034070 con 0x7f60800d1b60 2024-04-15T15:20:37.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.528+0000 7f607dffb640 1 -- 172.21.15.105:0/1976418267 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6074030610 con 0x7f60800d1b60 2024-04-15T15:20:37.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.528+0000 7f607dffb640 1 -- 172.21.15.105:0/1976418267 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60740309e0 con 0x7f60800d1b60 2024-04-15T15:20:37.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.528+0000 7f6085382640 1 -- 172.21.15.105:0/1976418267 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60800d1b60 msgr2=0x7f60800734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:37.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.528+0000 7f6085382640 1 --2- 172.21.15.105:0/1976418267 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60800d1b60 0x7f60800734b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f60740097a0 tx=0x7f607402b960 comp rx=0 tx=0).stop 2024-04-15T15:20:37.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.528+0000 7f6085382640 1 -- 172.21.15.105:0/1976418267 shutdown_connections 2024-04-15T15:20:37.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.528+0000 7f6085382640 1 --2- 172.21.15.105:0/1976418267 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60800d1b60 0x7f60800734b0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:37.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.528+0000 7f6085382640 1 -- 172.21.15.105:0/1976418267 >> 172.21.15.105:0/1976418267 conn(0x7f60800b6b10 msgr2=0x7f608009aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:37.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.528+0000 7f6085382640 1 -- 172.21.15.105:0/1976418267 shutdown_connections 2024-04-15T15:20:37.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.528+0000 7f6085382640 1 -- 172.21.15.105:0/1976418267 wait complete. 2024-04-15T15:20:37.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.530+0000 7f6085382640 1 Processor -- start 2024-04-15T15:20:37.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.530+0000 7f6085382640 1 -- start start 2024-04-15T15:20:37.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.530+0000 7f6085382640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f60800d1b60 0x7f6080159060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:37.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.530+0000 7f6085382640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f60800d2030 0x7f6080135d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:37.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.530+0000 7f6085382640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6080145f90 0x7f60801362c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:37.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.530+0000 7f6085382640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f60800ce5a0 con 0x7f60800d2030 2024-04-15T15:20:37.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.530+0000 7f6085382640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f60800ce2c0 con 0x7f6080145f90 2024-04-15T15:20:37.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.530+0000 7f6085382640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f60800cf1d0 con 0x7f60800d1b60 2024-04-15T15:20:37.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.530+0000 7f607e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f60800d2030 0x7f6080135d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:37.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.530+0000 7f607e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f60800d2030 0x7f6080135d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45362/0 (socket says 172.21.15.105:45362) 2024-04-15T15:20:37.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.530+0000 7f607e7fc640 1 -- 172.21.15.105:0/3973295070 learned_addr learned my addr 172.21.15.105:0/3973295070 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:37.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.531+0000 7f607f7fe640 1 --2- 172.21.15.105:0/3973295070 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6080145f90 0x7f60801362c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:37.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.531+0000 7f607effd640 1 --2- 172.21.15.105:0/3973295070 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f60800d1b60 0x7f6080159060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:37.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.531+0000 7f607e7fc640 1 -- 172.21.15.105:0/3973295070 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f60800d1b60 msgr2=0x7f6080159060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:37.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.531+0000 7f607e7fc640 1 --2- 172.21.15.105:0/3973295070 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f60800d1b60 0x7f6080159060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:37.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.531+0000 7f607e7fc640 1 -- 172.21.15.105:0/3973295070 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6080145f90 msgr2=0x7f60801362c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:37.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.531+0000 7f607e7fc640 1 --2- 172.21.15.105:0/3973295070 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6080145f90 0x7f60801362c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:37.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.531+0000 7f607e7fc640 1 -- 172.21.15.105:0/3973295070 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6080144320 con 0x7f60800d2030 2024-04-15T15:20:37.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.531+0000 7f607effd640 1 --2- 172.21.15.105:0/3973295070 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f60800d1b60 0x7f6080159060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:37.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.531+0000 7f607f7fe640 1 --2- 172.21.15.105:0/3973295070 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6080145f90 0x7f60801362c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:37.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.531+0000 7f607e7fc640 1 --2- 172.21.15.105:0/3973295070 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f60800d2030 0x7f6080135d80 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f60740098c0 tx=0x7f60740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:37.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.531+0000 7f60677fe640 1 -- 172.21.15.105:0/3973295070 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6074034040 con 0x7f60800d2030 2024-04-15T15:20:37.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.531+0000 7f60677fe640 1 -- 172.21.15.105:0/3973295070 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6074030410 con 0x7f60800d2030 2024-04-15T15:20:37.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.531+0000 7f6085382640 1 -- 172.21.15.105:0/3973295070 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6080144600 con 0x7f60800d2030 2024-04-15T15:20:37.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.531+0000 7f60677fe640 1 -- 172.21.15.105:0/3973295070 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6074039c50 con 0x7f60800d2030 2024-04-15T15:20:37.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.531+0000 7f6085382640 1 -- 172.21.15.105:0/3973295070 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60801595a0 con 0x7f60800d2030 2024-04-15T15:20:37.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.532+0000 7f6085382640 1 -- 172.21.15.105:0/3973295070 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6080068ab0 con 0x7f60800d2030 2024-04-15T15:20:37.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.533+0000 7f60677fe640 1 -- 172.21.15.105:0/3973295070 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6074037070 con 0x7f60800d2030 2024-04-15T15:20:37.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.533+0000 7f60677fe640 1 --2- 172.21.15.105:0/3973295070 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f606c06c8f0 0x7f606c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:37.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.533+0000 7f607effd640 1 --2- 172.21.15.105:0/3973295070 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f606c06c8f0 0x7f606c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:37.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.533+0000 7f60677fe640 1 -- 172.21.15.105:0/3973295070 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f60740ada80 con 0x7f60800d2030 2024-04-15T15:20:37.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.534+0000 7f607effd640 1 --2- 172.21.15.105:0/3973295070 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f606c06c8f0 0x7f606c06edb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6080145710 tx=0x7f6070009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:37.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.538+0000 7f60677fe640 1 -- 172.21.15.105:0/3973295070 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6074038e90 con 0x7f60800d2030 2024-04-15T15:20:37.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.783+0000 7f6085382640 1 -- 172.21.15.105:0/3973295070 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f60800d1230 con 0x7f60800d2030 2024-04-15T15:20:37.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.786+0000 7f60677fe640 1 -- 172.21.15.105:0/3973295070 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) ==== 190+0+147 (secure 0 0 0) 0x7f607407bf40 con 0x7f60800d2030 2024-04-15T15:20:37.788 INFO:teuthology.orchestra.run.smithi105.stdout:[client.0] 2024-04-15T15:20:37.788 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQBERR1mNnrzBBAAg6vNvYozUQe3sI+Xxt0QvA== 2024-04-15T15:20:37.788 INFO:teuthology.orchestra.run.smithi105.stdout: caps mds = "allow" 2024-04-15T15:20:37.788 INFO:teuthology.orchestra.run.smithi105.stdout: caps mon = "allow r" 2024-04-15T15:20:37.788 INFO:teuthology.orchestra.run.smithi105.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-15T15:20:37.788 INFO:teuthology.orchestra.run.smithi105.stderr:updated caps for client.0 2024-04-15T15:20:37.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.788+0000 7f6085382640 1 -- 172.21.15.105:0/3973295070 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f606c06c8f0 msgr2=0x7f606c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:37.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.788+0000 7f6085382640 1 --2- 172.21.15.105:0/3973295070 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f606c06c8f0 0x7f606c06edb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6080145710 tx=0x7f6070009040 comp rx=0 tx=0).stop 2024-04-15T15:20:37.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.788+0000 7f6085382640 1 -- 172.21.15.105:0/3973295070 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f60800d2030 msgr2=0x7f6080135d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:37.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.788+0000 7f6085382640 1 --2- 172.21.15.105:0/3973295070 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f60800d2030 0x7f6080135d80 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f60740098c0 tx=0x7f60740046c0 comp rx=0 tx=0).stop 2024-04-15T15:20:37.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.788+0000 7f6085382640 1 -- 172.21.15.105:0/3973295070 shutdown_connections 2024-04-15T15:20:37.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.788+0000 7f6085382640 1 --2- 172.21.15.105:0/3973295070 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f606c06c8f0 0x7f606c06edb0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:37.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.788+0000 7f6085382640 1 --2- 172.21.15.105:0/3973295070 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6080145f90 0x7f60801362c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:37.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.788+0000 7f6085382640 1 --2- 172.21.15.105:0/3973295070 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f60800d2030 0x7f6080135d80 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:37.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.788+0000 7f6085382640 1 --2- 172.21.15.105:0/3973295070 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f60800d1b60 0x7f6080159060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:37.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.788+0000 7f6085382640 1 -- 172.21.15.105:0/3973295070 >> 172.21.15.105:0/3973295070 conn(0x7f60800b6b10 msgr2=0x7f6080070360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:37.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.788+0000 7f6085382640 1 -- 172.21.15.105:0/3973295070 shutdown_connections 2024-04-15T15:20:37.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.788+0000 7f6085382640 1 -- 172.21.15.105:0/3973295070 wait complete. 2024-04-15T15:20:37.801 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-15T15:20:37.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.901+0000 7f3e80c2a640 1 Processor -- start 2024-04-15T15:20:37.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.901+0000 7f3e80c2a640 1 -- start start 2024-04-15T15:20:37.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.902+0000 7f3e80c2a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3e7c0d18e0 0x7f3e7c162630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:37.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.902+0000 7f3e80c2a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3e7c162b70 0x7f3e7c0d3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:37.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.902+0000 7f3e80c2a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e7c15ef40 con 0x7f3e7c15eb60 2024-04-15T15:20:37.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.902+0000 7f3e80c2a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f3e7c163540 con 0x7f3e7c0d18e0 2024-04-15T15:20:37.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.902+0000 7f3e80c2a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f3e7c0d2fc0 con 0x7f3e7c162b70 2024-04-15T15:20:37.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.902+0000 7f3e73fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3e7c0d18e0 0x7f3e7c162630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:37.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.902+0000 7f3e7a59c640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f3e7c15eb60 0x7f3e7c0d11d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:43746/0 (socket says 172.21.15.105:43746) 2024-04-15T15:20:37.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.902+0000 7f3e7a59c640 1 -- 172.21.15.105:0/583926753 learned_addr learned my addr 172.21.15.105:0/583926753 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:20:37.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.903+0000 7f3e7ad9d640 1 --2- 172.21.15.105:0/583926753 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3e7c162b70 0x7f3e7c0d3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:37.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.903+0000 7f3e73fff640 1 -- 172.21.15.105:0/583926753 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3e7c162b70 msgr2=0x7f3e7c0d3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:37.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.903+0000 7f3e73fff640 1 --2- 172.21.15.105:0/583926753 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3e7c162b70 0x7f3e7c0d3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:37.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.903+0000 7f3e73fff640 1 -- 172.21.15.105:0/583926753 >> v1:172.21.15.105:6789/0 conn(0x7f3e7c15eb60 legacy=0x7f3e7c0d11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:37.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.903+0000 7f3e73fff640 1 -- 172.21.15.105:0/583926753 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e7c163260 con 0x7f3e7c0d18e0 2024-04-15T15:20:37.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.903+0000 7f3e79d9b640 1 -- 172.21.15.105:0/583926753 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1210681407 0 0) 0x7f3e7c15ef40 con 0x7f3e7c15eb60 2024-04-15T15:20:37.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.903+0000 7f3e73fff640 1 --2- 172.21.15.105:0/583926753 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e7c0d18e0 0x7f3e7c162630 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f3e640097a0 tx=0x7f3e6402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a56ab4a2eb4a56fc server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:37.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.904+0000 7f3e79d9b640 1 -- 172.21.15.105:0/583926753 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e64035020 con 0x7f3e7c0d18e0 2024-04-15T15:20:37.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.904+0000 7f3e79d9b640 1 -- 172.21.15.105:0/583926753 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e64031830 con 0x7f3e7c0d18e0 2024-04-15T15:20:37.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.904+0000 7f3e79d9b640 1 -- 172.21.15.105:0/583926753 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e64031c00 con 0x7f3e7c0d18e0 2024-04-15T15:20:37.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.904+0000 7f3e80c2a640 1 -- 172.21.15.105:0/583926753 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e7c0d18e0 msgr2=0x7f3e7c162630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:37.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.904+0000 7f3e80c2a640 1 --2- 172.21.15.105:0/583926753 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e7c0d18e0 0x7f3e7c162630 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f3e640097a0 tx=0x7f3e6402b8d0 comp rx=0 tx=0).stop 2024-04-15T15:20:37.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.904+0000 7f3e80c2a640 1 -- 172.21.15.105:0/583926753 shutdown_connections 2024-04-15T15:20:37.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.904+0000 7f3e80c2a640 1 --2- 172.21.15.105:0/583926753 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3e7c162b70 0x7f3e7c0d3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:37.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.904+0000 7f3e80c2a640 1 --2- 172.21.15.105:0/583926753 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e7c0d18e0 0x7f3e7c162630 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:37.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.904+0000 7f3e80c2a640 1 -- 172.21.15.105:0/583926753 >> 172.21.15.105:0/583926753 conn(0x7f3e7c067720 msgr2=0x7f3e7c067b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:37.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.904+0000 7f3e80c2a640 1 -- 172.21.15.105:0/583926753 shutdown_connections 2024-04-15T15:20:37.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.904+0000 7f3e80c2a640 1 -- 172.21.15.105:0/583926753 wait complete. 2024-04-15T15:20:37.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.906+0000 7f3e80c2a640 1 Processor -- start 2024-04-15T15:20:37.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.906+0000 7f3e80c2a640 1 -- start start 2024-04-15T15:20:37.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.906+0000 7f3e80c2a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e7c0d18e0 0x7f3e7c15f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:37.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.906+0000 7f3e80c2a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e7c15eb60 0x7f3e7c1f43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:37.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.906+0000 7f3e80c2a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3e7c162b70 0x7f3e7c15d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:37.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.906+0000 7f3e80c2a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3e7c162f80 con 0x7f3e7c0d18e0 2024-04-15T15:20:37.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.906+0000 7f3e80c2a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3e7c0d2c50 con 0x7f3e7c15eb60 2024-04-15T15:20:37.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.906+0000 7f3e80c2a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f3e7c0b78f0 con 0x7f3e7c162b70 2024-04-15T15:20:37.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.906+0000 7f3e7a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e7c0d18e0 0x7f3e7c15f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:37.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.907+0000 7f3e7a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e7c0d18e0 0x7f3e7c15f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45378/0 (socket says 172.21.15.105:45378) 2024-04-15T15:20:37.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.907+0000 7f3e7a59c640 1 -- 172.21.15.105:0/1632925265 learned_addr learned my addr 172.21.15.105:0/1632925265 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:37.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.907+0000 7f3e7ad9d640 1 --2- 172.21.15.105:0/1632925265 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3e7c162b70 0x7f3e7c15d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:37.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.907+0000 7f3e73fff640 1 --2- 172.21.15.105:0/1632925265 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e7c15eb60 0x7f3e7c1f43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:37.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.907+0000 7f3e7a59c640 1 -- 172.21.15.105:0/1632925265 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3e7c162b70 msgr2=0x7f3e7c15d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:37.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.907+0000 7f3e7a59c640 1 --2- 172.21.15.105:0/1632925265 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3e7c162b70 0x7f3e7c15d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:37.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.907+0000 7f3e7a59c640 1 -- 172.21.15.105:0/1632925265 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e7c15eb60 msgr2=0x7f3e7c1f43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:37.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.907+0000 7f3e7a59c640 1 --2- 172.21.15.105:0/1632925265 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e7c15eb60 0x7f3e7c1f43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:37.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.907+0000 7f3e7a59c640 1 -- 172.21.15.105:0/1632925265 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e7c1f4930 con 0x7f3e7c0d18e0 2024-04-15T15:20:37.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.907+0000 7f3e7ad9d640 1 --2- 172.21.15.105:0/1632925265 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3e7c162b70 0x7f3e7c15d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:20:37.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.907+0000 7f3e7a59c640 1 --2- 172.21.15.105:0/1632925265 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e7c0d18e0 0x7f3e7c15f130 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c00c4d0 tx=0x7f3e6c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:37.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.907+0000 7f3e72ffd640 1 -- 172.21.15.105:0/1632925265 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c018070 con 0x7f3e7c0d18e0 2024-04-15T15:20:37.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.907+0000 7f3e72ffd640 1 -- 172.21.15.105:0/1632925265 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e6c007500 con 0x7f3e7c0d18e0 2024-04-15T15:20:37.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.907+0000 7f3e72ffd640 1 -- 172.21.15.105:0/1632925265 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c004a20 con 0x7f3e7c0d18e0 2024-04-15T15:20:37.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.907+0000 7f3e80c2a640 1 -- 172.21.15.105:0/1632925265 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e7c1f4bd0 con 0x7f3e7c0d18e0 2024-04-15T15:20:37.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.907+0000 7f3e80c2a640 1 -- 172.21.15.105:0/1632925265 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e7c1f5190 con 0x7f3e7c0d18e0 2024-04-15T15:20:37.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.908+0000 7f3e80c2a640 1 -- 172.21.15.105:0/1632925265 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e7c068a40 con 0x7f3e7c0d18e0 2024-04-15T15:20:37.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.909+0000 7f3e72ffd640 1 -- 172.21.15.105:0/1632925265 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3e6c01b070 con 0x7f3e7c0d18e0 2024-04-15T15:20:37.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.909+0000 7f3e72ffd640 1 --2- 172.21.15.105:0/1632925265 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f3e5c06c8f0 0x7f3e5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:37.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.909+0000 7f3e72ffd640 1 -- 172.21.15.105:0/1632925265 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f3e6c08ae90 con 0x7f3e7c0d18e0 2024-04-15T15:20:37.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.909+0000 7f3e73fff640 1 --2- 172.21.15.105:0/1632925265 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f3e5c06c8f0 0x7f3e5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:37.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.910+0000 7f3e73fff640 1 --2- 172.21.15.105:0/1632925265 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f3e5c06c8f0 0x7f3e5c06edb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3e64036000 tx=0x7f3e64008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:37.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:37.914+0000 7f3e72ffd640 1 -- 172.21.15.105:0/1632925265 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3e6c0592c0 con 0x7f3e7c0d18e0 2024-04-15T15:20:38.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.125+0000 7f3e80c2a640 1 -- 172.21.15.105:0/1632925265 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3e7c0d23f0 con 0x7f3e7c0d18e0 2024-04-15T15:20:38.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.125+0000 7f3e72ffd640 1 -- 172.21.15.105:0/1632925265 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+5180 (secure 0 0 0) 0x7f3e6c058bc0 con 0x7f3e7c0d18e0 2024-04-15T15:20:38.127 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:20:38.127 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4778,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/4848508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":4848508},{"type":"v1","addr":"172.21.15.154:6835","nonce":4848508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5005,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/1127070021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1127070021},{"type":"v1","addr":"172.21.15.154:6833","nonce":1127070021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5193,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6836/3825669416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3825669416},{"type":"v1","addr":"172.21.15.105:6836","nonce":3825669416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-15T15:20:29.000616+0000","modified":"2024-04-15T15:20:31.008997+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5196},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5196":{"gid":5196,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6837/958066041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":958066041},{"type":"v1","addr":"172.21.15.105:6837","nonce":958066041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5196,"qdb_cluster":[5196]},"id":2}]} 2024-04-15T15:20:38.127 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 14 2024-04-15T15:20:38.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.127+0000 7f3e80c2a640 1 -- 172.21.15.105:0/1632925265 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f3e5c06c8f0 msgr2=0x7f3e5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:38.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.127+0000 7f3e80c2a640 1 --2- 172.21.15.105:0/1632925265 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f3e5c06c8f0 0x7f3e5c06edb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3e64036000 tx=0x7f3e64008030 comp rx=0 tx=0).stop 2024-04-15T15:20:38.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.127+0000 7f3e80c2a640 1 -- 172.21.15.105:0/1632925265 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e7c0d18e0 msgr2=0x7f3e7c15f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:38.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.127+0000 7f3e80c2a640 1 --2- 172.21.15.105:0/1632925265 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e7c0d18e0 0x7f3e7c15f130 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c00c4d0 tx=0x7f3e6c00c990 comp rx=0 tx=0).stop 2024-04-15T15:20:38.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.127+0000 7f3e80c2a640 1 -- 172.21.15.105:0/1632925265 shutdown_connections 2024-04-15T15:20:38.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.127+0000 7f3e80c2a640 1 --2- 172.21.15.105:0/1632925265 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3e7c162b70 0x7f3e7c15d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:38.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.127+0000 7f3e80c2a640 1 --2- 172.21.15.105:0/1632925265 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f3e5c06c8f0 0x7f3e5c06edb0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:38.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.127+0000 7f3e80c2a640 1 --2- 172.21.15.105:0/1632925265 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e7c15eb60 0x7f3e7c1f43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:38.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.127+0000 7f3e80c2a640 1 --2- 172.21.15.105:0/1632925265 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e7c0d18e0 0x7f3e7c15f130 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:38.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.127+0000 7f3e80c2a640 1 -- 172.21.15.105:0/1632925265 >> 172.21.15.105:0/1632925265 conn(0x7f3e7c067720 msgr2=0x7f3e7c0b37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:38.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.127+0000 7f3e80c2a640 1 -- 172.21.15.105:0/1632925265 shutdown_connections 2024-04-15T15:20:38.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.127+0000 7f3e80c2a640 1 -- 172.21.15.105:0/1632925265 wait complete. 2024-04-15T15:20:38.544 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 14, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-15T15:20:29.000616+0000', 'modified': '2024-04-15T15:20:31.008997+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5196}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5196': {'gid': 5196, 'name': 'a', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.105:6837/958066041', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6835', 'nonce': 958066041}, {'type': 'v1', 'addr': '172.21.15.105:6837', 'nonce': 958066041}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5196, 'qdb_cluster': [5196]} 2024-04-15T15:20:38.544 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-15T15:20:38.544 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-15T15:20:38.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.648+0000 7f6e3a655640 1 Processor -- start 2024-04-15T15:20:38.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.648+0000 7f6e3a655640 1 -- start start 2024-04-15T15:20:38.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.648+0000 7f6e3a655640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e340d18e0 0x7f6e34162630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:38.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.648+0000 7f6e3a655640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e3415ef40 con 0x7f6e34162b70 2024-04-15T15:20:38.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.648+0000 7f6e3a655640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e34162fc0 con 0x7f6e3415eb60 2024-04-15T15:20:38.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.648+0000 7f6e3a655640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f6e34163540 con 0x7f6e340d18e0 2024-04-15T15:20:38.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.649+0000 7f6e38bf2640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f6e34162b70 0x7f6e340d3450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:43760/0 (socket says 172.21.15.105:43760) 2024-04-15T15:20:38.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.649+0000 7f6e38bf2640 1 -- 172.21.15.105:0/708037705 learned_addr learned my addr 172.21.15.105:0/708037705 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:20:38.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.649+0000 7f6e2bfff640 1 --2- 172.21.15.105:0/708037705 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e340d18e0 0x7f6e34162630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:38.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.649+0000 7f6e337fe640 1 -- 172.21.15.105:0/708037705 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2407286919 0 0) 0x7f6e3415ef40 con 0x7f6e34162b70 2024-04-15T15:20:38.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.649+0000 7f6e337fe640 1 -- 172.21.15.105:0/708037705 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6e20003520 con 0x7f6e34162b70 2024-04-15T15:20:38.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.649+0000 7f6e337fe640 1 -- 172.21.15.105:0/708037705 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4232698610 0 0) 0x7f6e20003520 con 0x7f6e34162b70 2024-04-15T15:20:38.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.649+0000 7f6e337fe640 1 -- 172.21.15.105:0/708037705 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e340d18e0 msgr2=0x7f6e34162630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:38.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.649+0000 7f6e337fe640 1 --2- 172.21.15.105:0/708037705 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e340d18e0 0x7f6e34162630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:38.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.649+0000 7f6e337fe640 1 -- 172.21.15.105:0/708037705 >> v1:172.21.15.154:6789/0 conn(0x7f6e3415eb60 legacy=0x7f6e340d11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:38.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.649+0000 7f6e337fe640 1 -- 172.21.15.105:0/708037705 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e34163260 con 0x7f6e34162b70 2024-04-15T15:20:38.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.649+0000 7f6e337fe640 1 -- 172.21.15.105:0/708037705 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7f6e2c0019f0 con 0x7f6e34162b70 2024-04-15T15:20:38.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.649+0000 7f6e337fe640 1 -- 172.21.15.105:0/708037705 >> v1:172.21.15.105:6789/0 conn(0x7f6e34162b70 legacy=0x7f6e340d3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:38.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.649+0000 7f6e337fe640 1 --2- 172.21.15.105:0/708037705 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e200039e0 0x7f6e20003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:38.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.649+0000 7f6e337fe640 1 --2- 172.21.15.105:0/708037705 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e20004390 0x7f6e20004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:38.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.649+0000 7f6e337fe640 1 --2- 172.21.15.105:0/708037705 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e340d18e0 0x7f6e200050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:38.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.649+0000 7f6e337fe640 1 -- 172.21.15.105:0/708037705 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6e34163540 con 0x7f6e200039e0 2024-04-15T15:20:38.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.649+0000 7f6e337fe640 1 -- 172.21.15.105:0/708037705 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2000b9f0 con 0x7f6e20004390 2024-04-15T15:20:38.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.649+0000 7f6e337fe640 1 -- 172.21.15.105:0/708037705 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f6e2000bbf0 con 0x7f6e340d18e0 2024-04-15T15:20:38.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.649+0000 7f6e33fff640 1 --2- 172.21.15.105:0/708037705 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e200039e0 0x7f6e20003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:38.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.649+0000 7f6e2bfff640 1 --2- 172.21.15.105:0/708037705 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e20004390 0x7f6e20004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:38.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.650+0000 7f6e38bf2640 1 --2- 172.21.15.105:0/708037705 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e340d18e0 0x7f6e200050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:38.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.650+0000 7f6e33fff640 1 -- 172.21.15.105:0/708037705 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e340d18e0 msgr2=0x7f6e200050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:38.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.650+0000 7f6e33fff640 1 --2- 172.21.15.105:0/708037705 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e340d18e0 0x7f6e200050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:38.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.650+0000 7f6e33fff640 1 -- 172.21.15.105:0/708037705 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e20004390 msgr2=0x7f6e20004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:38.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.650+0000 7f6e33fff640 1 --2- 172.21.15.105:0/708037705 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e20004390 0x7f6e20004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:38.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.650+0000 7f6e33fff640 1 -- 172.21.15.105:0/708037705 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e20003520 con 0x7f6e200039e0 2024-04-15T15:20:38.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.650+0000 7f6e33fff640 1 --2- 172.21.15.105:0/708037705 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e200039e0 0x7f6e20003dc0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f6e24001910 tx=0x7f6e20005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b9e67a50ad5cfdd server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:38.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.651+0000 7f6e337fe640 1 -- 172.21.15.105:0/708037705 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e240014e0 con 0x7f6e200039e0 2024-04-15T15:20:38.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.651+0000 7f6e337fe640 1 -- 172.21.15.105:0/708037705 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e24037070 con 0x7f6e200039e0 2024-04-15T15:20:38.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.651+0000 7f6e337fe640 1 -- 172.21.15.105:0/708037705 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e24032870 con 0x7f6e200039e0 2024-04-15T15:20:38.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.651+0000 7f6e3a655640 1 -- 172.21.15.105:0/708037705 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e200039e0 msgr2=0x7f6e20003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:38.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.651+0000 7f6e3a655640 1 --2- 172.21.15.105:0/708037705 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e200039e0 0x7f6e20003dc0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f6e24001910 tx=0x7f6e20005830 comp rx=0 tx=0).stop 2024-04-15T15:20:38.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.651+0000 7f6e38bf2640 1 -- 172.21.15.105:0/708037705 reap_dead start 2024-04-15T15:20:38.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.651+0000 7f6e3a655640 1 -- 172.21.15.105:0/708037705 shutdown_connections 2024-04-15T15:20:38.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.651+0000 7f6e3a655640 1 -- 172.21.15.105:0/708037705 >> 172.21.15.105:0/708037705 conn(0x7f6e34067720 msgr2=0x7f6e34067b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:38.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.651+0000 7f6e3a655640 1 -- 172.21.15.105:0/708037705 shutdown_connections 2024-04-15T15:20:38.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.651+0000 7f6e3a655640 1 -- 172.21.15.105:0/708037705 wait complete. 2024-04-15T15:20:38.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.652+0000 7f6e3a655640 1 Processor -- start 2024-04-15T15:20:38.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.653+0000 7f6e3a655640 1 -- start start 2024-04-15T15:20:38.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.653+0000 7f6e3a655640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e200039e0 0x7f6e341f4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:38.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.653+0000 7f6e3a655640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e340cc8e0 0x7f6e341615a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:38.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.653+0000 7f6e3a655640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e340b2920 0x7f6e340b2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:38.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.653+0000 7f6e3a655640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6e340d2c50 con 0x7f6e340cc8e0 2024-04-15T15:20:38.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.653+0000 7f6e3a655640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6e340d2fc0 con 0x7f6e340b2920 2024-04-15T15:20:38.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.653+0000 7f6e3a655640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f6e340b78f0 con 0x7f6e200039e0 2024-04-15T15:20:38.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.653+0000 7f6e2bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e340cc8e0 0x7f6e341615a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:38.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.653+0000 7f6e2bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e340cc8e0 0x7f6e341615a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45406/0 (socket says 172.21.15.105:45406) 2024-04-15T15:20:38.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.653+0000 7f6e2bfff640 1 -- 172.21.15.105:0/3474115540 learned_addr learned my addr 172.21.15.105:0/3474115540 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:38.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.654+0000 7f6e33fff640 1 --2- 172.21.15.105:0/3474115540 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e200039e0 0x7f6e341f4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:38.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.654+0000 7f6e38bf2640 1 --2- 172.21.15.105:0/3474115540 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e340b2920 0x7f6e340b2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:38.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.654+0000 7f6e2bfff640 1 -- 172.21.15.105:0/3474115540 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e200039e0 msgr2=0x7f6e341f4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:38.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.654+0000 7f6e2bfff640 1 --2- 172.21.15.105:0/3474115540 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e200039e0 0x7f6e341f4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:38.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.654+0000 7f6e2bfff640 1 -- 172.21.15.105:0/3474115540 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e340b2920 msgr2=0x7f6e340b2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:38.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.654+0000 7f6e2bfff640 1 --2- 172.21.15.105:0/3474115540 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e340b2920 0x7f6e340b2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:38.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.654+0000 7f6e2bfff640 1 -- 172.21.15.105:0/3474115540 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e340b3310 con 0x7f6e340cc8e0 2024-04-15T15:20:38.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.654+0000 7f6e38bf2640 1 --2- 172.21.15.105:0/3474115540 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e340b2920 0x7f6e340b2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:38.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.654+0000 7f6e2bfff640 1 --2- 172.21.15.105:0/3474115540 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e340cc8e0 0x7f6e341615a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c0118e0 tx=0x7f6e1c011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:38.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.654+0000 7f6e317fa640 1 -- 172.21.15.105:0/3474115540 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c00faf0 con 0x7f6e340cc8e0 2024-04-15T15:20:38.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.654+0000 7f6e317fa640 1 -- 172.21.15.105:0/3474115540 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e1c00fc90 con 0x7f6e340cc8e0 2024-04-15T15:20:38.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.654+0000 7f6e3a655640 1 -- 172.21.15.105:0/3474115540 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e340cd110 con 0x7f6e340cc8e0 2024-04-15T15:20:38.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.654+0000 7f6e317fa640 1 -- 172.21.15.105:0/3474115540 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c014440 con 0x7f6e340cc8e0 2024-04-15T15:20:38.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.654+0000 7f6e3a655640 1 -- 172.21.15.105:0/3474115540 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e340cd9b0 con 0x7f6e340cc8e0 2024-04-15T15:20:38.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.656+0000 7f6e317fa640 1 -- 172.21.15.105:0/3474115540 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6e1c012040 con 0x7f6e340cc8e0 2024-04-15T15:20:38.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.656+0000 7f6e317fa640 1 --2- 172.21.15.105:0/3474115540 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f6e1406c920 0x7f6e1406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:38.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.656+0000 7f6e33fff640 1 --2- 172.21.15.105:0/3474115540 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f6e1406c920 0x7f6e1406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:38.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.656+0000 7f6e317fa640 1 -- 172.21.15.105:0/3474115540 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f6e1c08c170 con 0x7f6e340cc8e0 2024-04-15T15:20:38.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.657+0000 7f6e33fff640 1 --2- 172.21.15.105:0/3474115540 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f6e1406c920 0x7f6e1406ede0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6e24036000 tx=0x7f6e24009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:38.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.657+0000 7f6e3d240740 1 Processor -- start 2024-04-15T15:20:38.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.657+0000 7f6e3d240740 1 -- start start 2024-04-15T15:20:38.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.658+0000 7f6e3d240740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5642a0310e10 0x5642a02e3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:38.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.658+0000 7f6e3d240740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5642a03111f0 0x5642a0313ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:38.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.658+0000 7f6e3d240740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5642a02e42f0 0x5642a03140e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:38.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.658+0000 7f6e3d240740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5642a024e9c0 con 0x5642a03111f0 2024-04-15T15:20:38.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.658+0000 7f6e3d240740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5642a01f5650 con 0x5642a02e42f0 2024-04-15T15:20:38.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.658+0000 7f6e3d240740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x5642a024a980 con 0x5642a0310e10 2024-04-15T15:20:38.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.658+0000 7f6e2bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5642a03111f0 0x5642a0313ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:38.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.658+0000 7f6e2bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5642a03111f0 0x5642a0313ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45418/0 (socket says 172.21.15.105:45418) 2024-04-15T15:20:38.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.658+0000 7f6e2bfff640 1 -- 172.21.15.105:0/745524678 learned_addr learned my addr 172.21.15.105:0/745524678 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:38.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.658+0000 7f6e2bfff640 1 -- 172.21.15.105:0/745524678 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5642a0310e10 msgr2=0x5642a02e3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:38.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.658+0000 7f6e2bfff640 1 --2- 172.21.15.105:0/745524678 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5642a0310e10 0x5642a02e3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:38.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.658+0000 7f6e2bfff640 1 -- 172.21.15.105:0/745524678 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5642a02e42f0 msgr2=0x5642a03140e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:38.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.658+0000 7f6e2bfff640 1 --2- 172.21.15.105:0/745524678 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5642a02e42f0 0x5642a03140e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:38.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.658+0000 7f6e2bfff640 1 -- 172.21.15.105:0/745524678 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56429ff612d0 con 0x5642a03111f0 2024-04-15T15:20:38.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.658+0000 7f6e2bfff640 1 --2- 172.21.15.105:0/745524678 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5642a03111f0 0x5642a0313ba0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c062550 tx=0x7f6e1c05a090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a13ea1fba9276d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:38.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.659+0000 7f6e29ffb640 1 -- 172.21.15.105:0/745524678 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c072020 con 0x5642a03111f0 2024-04-15T15:20:38.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.659+0000 7f6e29ffb640 1 -- 172.21.15.105:0/745524678 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e1c057ff0 con 0x5642a03111f0 2024-04-15T15:20:38.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.659+0000 7f6e29ffb640 1 -- 172.21.15.105:0/745524678 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c0565c0 con 0x5642a03111f0 2024-04-15T15:20:38.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.659+0000 7f6e3d240740 1 -- 172.21.15.105:0/745524678 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5642a03111f0 msgr2=0x5642a0313ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:38.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.659+0000 7f6e3d240740 1 --2- 172.21.15.105:0/745524678 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5642a03111f0 0x5642a0313ba0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c062550 tx=0x7f6e1c05a090 comp rx=0 tx=0).stop 2024-04-15T15:20:38.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.659+0000 7f6e3d240740 1 -- 172.21.15.105:0/745524678 shutdown_connections 2024-04-15T15:20:38.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.659+0000 7f6e3d240740 1 --2- 172.21.15.105:0/745524678 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5642a02e42f0 0x5642a03140e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:38.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.659+0000 7f6e3d240740 1 --2- 172.21.15.105:0/745524678 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5642a03111f0 0x5642a0313ba0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:38.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.659+0000 7f6e3d240740 1 --2- 172.21.15.105:0/745524678 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5642a0310e10 0x5642a02e3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:38.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.659+0000 7f6e3d240740 1 -- 172.21.15.105:0/745524678 >> 172.21.15.105:0/745524678 conn(0x5642a02e6020 msgr2=0x5642a032eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:38.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.659+0000 7f6e3d240740 1 -- 172.21.15.105:0/745524678 shutdown_connections 2024-04-15T15:20:38.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.659+0000 7f6e3d240740 1 -- 172.21.15.105:0/745524678 wait complete. 2024-04-15T15:20:38.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.660+0000 7f6e3d240740 1 -- start start 2024-04-15T15:20:38.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.660+0000 7f6e3d240740 1 Processor -- start 2024-04-15T15:20:38.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.661+0000 7f6e3d240740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5642a0310e10 0x5642a030ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:38.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.661+0000 7f6e3d240740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5642a03111f0 0x5642a03279d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:38.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.661+0000 7f6e3d240740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5642a0327f10 0x5642a033ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:38.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.661+0000 7f6e3d240740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5642a02e85c0 con 0x5642a0310e10 2024-04-15T15:20:38.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.661+0000 7f6e3d240740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5642a0208a10 con 0x5642a03111f0 2024-04-15T15:20:38.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.661+0000 7f6e3d240740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x5642a02e79b0 con 0x5642a0327f10 2024-04-15T15:20:38.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.661+0000 7f6e3d240740 10 client.0 _ll_register_callbacks cb 0x5642a0244250 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-15T15:20:38.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.661+0000 7f6e33fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5642a0310e10 0x5642a030ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:38.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.661+0000 7f6e33fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5642a0310e10 0x5642a030ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45430/0 (socket says 172.21.15.105:45430) 2024-04-15T15:20:38.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.661+0000 7f6e33fff640 1 -- 172.21.15.105:0/1506823172 learned_addr learned my addr 172.21.15.105:0/1506823172 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:38.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.661+0000 7f6e33fff640 1 -- 172.21.15.105:0/1506823172 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5642a0327f10 msgr2=0x5642a033ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:38.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.661+0000 7f6e33fff640 1 --2- 172.21.15.105:0/1506823172 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5642a0327f10 0x5642a033ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:38.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.661+0000 7f6e33fff640 1 -- 172.21.15.105:0/1506823172 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5642a03111f0 msgr2=0x5642a03279d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:38.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.661+0000 7f6e33fff640 1 --2- 172.21.15.105:0/1506823172 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5642a03111f0 0x5642a03279d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:38.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.661+0000 7f6e33fff640 1 -- 172.21.15.105:0/1506823172 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642a002b6b0 con 0x5642a0310e10 2024-04-15T15:20:38.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.661+0000 7f6e33fff640 1 --2- 172.21.15.105:0/1506823172 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5642a0310e10 0x5642a030ec20 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6e24001a00 tx=0x7f6e240012b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f60c74b37f5564e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:38.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.661+0000 7f6e12ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-04-15T15:20:38.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.661+0000 7f6e12ffd640 1 -- 172.21.15.105:0/1506823172 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e24032b50 con 0x5642a0310e10 2024-04-15T15:20:38.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.661+0000 7f6e12ffd640 1 -- 172.21.15.105:0/1506823172 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f6e24032cf0 con 0x5642a0310e10 2024-04-15T15:20:38.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.661+0000 7f6e3d240740 1 -- 172.21.15.105:0/1506823172 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e1c057ff0 con 0x5642a0310e10 2024-04-15T15:20:38.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.662+0000 7f6e12ffd640 1 -- 172.21.15.105:0/1506823172 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) ==== 24+0+0 (secure 0 0 0) 0x7f6e2403c400 con 0x5642a0310e10 2024-04-15T15:20:38.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.662+0000 7f6e3d240740 10 client.5397 fetch_fsmap learned FSMap version 14 2024-04-15T15:20:38.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.662+0000 7f6e3d240740 1 -- 172.21.15.105:0/1506823172 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) -- 0x5642a0315bf0 con 0x5642a0310e10 2024-04-15T15:20:38.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.662+0000 7f6e12ffd640 1 -- 172.21.15.105:0/1506823172 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e1c057ff0 con 0x5642a0310e10 2024-04-15T15:20:38.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.662+0000 7f6e12ffd640 1 -- 172.21.15.105:0/1506823172 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 14) ==== 3494+0+0 (secure 0 0 0) 0x7f6e2403c5a0 con 0x5642a0310e10 2024-04-15T15:20:38.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.662+0000 7f6e3d240740 10 client.5397 fetch_fsmap finished waiting for FSMap version 14 2024-04-15T15:20:38.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.662+0000 7f6e3d240740 10 client.5397 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-15T15:20:38.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.662+0000 7f6e3d240740 20 client.5397 populate_metadata read hostname 'smithi105' 2024-04-15T15:20:38.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.662+0000 7f6e3d240740 1 --2- 172.21.15.105:0/1506823172 >> [v2:172.21.15.105:6835/958066041,v1:172.21.15.105:6837/958066041] conn(0x5642a034aa20 0x5642a034ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:38.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.662+0000 7f6e3d240740 4 client.5397 mds_command: new command op to 5196 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-15T15:20:38.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.662+0000 7f6e3d240740 1 -- 172.21.15.105:0/1506823172 --> [v2:172.21.15.105:6835/958066041,v1:172.21.15.105:6837/958066041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642a025c3e0 con 0x5642a034aa20 2024-04-15T15:20:38.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.662+0000 7f6e2bfff640 1 --2- 172.21.15.105:0/1506823172 >> [v2:172.21.15.105:6835/958066041,v1:172.21.15.105:6837/958066041] conn(0x5642a034aa20 0x5642a034ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:38.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.662+0000 7f6e12ffd640 10 client.5397 ms_handle_connect on v2:172.21.15.105:6835/958066041 2024-04-15T15:20:38.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.662+0000 7f6e2bfff640 1 --2- 172.21.15.105:0/1506823172 >> [v2:172.21.15.105:6835/958066041,v1:172.21.15.105:6837/958066041] conn(0x5642a034aa20 0x5642a034ce10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ffdda38a60da88a server_cookie=bb11f0d1416d9e23 in_seq=0 out_seq=0 2024-04-15T15:20:38.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.663+0000 7f6e12ffd640 1 -- 172.21.15.105:0/1506823172 <== mds.0 v2:172.21.15.105:6835/958066041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5642a025c3e0 con 0x5642a034aa20 2024-04-15T15:20:38.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.663+0000 7f6e12ffd640 10 client.5397 handle_command_reply: tid=0 2024-04-15T15:20:38.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.663+0000 7f6e3d240740 1 client.5397 shutdown 2024-04-15T15:20:38.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.663+0000 7f6e3d240740 1 -- 172.21.15.105:0/1506823172 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5642a0310e10 msgr2=0x5642a030ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:38.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.663+0000 7f6e3d240740 1 --2- 172.21.15.105:0/1506823172 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5642a0310e10 0x5642a030ec20 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6e24001a00 tx=0x7f6e240012b0 comp rx=0 tx=0).stop 2024-04-15T15:20:38.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.663+0000 7f6e3d240740 1 -- 172.21.15.105:0/1506823172 shutdown_connections 2024-04-15T15:20:38.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.663+0000 7f6e3d240740 1 --2- 172.21.15.105:0/1506823172 >> [v2:172.21.15.105:6835/958066041,v1:172.21.15.105:6837/958066041] conn(0x5642a034aa20 0x5642a034ce10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:38.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.663+0000 7f6e3d240740 1 --2- 172.21.15.105:0/1506823172 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5642a0327f10 0x5642a033ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:38.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.663+0000 7f6e12ffd640 0 client.5397 ms_handle_reset on v2:172.21.15.105:6835/958066041 2024-04-15T15:20:38.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.663+0000 7f6e3d240740 1 --2- 172.21.15.105:0/1506823172 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5642a03111f0 0x5642a03279d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:38.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.663+0000 7f6e3d240740 1 --2- 172.21.15.105:0/1506823172 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5642a0310e10 0x5642a030ec20 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:38.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.663+0000 7f6e3d240740 1 -- 172.21.15.105:0/1506823172 >> 172.21.15.105:0/1506823172 conn(0x5642a02e42f0 msgr2=0x5642a03136a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:38.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.663+0000 7f6e3d240740 1 -- 172.21.15.105:0/1506823172 shutdown_connections 2024-04-15T15:20:38.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.663+0000 7f6e3d240740 1 -- 172.21.15.105:0/1506823172 wait complete. 2024-04-15T15:20:38.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.663+0000 7f6e3d240740 20 client.5397 trim_cache size 0 max 16384 2024-04-15T15:20:38.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.681+0000 7f6e3d240740 1 Processor -- start 2024-04-15T15:20:38.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.681+0000 7f6e3d240740 1 -- start start 2024-04-15T15:20:38.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.682+0000 7f6e3d240740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5642a0310e10 0x5642a033edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:38.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.682+0000 7f6e3d240740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5642a03111f0 0x5642a02e7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:38.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.682+0000 7f6e3d240740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5642a0327f10 0x5642a0328c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:38.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.682+0000 7f6e3d240740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6e34201550 con 0x5642a0327f10 2024-04-15T15:20:38.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.682+0000 7f6e3d240740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6e34200530 con 0x5642a0310e10 2024-04-15T15:20:38.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.682+0000 7f6e3d240740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f6e3415fb30 con 0x5642a03111f0 2024-04-15T15:20:38.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.682+0000 7f6e38bf2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5642a0327f10 0x5642a0328c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:38.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.682+0000 7f6e38bf2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5642a0327f10 0x5642a0328c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45444/0 (socket says 172.21.15.105:45444) 2024-04-15T15:20:38.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.682+0000 7f6e38bf2640 1 -- 172.21.15.105:0/1339113546 learned_addr learned my addr 172.21.15.105:0/1339113546 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:38.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.682+0000 7f6e2bfff640 1 --2- 172.21.15.105:0/1339113546 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5642a03111f0 0x5642a02e7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:38.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.682+0000 7f6e38bf2640 1 -- 172.21.15.105:0/1339113546 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5642a03111f0 msgr2=0x5642a02e7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:38.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.682+0000 7f6e38bf2640 1 --2- 172.21.15.105:0/1339113546 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5642a03111f0 0x5642a02e7d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:38.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.682+0000 7f6e38bf2640 1 -- 172.21.15.105:0/1339113546 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5642a0310e10 msgr2=0x5642a033edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:38.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.682+0000 7f6e38bf2640 1 --2- 172.21.15.105:0/1339113546 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5642a0310e10 0x5642a033edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:38.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.682+0000 7f6e38bf2640 1 -- 172.21.15.105:0/1339113546 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642a032f860 con 0x5642a0327f10 2024-04-15T15:20:38.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.683+0000 7f6e38bf2640 1 --2- 172.21.15.105:0/1339113546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5642a0327f10 0x5642a0328c60 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c000f80 tx=0x7f6e2c004f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a7a6567c946eb78 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:38.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.683+0000 7f6e297fa640 1 -- 172.21.15.105:0/1339113546 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c0076b0 con 0x5642a0327f10 2024-04-15T15:20:38.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.683+0000 7f6e297fa640 1 -- 172.21.15.105:0/1339113546 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e2c019070 con 0x5642a0327f10 2024-04-15T15:20:38.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.683+0000 7f6e297fa640 1 -- 172.21.15.105:0/1339113546 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c007a90 con 0x5642a0327f10 2024-04-15T15:20:38.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.683+0000 7f6e3d240740 1 -- 172.21.15.105:0/1339113546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5642a0327f10 msgr2=0x5642a0328c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:38.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.683+0000 7f6e3d240740 1 --2- 172.21.15.105:0/1339113546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5642a0327f10 0x5642a0328c60 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c000f80 tx=0x7f6e2c004f30 comp rx=0 tx=0).stop 2024-04-15T15:20:38.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.683+0000 7f6e3d240740 1 -- 172.21.15.105:0/1339113546 shutdown_connections 2024-04-15T15:20:38.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.683+0000 7f6e3d240740 1 --2- 172.21.15.105:0/1339113546 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5642a03111f0 0x5642a02e7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:38.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.683+0000 7f6e3d240740 1 --2- 172.21.15.105:0/1339113546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5642a0327f10 0x5642a0328c60 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:38.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.683+0000 7f6e3d240740 1 --2- 172.21.15.105:0/1339113546 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5642a0310e10 0x5642a033edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:38.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.683+0000 7f6e3d240740 1 -- 172.21.15.105:0/1339113546 >> 172.21.15.105:0/1339113546 conn(0x5642a02e42f0 msgr2=0x5642a02eadd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:38.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.683+0000 7f6e3d240740 1 -- 172.21.15.105:0/1339113546 shutdown_connections 2024-04-15T15:20:38.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.683+0000 7f6e3d240740 1 -- 172.21.15.105:0/1339113546 wait complete. 2024-04-15T15:20:38.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.684+0000 7f6e3d240740 1 -- start start 2024-04-15T15:20:38.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.684+0000 7f6e3d240740 1 Processor -- start 2024-04-15T15:20:38.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.685+0000 7f6e3d240740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5642a0310e10 0x5642a038e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:38.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.685+0000 7f6e3d240740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5642a03111f0 0x5642a0328bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:38.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.685+0000 7f6e3d240740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5642a0327f10 0x5642a0329130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:38.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.685+0000 7f6e3d240740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5642a0315f70 con 0x5642a03111f0 2024-04-15T15:20:38.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.685+0000 7f6e3d240740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6e341fff10 con 0x5642a0310e10 2024-04-15T15:20:38.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.685+0000 7f6e3d240740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x5642a033feb0 con 0x5642a0327f10 2024-04-15T15:20:38.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.685+0000 7f6e3d240740 10 client.0 _ll_register_callbacks cb 0x7f6e200036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-15T15:20:38.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.685+0000 7f6e2bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5642a03111f0 0x5642a0328bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:38.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.685+0000 7f6e2bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5642a03111f0 0x5642a0328bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45450/0 (socket says 172.21.15.105:45450) 2024-04-15T15:20:38.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.685+0000 7f6e2bfff640 1 -- 172.21.15.105:0/1819845335 learned_addr learned my addr 172.21.15.105:0/1819845335 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:38.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.685+0000 7f6e33fff640 1 --2- 172.21.15.105:0/1819845335 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5642a0310e10 0x5642a038e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:38.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.685+0000 7f6e38bf2640 1 --2- 172.21.15.105:0/1819845335 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5642a0327f10 0x5642a0329130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:38.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.685+0000 7f6e2bfff640 1 -- 172.21.15.105:0/1819845335 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5642a0327f10 msgr2=0x5642a0329130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:38.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.685+0000 7f6e2bfff640 1 --2- 172.21.15.105:0/1819845335 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5642a0327f10 0x5642a0329130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:38.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.685+0000 7f6e2bfff640 1 -- 172.21.15.105:0/1819845335 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5642a0310e10 msgr2=0x5642a038e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:38.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.685+0000 7f6e2bfff640 1 --2- 172.21.15.105:0/1819845335 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5642a0310e10 0x5642a038e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:38.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.685+0000 7f6e2bfff640 1 -- 172.21.15.105:0/1819845335 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642a030f320 con 0x5642a03111f0 2024-04-15T15:20:38.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.685+0000 7f6e33fff640 1 --2- 172.21.15.105:0/1819845335 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5642a0310e10 0x5642a038e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:20:38.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.685+0000 7f6e2bfff640 1 --2- 172.21.15.105:0/1819845335 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5642a03111f0 0x5642a0328bf0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c08a5e0 tx=0x7f6e1c016070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=908f67c92b923aed server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:38.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.685+0000 7f6e12ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-04-15T15:20:38.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.685+0000 7f6e12ffd640 1 -- 172.21.15.105:0/1819845335 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c072040 con 0x5642a03111f0 2024-04-15T15:20:38.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.685+0000 7f6e12ffd640 1 -- 172.21.15.105:0/1819845335 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f6e1c08a040 con 0x5642a03111f0 2024-04-15T15:20:38.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.685+0000 7f6e3d240740 1 -- 172.21.15.105:0/1819845335 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e2c019070 con 0x5642a03111f0 2024-04-15T15:20:38.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.686+0000 7f6e12ffd640 1 -- 172.21.15.105:0/1819845335 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) ==== 24+0+0 (secure 0 0 0) 0x7f6e1c083050 con 0x5642a03111f0 2024-04-15T15:20:38.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.686+0000 7f6e3d240740 10 client.5403 fetch_fsmap learned FSMap version 14 2024-04-15T15:20:38.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.686+0000 7f6e3d240740 1 -- 172.21.15.105:0/1819845335 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) -- 0x5642a03526a0 con 0x5642a03111f0 2024-04-15T15:20:38.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.686+0000 7f6e12ffd640 1 -- 172.21.15.105:0/1819845335 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e2c019070 con 0x5642a03111f0 2024-04-15T15:20:38.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.686+0000 7f6e12ffd640 1 -- 172.21.15.105:0/1819845335 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 14) ==== 3494+0+0 (secure 0 0 0) 0x7f6e1c025030 con 0x5642a03111f0 2024-04-15T15:20:38.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.686+0000 7f6e3d240740 10 client.5403 fetch_fsmap finished waiting for FSMap version 14 2024-04-15T15:20:38.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.686+0000 7f6e3d240740 10 client.5403 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-15T15:20:38.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.686+0000 7f6e3d240740 20 client.5403 populate_metadata read hostname 'smithi105' 2024-04-15T15:20:38.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.686+0000 7f6e3d240740 1 --2- 172.21.15.105:0/1819845335 >> [v2:172.21.15.105:6835/958066041,v1:172.21.15.105:6837/958066041] conn(0x5642a0352020 0x5642a030d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:38.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.686+0000 7f6e3d240740 4 client.5403 mds_command: new command op to 5196 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-15T15:20:38.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.686+0000 7f6e3d240740 1 -- 172.21.15.105:0/1819845335 --> [v2:172.21.15.105:6835/958066041,v1:172.21.15.105:6837/958066041] -- command(tid 0: {"prefix": "status"}) -- 0x5642a0256250 con 0x5642a0352020 2024-04-15T15:20:38.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.686+0000 7f6e33fff640 1 --2- 172.21.15.105:0/1819845335 >> [v2:172.21.15.105:6835/958066041,v1:172.21.15.105:6837/958066041] conn(0x5642a0352020 0x5642a030d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:38.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.686+0000 7f6e12ffd640 10 client.5403 ms_handle_connect on v2:172.21.15.105:6835/958066041 2024-04-15T15:20:38.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.686+0000 7f6e33fff640 1 --2- 172.21.15.105:0/1819845335 >> [v2:172.21.15.105:6835/958066041,v1:172.21.15.105:6837/958066041] conn(0x5642a0352020 0x5642a030d700 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5842b476a5a27088 server_cookie=e5fcfbd6f0038f32 in_seq=0 out_seq=0 2024-04-15T15:20:38.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.686+0000 7f6e12ffd640 1 -- 172.21.15.105:0/1819845335 <== mds.0 v2:172.21.15.105:6835/958066041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x5642a0256250 con 0x5642a0352020 2024-04-15T15:20:38.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.686+0000 7f6e12ffd640 10 client.5403 handle_command_reply: tid=0 2024-04-15T15:20:38.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.686+0000 7f6e3d240740 1 client.5403 shutdown 2024-04-15T15:20:38.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.687+0000 7f6e3d240740 1 -- 172.21.15.105:0/1819845335 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5642a03111f0 msgr2=0x5642a0328bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:38.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.687+0000 7f6e3d240740 1 --2- 172.21.15.105:0/1819845335 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5642a03111f0 0x5642a0328bf0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c08a5e0 tx=0x7f6e1c016070 comp rx=0 tx=0).stop 2024-04-15T15:20:38.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.687+0000 7f6e3d240740 1 -- 172.21.15.105:0/1819845335 shutdown_connections 2024-04-15T15:20:38.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.687+0000 7f6e3d240740 1 --2- 172.21.15.105:0/1819845335 >> [v2:172.21.15.105:6835/958066041,v1:172.21.15.105:6837/958066041] conn(0x5642a0352020 0x5642a030d700 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:38.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.687+0000 7f6e3d240740 1 --2- 172.21.15.105:0/1819845335 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5642a0327f10 0x5642a0329130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:38.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.687+0000 7f6e3d240740 1 --2- 172.21.15.105:0/1819845335 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5642a03111f0 0x5642a0328bf0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:38.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.687+0000 7f6e12ffd640 0 client.5403 ms_handle_reset on v2:172.21.15.105:6835/958066041 2024-04-15T15:20:38.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.687+0000 7f6e3d240740 1 --2- 172.21.15.105:0/1819845335 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5642a0310e10 0x5642a038e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:38.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.687+0000 7f6e3d240740 1 -- 172.21.15.105:0/1819845335 >> 172.21.15.105:0/1819845335 conn(0x5642a02e42f0 msgr2=0x5642a033eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:38.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.687+0000 7f6e3d240740 1 -- 172.21.15.105:0/1819845335 shutdown_connections 2024-04-15T15:20:38.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.687+0000 7f6e3d240740 1 -- 172.21.15.105:0/1819845335 wait complete. 2024-04-15T15:20:38.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.687+0000 7f6e3d240740 20 client.5403 trim_cache size 0 max 16384 2024-04-15T15:20:38.695 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-04-15T15:20:38.695 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "09e4d2a4-a324-4793-af77-644b25e520e1", 2024-04-15T15:20:38.695 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 0, 2024-04-15T15:20:38.695 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 5196, 2024-04-15T15:20:38.696 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2024-04-15T15:20:38.696 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2024-04-15T15:20:38.696 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2024-04-15T15:20:38.696 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 9.0416286830000008, 2024-04-15T15:20:38.696 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 14, 2024-04-15T15:20:38.696 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 34, 2024-04-15T15:20:38.696 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 34, 2024-04-15T15:20:38.696 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 23.577000352999999 2024-04-15T15:20:38.696 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-04-15T15:20:38.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.687+0000 7f6e3a655640 1 -- 172.21.15.105:0/3474115540 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f6e1406c920 msgr2=0x7f6e1406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:38.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.687+0000 7f6e3a655640 1 --2- 172.21.15.105:0/3474115540 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f6e1406c920 0x7f6e1406ede0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6e24036000 tx=0x7f6e24009030 comp rx=0 tx=0).stop 2024-04-15T15:20:38.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.687+0000 7f6e3a655640 1 -- 172.21.15.105:0/3474115540 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e340cc8e0 msgr2=0x7f6e341615a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:38.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.687+0000 7f6e3a655640 1 --2- 172.21.15.105:0/3474115540 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e340cc8e0 0x7f6e341615a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c0118e0 tx=0x7f6e1c011da0 comp rx=0 tx=0).stop 2024-04-15T15:20:38.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.687+0000 7f6e3a655640 1 -- 172.21.15.105:0/3474115540 shutdown_connections 2024-04-15T15:20:38.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.687+0000 7f6e3a655640 1 --2- 172.21.15.105:0/3474115540 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f6e1406c920 0x7f6e1406ede0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:38.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.687+0000 7f6e3a655640 1 --2- 172.21.15.105:0/3474115540 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e340b2920 0x7f6e340b2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:38.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.687+0000 7f6e3a655640 1 --2- 172.21.15.105:0/3474115540 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e340cc8e0 0x7f6e341615a0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:38.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.687+0000 7f6e3a655640 1 --2- 172.21.15.105:0/3474115540 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e200039e0 0x7f6e341f4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:38.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.687+0000 7f6e3a655640 1 -- 172.21.15.105:0/3474115540 >> 172.21.15.105:0/3474115540 conn(0x7f6e34067720 msgr2=0x7f6e340b3830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:38.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.687+0000 7f6e3a655640 1 -- 172.21.15.105:0/3474115540 shutdown_connections 2024-04-15T15:20:38.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.687+0000 7f6e3a655640 1 -- 172.21.15.105:0/3474115540 wait complete. 2024-04-15T15:20:38.700 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-15T15:20:38.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.801+0000 7fcb54031640 1 Processor -- start 2024-04-15T15:20:38.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.802+0000 7fcb54031640 1 -- start start 2024-04-15T15:20:38.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.802+0000 7fcb54031640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcb4c15eab0 0x7fcb4c162530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:38.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.802+0000 7fcb54031640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fcb4c0b78f0 0x7fcb4c0d11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:38.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.802+0000 7fcb54031640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb4c162ff0 con 0x7fcb4c162a70 2024-04-15T15:20:38.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.802+0000 7fcb54031640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fcb4c0d4540 con 0x7fcb4c0b78f0 2024-04-15T15:20:38.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.802+0000 7fcb54031640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fcb4c15ee90 con 0x7fcb4c15eab0 2024-04-15T15:20:38.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.803+0000 7fcb525ce640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fcb4c162a70 0x7fcb4c0b6240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:43766/0 (socket says 172.21.15.105:43766) 2024-04-15T15:20:38.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.803+0000 7fcb525ce640 1 -- 172.21.15.105:0/1039398237 learned_addr learned my addr 172.21.15.105:0/1039398237 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:20:38.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.803+0000 7fcb51dcd640 1 --2- 172.21.15.105:0/1039398237 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcb4c15eab0 0x7fcb4c162530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:38.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.803+0000 7fcb515cc640 1 --2- 172.21.15.105:0/1039398237 >> v2:172.21.15.154:3300/0 conn(0x7fcb4c0b78f0 0x7fcb4c0d11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:38.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.803+0000 7fcb50dcb640 1 -- 172.21.15.105:0/1039398237 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1137796064 0 0) 0x7fcb4c162ff0 con 0x7fcb4c162a70 2024-04-15T15:20:38.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.803+0000 7fcb50dcb640 1 -- 172.21.15.105:0/1039398237 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcb38003520 con 0x7fcb4c162a70 2024-04-15T15:20:38.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.803+0000 7fcb50dcb640 1 -- 172.21.15.105:0/1039398237 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3251094053 0 0) 0x7fcb38003520 con 0x7fcb4c162a70 2024-04-15T15:20:38.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.803+0000 7fcb50dcb640 1 -- 172.21.15.105:0/1039398237 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcb4c15eab0 msgr2=0x7fcb4c162530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:38.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.803+0000 7fcb50dcb640 1 --2- 172.21.15.105:0/1039398237 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcb4c15eab0 0x7fcb4c162530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:38.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.803+0000 7fcb50dcb640 1 -- 172.21.15.105:0/1039398237 >> v2:172.21.15.154:3300/0 conn(0x7fcb4c0b78f0 msgr2=0x7fcb4c0d11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:38.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.803+0000 7fcb50dcb640 1 --2- 172.21.15.105:0/1039398237 >> v2:172.21.15.154:3300/0 conn(0x7fcb4c0b78f0 0x7fcb4c0d11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:38.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.803+0000 7fcb50dcb640 1 -- 172.21.15.105:0/1039398237 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb4c1632d0 con 0x7fcb4c162a70 2024-04-15T15:20:38.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.803+0000 7fcb50dcb640 1 -- 172.21.15.105:0/1039398237 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7fcb440019f0 con 0x7fcb4c162a70 2024-04-15T15:20:38.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.803+0000 7fcb50dcb640 1 -- 172.21.15.105:0/1039398237 >> v1:172.21.15.105:6789/0 conn(0x7fcb4c162a70 legacy=0x7fcb4c0b6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:38.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.803+0000 7fcb50dcb640 1 --2- 172.21.15.105:0/1039398237 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb380039b0 0x7fcb38003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:38.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.803+0000 7fcb50dcb640 1 --2- 172.21.15.105:0/1039398237 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcb4c15eab0 0x7fcb38004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:38.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.803+0000 7fcb50dcb640 1 --2- 172.21.15.105:0/1039398237 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb38004d40 0x7fcb380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:38.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.803+0000 7fcb50dcb640 1 -- 172.21.15.105:0/1039398237 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4c15ee90 con 0x7fcb38004d40 2024-04-15T15:20:38.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.803+0000 7fcb50dcb640 1 -- 172.21.15.105:0/1039398237 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3800b9c0 con 0x7fcb380039b0 2024-04-15T15:20:38.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.803+0000 7fcb50dcb640 1 -- 172.21.15.105:0/1039398237 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fcb3800bb40 con 0x7fcb4c15eab0 2024-04-15T15:20:38.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.803+0000 7fcb525ce640 1 --2- 172.21.15.105:0/1039398237 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb38004d40 0x7fcb380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:38.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.803+0000 7fcb515cc640 1 --2- 172.21.15.105:0/1039398237 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcb4c15eab0 0x7fcb38004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:38.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.803+0000 7fcb51dcd640 1 --2- 172.21.15.105:0/1039398237 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb380039b0 0x7fcb38003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:38.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.804+0000 7fcb525ce640 1 -- 172.21.15.105:0/1039398237 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcb4c15eab0 msgr2=0x7fcb38004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:38.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.804+0000 7fcb525ce640 1 --2- 172.21.15.105:0/1039398237 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcb4c15eab0 0x7fcb38004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:38.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.804+0000 7fcb525ce640 1 -- 172.21.15.105:0/1039398237 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb380039b0 msgr2=0x7fcb38003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:38.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.804+0000 7fcb525ce640 1 --2- 172.21.15.105:0/1039398237 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb380039b0 0x7fcb38003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:38.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.804+0000 7fcb525ce640 1 -- 172.21.15.105:0/1039398237 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb38003520 con 0x7fcb38004d40 2024-04-15T15:20:38.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.805+0000 7fcb525ce640 1 --2- 172.21.15.105:0/1039398237 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb38004d40 0x7fcb380051c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fcb440013d0 tx=0x7fcb38005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2051871d8fb5cb7a server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:38.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.806+0000 7fcb50dcb640 1 -- 172.21.15.105:0/1039398237 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4400a4e0 con 0x7fcb38004d40 2024-04-15T15:20:38.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.806+0000 7fcb50dcb640 1 -- 172.21.15.105:0/1039398237 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb44034650 con 0x7fcb38004d40 2024-04-15T15:20:38.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.806+0000 7fcb50dcb640 1 -- 172.21.15.105:0/1039398237 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb44034a40 con 0x7fcb38004d40 2024-04-15T15:20:38.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.806+0000 7fcb54031640 1 -- 172.21.15.105:0/1039398237 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb38004d40 msgr2=0x7fcb380051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:38.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.806+0000 7fcb54031640 1 --2- 172.21.15.105:0/1039398237 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb38004d40 0x7fcb380051c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fcb440013d0 tx=0x7fcb38005830 comp rx=0 tx=0).stop 2024-04-15T15:20:38.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.806+0000 7fcb525ce640 1 -- 172.21.15.105:0/1039398237 reap_dead start 2024-04-15T15:20:38.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.806+0000 7fcb54031640 1 -- 172.21.15.105:0/1039398237 shutdown_connections 2024-04-15T15:20:38.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.806+0000 7fcb54031640 1 -- 172.21.15.105:0/1039398237 >> 172.21.15.105:0/1039398237 conn(0x7fcb4c067720 msgr2=0x7fcb4c067b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:38.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.807+0000 7fcb54031640 1 -- 172.21.15.105:0/1039398237 shutdown_connections 2024-04-15T15:20:38.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.807+0000 7fcb54031640 1 -- 172.21.15.105:0/1039398237 wait complete. 2024-04-15T15:20:38.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.808+0000 7fcb54031640 1 Processor -- start 2024-04-15T15:20:38.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.808+0000 7fcb54031640 1 -- start start 2024-04-15T15:20:38.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.809+0000 7fcb54031640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcb38004d40 0x7fcb4c14ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:38.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.809+0000 7fcb54031640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb4c14fa20 0x7fcb4c0ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:38.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.809+0000 7fcb54031640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb4c0ccff0 0x7fcb4c0cdc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:38.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.809+0000 7fcb54031640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4c0ba610 con 0x7fcb4c0ccff0 2024-04-15T15:20:38.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.809+0000 7fcb54031640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4c0d3010 con 0x7fcb4c14fa20 2024-04-15T15:20:38.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.809+0000 7fcb54031640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fcb4c0ba790 con 0x7fcb38004d40 2024-04-15T15:20:38.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.809+0000 7fcb525ce640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb4c0ccff0 0x7fcb4c0cdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:38.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.809+0000 7fcb525ce640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb4c0ccff0 0x7fcb4c0cdc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45456/0 (socket says 172.21.15.105:45456) 2024-04-15T15:20:38.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.809+0000 7fcb51dcd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcb38004d40 0x7fcb4c14ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:38.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.809+0000 7fcb525ce640 1 -- 172.21.15.105:0/528546982 learned_addr learned my addr 172.21.15.105:0/528546982 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:38.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.809+0000 7fcb515cc640 1 --2- 172.21.15.105:0/528546982 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb4c14fa20 0x7fcb4c0ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:38.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.809+0000 7fcb525ce640 1 -- 172.21.15.105:0/528546982 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcb38004d40 msgr2=0x7fcb4c14ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:38.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.809+0000 7fcb525ce640 1 --2- 172.21.15.105:0/528546982 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcb38004d40 0x7fcb4c14ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:38.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.809+0000 7fcb525ce640 1 -- 172.21.15.105:0/528546982 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb4c14fa20 msgr2=0x7fcb4c0ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:38.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.809+0000 7fcb525ce640 1 --2- 172.21.15.105:0/528546982 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb4c14fa20 0x7fcb4c0ccab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:38.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.809+0000 7fcb525ce640 1 -- 172.21.15.105:0/528546982 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb4c1501a0 con 0x7fcb4c0ccff0 2024-04-15T15:20:38.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.809+0000 7fcb525ce640 1 --2- 172.21.15.105:0/528546982 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb4c0ccff0 0x7fcb4c0cdc10 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fcb4400a0b0 tx=0x7fcb44001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:38.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.809+0000 7fcb427fc640 1 -- 172.21.15.105:0/528546982 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb44039040 con 0x7fcb4c0ccff0 2024-04-15T15:20:38.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.809+0000 7fcb427fc640 1 -- 172.21.15.105:0/528546982 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb44043070 con 0x7fcb4c0ccff0 2024-04-15T15:20:38.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.809+0000 7fcb54031640 1 -- 172.21.15.105:0/528546982 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb4c0cba10 con 0x7fcb4c0ccff0 2024-04-15T15:20:38.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.809+0000 7fcb427fc640 1 -- 172.21.15.105:0/528546982 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb44004d60 con 0x7fcb4c0ccff0 2024-04-15T15:20:38.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.809+0000 7fcb54031640 1 -- 172.21.15.105:0/528546982 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb4c0cbfd0 con 0x7fcb4c0ccff0 2024-04-15T15:20:38.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.810+0000 7fcb54031640 1 -- 172.21.15.105:0/528546982 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb4c068a40 con 0x7fcb4c0ccff0 2024-04-15T15:20:38.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.811+0000 7fcb427fc640 1 -- 172.21.15.105:0/528546982 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcb44034650 con 0x7fcb4c0ccff0 2024-04-15T15:20:38.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.811+0000 7fcb427fc640 1 --2- 172.21.15.105:0/528546982 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fcb3006c920 0x7fcb3006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:38.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.812+0000 7fcb51dcd640 1 --2- 172.21.15.105:0/528546982 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fcb3006c920 0x7fcb3006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:38.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.812+0000 7fcb427fc640 1 -- 172.21.15.105:0/528546982 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fcb440ad880 con 0x7fcb4c0ccff0 2024-04-15T15:20:38.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.812+0000 7fcb51dcd640 1 --2- 172.21.15.105:0/528546982 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fcb3006c920 0x7fcb3006ede0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fcb48009bc0 tx=0x7fcb48009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:38.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:38.816+0000 7fcb427fc640 1 -- 172.21.15.105:0/528546982 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcb4407bd70 con 0x7fcb4c0ccff0 2024-04-15T15:20:39.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:39.027+0000 7fcb54031640 1 -- 172.21.15.105:0/528546982 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fcb4c0d23f0 con 0x7fcb4c0ccff0 2024-04-15T15:20:39.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:39.028+0000 7fcb427fc640 1 -- 172.21.15.105:0/528546982 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v14) ==== 53+0+73 (secure 0 0 0) 0x7fcb4407b670 con 0x7fcb4c0ccff0 2024-04-15T15:20:39.029 INFO:teuthology.orchestra.run.smithi105.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-15T15:20:39.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:39.029+0000 7fcb54031640 1 -- 172.21.15.105:0/528546982 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fcb3006c920 msgr2=0x7fcb3006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:39.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:39.029+0000 7fcb54031640 1 --2- 172.21.15.105:0/528546982 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fcb3006c920 0x7fcb3006ede0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fcb48009bc0 tx=0x7fcb48009190 comp rx=0 tx=0).stop 2024-04-15T15:20:39.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:39.029+0000 7fcb54031640 1 -- 172.21.15.105:0/528546982 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb4c0ccff0 msgr2=0x7fcb4c0cdc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:39.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:39.029+0000 7fcb54031640 1 --2- 172.21.15.105:0/528546982 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb4c0ccff0 0x7fcb4c0cdc10 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fcb4400a0b0 tx=0x7fcb44001360 comp rx=0 tx=0).stop 2024-04-15T15:20:39.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:39.030+0000 7fcb54031640 1 -- 172.21.15.105:0/528546982 shutdown_connections 2024-04-15T15:20:39.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:39.030+0000 7fcb54031640 1 --2- 172.21.15.105:0/528546982 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fcb3006c920 0x7fcb3006ede0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:39.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:39.030+0000 7fcb54031640 1 --2- 172.21.15.105:0/528546982 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb4c0ccff0 0x7fcb4c0cdc10 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:39.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:39.030+0000 7fcb54031640 1 --2- 172.21.15.105:0/528546982 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb4c14fa20 0x7fcb4c0ccab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:39.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:39.030+0000 7fcb54031640 1 --2- 172.21.15.105:0/528546982 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcb38004d40 0x7fcb4c14ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:39.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:39.030+0000 7fcb54031640 1 -- 172.21.15.105:0/528546982 >> 172.21.15.105:0/528546982 conn(0x7fcb4c067720 msgr2=0x7fcb4c0b9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:39.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:39.030+0000 7fcb54031640 1 -- 172.21.15.105:0/528546982 shutdown_connections 2024-04-15T15:20:39.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:39.030+0000 7fcb54031640 1 -- 172.21.15.105:0/528546982 wait complete. 2024-04-15T15:20:39.042 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-15T15:20:39.042 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-15T15:20:39.042 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi105.front.sepia.ceph.com 2024-04-15T15:20:39.043 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-15T15:20:39.043 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-15T15:20:39.043 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-15T15:20:39.043 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-15T15:20:39.043 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-15T15:20:39.043 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-15T15:20:39.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:39.043 DEBUG:teuthology.orchestra.run.smithi105:> ip addr 2024-04-15T15:20:39.059 INFO:teuthology.orchestra.run.smithi105.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-15T15:20:39.060 INFO:teuthology.orchestra.run.smithi105.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-15T15:20:39.060 INFO:teuthology.orchestra.run.smithi105.stdout: inet 127.0.0.1/8 scope host lo 2024-04-15T15:20:39.060 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:39.060 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 ::1/128 scope host 2024-04-15T15:20:39.060 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:39.060 INFO:teuthology.orchestra.run.smithi105.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:20:39.060 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b8 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:20:39.060 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f0 2024-04-15T15:20:39.060 INFO:teuthology.orchestra.run.smithi105.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:20:39.060 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b9 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:20:39.060 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f1 2024-04-15T15:20:39.060 INFO:teuthology.orchestra.run.smithi105.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:20:39.061 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:28 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:20:39.061 INFO:teuthology.orchestra.run.smithi105.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-15T15:20:39.061 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:29 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:20:39.061 INFO:teuthology.orchestra.run.smithi105.stdout: inet 172.21.15.105/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-15T15:20:39.061 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft 42648sec preferred_lft 42648sec 2024-04-15T15:20:39.061 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::ec4:7aff:fe8f:d429/64 scope link 2024-04-15T15:20:39.061 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:39.061 INFO:teuthology.orchestra.run.smithi105.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-15T15:20:39.061 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 12:bc:19:27:c6:bf brd ff:ff:ff:ff:ff:ff 2024-04-15T15:20:39.061 INFO:teuthology.orchestra.run.smithi105.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-15T15:20:39.061 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:39.061 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::7c5f:a9ff:fe78:3a4f/64 scope link 2024-04-15T15:20:39.061 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:39.061 INFO:teuthology.orchestra.run.smithi105.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-15T15:20:39.062 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 9e:e8:fb:17:06:9d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-15T15:20:39.062 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::9ce8:fbff:fe17:69d/64 scope link 2024-04-15T15:20:39.062 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:39.062 INFO:teuthology.orchestra.run.smithi105.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-15T15:20:39.062 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 12:bc:19:27:c6:bf brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-15T15:20:39.062 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::10bc:19ff:fe27:c6bf/64 scope link 2024-04-15T15:20:39.062 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:39.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:39.062 DEBUG:teuthology.orchestra.run.smithi105:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-15T15:20:39.127 INFO:teuthology.orchestra.run.smithi105.stdout:1 2024-04-15T15:20:39.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:39.129 DEBUG:teuthology.orchestra.run.smithi105:> route 2024-04-15T15:20:39.185 INFO:teuthology.orchestra.run.smithi105.stdout:Kernel IP routing table 2024-04-15T15:20:39.185 INFO:teuthology.orchestra.run.smithi105.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-15T15:20:39.185 INFO:teuthology.orchestra.run.smithi105.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-15T15:20:39.185 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-15T15:20:39.185 INFO:teuthology.orchestra.run.smithi105.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-15T15:20:39.186 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:20:39.186 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-04-15T15:20:39.186 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-15T15:20:39.186 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-15T15:20:39.186 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-15T15:20:39.186 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-04-15T15:20:39.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:20:39.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:20:39.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:39.324 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2024-04-15T15:20:39.339 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-15T15:20:39.339 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-15T15:20:39.340 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-15T15:20:39.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:39.340 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set brx.0 up 2024-04-15T15:20:39.408 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-15T15:20:39.408 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-15T15:20:39.408 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-15T15:20:39.463 INFO:teuthology.orchestra.run.smithi105.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-15T15:20:39.463 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-15T15:20:39.464 DEBUG:teuthology.orchestra.run.smithi105:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-15T15:20:39.519 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2024-04-15T15:20:39.587 DEBUG:teuthology.orchestra.run.smithi105:> cat /proc/self/mounts | awk '{print $2}' 2024-04-15T15:20:39.644 INFO:teuthology.orchestra.run.smithi105.stdout:/proc 2024-04-15T15:20:39.644 INFO:teuthology.orchestra.run.smithi105.stdout:/sys 2024-04-15T15:20:39.644 INFO:teuthology.orchestra.run.smithi105.stdout:/dev 2024-04-15T15:20:39.644 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/security 2024-04-15T15:20:39.645 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/shm 2024-04-15T15:20:39.645 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/pts 2024-04-15T15:20:39.645 INFO:teuthology.orchestra.run.smithi105.stdout:/run 2024-04-15T15:20:39.645 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/cgroup 2024-04-15T15:20:39.645 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/pstore 2024-04-15T15:20:39.645 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/bpf 2024-04-15T15:20:39.645 INFO:teuthology.orchestra.run.smithi105.stdout:/ 2024-04-15T15:20:39.645 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/selinux 2024-04-15T15:20:39.645 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/sys/fs/binfmt_misc 2024-04-15T15:20:39.645 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/mqueue 2024-04-15T15:20:39.645 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/hugepages 2024-04-15T15:20:39.645 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/debug 2024-04-15T15:20:39.645 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/tracing 2024-04-15T15:20:39.645 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/fuse/connections 2024-04-15T15:20:39.645 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/config 2024-04-15T15:20:39.645 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-sysctl.service 2024-04-15T15:20:39.646 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-15T15:20:39.646 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-15T15:20:39.646 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/nfs/rpc_pipefs 2024-04-15T15:20:39.646 INFO:teuthology.orchestra.run.smithi105.stdout:/run/user/1000 2024-04-15T15:20:39.646 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph 2024-04-15T15:20:39.646 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-0 2024-04-15T15:20:39.646 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-1 2024-04-15T15:20:39.646 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-2 2024-04-15T15:20:39.646 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-3 2024-04-15T15:20:39.646 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns 2024-04-15T15:20:39.646 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-15T15:20:39.646 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-15T15:20:39.646 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-15T15:20:39.647 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-15T15:20:39.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:39.647 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2024-04-15T15:20:39.700 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-15T15:20:39.700 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-15T15:20:39.743 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2024-04-15T15:20:39.771 DEBUG:teuthology.orchestra.run.smithi105:> cat /proc/self/mounts | awk '{print $2}' 2024-04-15T15:20:39.828 INFO:teuthology.orchestra.run.smithi105.stdout:/proc 2024-04-15T15:20:39.828 INFO:teuthology.orchestra.run.smithi105.stdout:/sys 2024-04-15T15:20:39.828 INFO:teuthology.orchestra.run.smithi105.stdout:/dev 2024-04-15T15:20:39.828 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/security 2024-04-15T15:20:39.828 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/shm 2024-04-15T15:20:39.828 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/pts 2024-04-15T15:20:39.828 INFO:teuthology.orchestra.run.smithi105.stdout:/run 2024-04-15T15:20:39.828 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/cgroup 2024-04-15T15:20:39.829 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/pstore 2024-04-15T15:20:39.829 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/bpf 2024-04-15T15:20:39.829 INFO:teuthology.orchestra.run.smithi105.stdout:/ 2024-04-15T15:20:39.829 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/selinux 2024-04-15T15:20:39.829 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/sys/fs/binfmt_misc 2024-04-15T15:20:39.829 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/mqueue 2024-04-15T15:20:39.829 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/hugepages 2024-04-15T15:20:39.829 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/debug 2024-04-15T15:20:39.829 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/tracing 2024-04-15T15:20:39.829 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/fuse/connections 2024-04-15T15:20:39.829 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/config 2024-04-15T15:20:39.829 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-sysctl.service 2024-04-15T15:20:39.829 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-15T15:20:39.829 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-15T15:20:39.829 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/nfs/rpc_pipefs 2024-04-15T15:20:39.829 INFO:teuthology.orchestra.run.smithi105.stdout:/run/user/1000 2024-04-15T15:20:39.830 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph 2024-04-15T15:20:39.830 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-0 2024-04-15T15:20:39.830 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-1 2024-04-15T15:20:39.830 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-2 2024-04-15T15:20:39.830 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-3 2024-04-15T15:20:39.830 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns 2024-04-15T15:20:39.830 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-15T15:20:39.830 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-15T15:20:39.830 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-15T15:20:39.830 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-15T15:20:39.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:39.830 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2024-04-15T15:20:39.832 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi105.stderr:2024-04-15T15:20:39.830+0000 7fecd0f60480 -1 init, newargv = 0x5603dc34b580 newargc=15 2024-04-15T15:20:39.832 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi105.stderr:ceph-fuse[52166]: starting ceph client 2024-04-15T15:20:39.836 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi105.stderr:ceph-fuse[52166]: starting fuse 2024-04-15T15:20:39.847 INFO:teuthology.orchestra.run.smithi105.stdout:42 2024-04-15T15:20:39.847 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-15T15:20:39.847 DEBUG:teuthology.orchestra.run.smithi105:> sudo stdin-killer -- python3 -c ' 2024-04-15T15:20:39.847 DEBUG:teuthology.orchestra.run.smithi105:> import glob 2024-04-15T15:20:39.847 DEBUG:teuthology.orchestra.run.smithi105:> import re 2024-04-15T15:20:39.847 DEBUG:teuthology.orchestra.run.smithi105:> import os 2024-04-15T15:20:39.847 DEBUG:teuthology.orchestra.run.smithi105:> import subprocess 2024-04-15T15:20:39.847 DEBUG:teuthology.orchestra.run.smithi105:> 2024-04-15T15:20:39.847 DEBUG:teuthology.orchestra.run.smithi105:> def _find_admin_socket(client_name): 2024-04-15T15:20:39.848 DEBUG:teuthology.orchestra.run.smithi105:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-15T15:20:39.848 DEBUG:teuthology.orchestra.run.smithi105:> files = glob.glob(asok_path) 2024-04-15T15:20:39.848 DEBUG:teuthology.orchestra.run.smithi105:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-15T15:20:39.848 DEBUG:teuthology.orchestra.run.smithi105:> 2024-04-15T15:20:39.848 DEBUG:teuthology.orchestra.run.smithi105:> # Given a non-glob path, it better be there 2024-04-15T15:20:39.848 DEBUG:teuthology.orchestra.run.smithi105:> if "*" not in asok_path: 2024-04-15T15:20:39.848 DEBUG:teuthology.orchestra.run.smithi105:> assert(len(files) == 1) 2024-04-15T15:20:39.848 DEBUG:teuthology.orchestra.run.smithi105:> return files[0] 2024-04-15T15:20:39.848 DEBUG:teuthology.orchestra.run.smithi105:> 2024-04-15T15:20:39.848 DEBUG:teuthology.orchestra.run.smithi105:> for f in files: 2024-04-15T15:20:39.848 DEBUG:teuthology.orchestra.run.smithi105:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-15T15:20:39.848 DEBUG:teuthology.orchestra.run.smithi105:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-15T15:20:39.848 DEBUG:teuthology.orchestra.run.smithi105:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-15T15:20:39.848 DEBUG:teuthology.orchestra.run.smithi105:> contents = proc_f.read() 2024-04-15T15:20:39.848 DEBUG:teuthology.orchestra.run.smithi105:> if mountpoint in contents: 2024-04-15T15:20:39.848 DEBUG:teuthology.orchestra.run.smithi105:> return f 2024-04-15T15:20:39.848 DEBUG:teuthology.orchestra.run.smithi105:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-15T15:20:39.849 DEBUG:teuthology.orchestra.run.smithi105:> 2024-04-15T15:20:39.849 DEBUG:teuthology.orchestra.run.smithi105:> print(_find_admin_socket("client.0")) 2024-04-15T15:20:39.849 DEBUG:teuthology.orchestra.run.smithi105:> ' 2024-04-15T15:20:39.968 INFO:teuthology.orchestra.run.smithi105.stdout:/var/run/ceph/ceph-client.0.52166.asok 2024-04-15T15:20:39.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:20:39.979 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.52166.asok 2024-04-15T15:20:39.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:39.979 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.52166.asok status 2024-04-15T15:20:40.118 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-04-15T15:20:40.118 INFO:teuthology.orchestra.run.smithi105.stdout: "metadata": { 2024-04-15T15:20:40.119 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_sha1": "9bf1c4ba967dd24077be6e5b87edda4ca87a6e14", 2024-04-15T15:20:40.119 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_version": "ceph version 19.0.0-2861-g9bf1c4ba (9bf1c4ba967dd24077be6e5b87edda4ca87a6e14) squid (dev)", 2024-04-15T15:20:40.119 INFO:teuthology.orchestra.run.smithi105.stdout: "entity_id": "0", 2024-04-15T15:20:40.119 INFO:teuthology.orchestra.run.smithi105.stdout: "hostname": "smithi105", 2024-04-15T15:20:40.119 INFO:teuthology.orchestra.run.smithi105.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-15T15:20:40.119 INFO:teuthology.orchestra.run.smithi105.stdout: "pid": "52166", 2024-04-15T15:20:40.119 INFO:teuthology.orchestra.run.smithi105.stdout: "root": "/" 2024-04-15T15:20:40.119 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-04-15T15:20:40.119 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_count": 0, 2024-04-15T15:20:40.119 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_pinned_count": 0, 2024-04-15T15:20:40.119 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 5415, 2024-04-15T15:20:40.119 INFO:teuthology.orchestra.run.smithi105.stdout: "inst": { 2024-04-15T15:20:40.119 INFO:teuthology.orchestra.run.smithi105.stdout: "name": { 2024-04-15T15:20:40.120 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "client", 2024-04-15T15:20:40.120 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 5415 2024-04-15T15:20:40.120 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-04-15T15:20:40.120 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": { 2024-04-15T15:20:40.120 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-04-15T15:20:40.120 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "192.168.0.1:0", 2024-04-15T15:20:40.120 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 3296406773 2024-04-15T15:20:40.120 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-04-15T15:20:40.120 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-04-15T15:20:40.120 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": { 2024-04-15T15:20:40.120 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-04-15T15:20:40.120 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "192.168.0.1:0", 2024-04-15T15:20:40.120 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 3296406773 2024-04-15T15:20:40.120 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-04-15T15:20:40.120 INFO:teuthology.orchestra.run.smithi105.stdout: "inst_str": "client.5415 192.168.0.1:0/3296406773", 2024-04-15T15:20:40.121 INFO:teuthology.orchestra.run.smithi105.stdout: "addr_str": "192.168.0.1:0/3296406773", 2024-04-15T15:20:40.121 INFO:teuthology.orchestra.run.smithi105.stdout: "inode_count": 1, 2024-04-15T15:20:40.121 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_epoch": 14, 2024-04-15T15:20:40.121 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_epoch": 34, 2024-04-15T15:20:40.121 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_epoch_barrier": 0, 2024-04-15T15:20:40.121 INFO:teuthology.orchestra.run.smithi105.stdout: "blocklisted": false, 2024-04-15T15:20:40.121 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs" 2024-04-15T15:20:40.121 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-04-15T15:20:40.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:40.127 DEBUG:teuthology.orchestra.run.smithi105:> stat --file-system '--printf=%T 2024-04-15T15:20:40.127 DEBUG:teuthology.orchestra.run.smithi105:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-15T15:20:40.143 INFO:teuthology.orchestra.run.smithi105.stdout:fuseblk 2024-04-15T15:20:40.143 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-15T15:20:40.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:40.144 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-15T15:20:40.212 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-15T15:20:40.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.354+0000 7f76ee2d3640 1 Processor -- start 2024-04-15T15:20:40.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.354+0000 7f76ee2d3640 1 -- start start 2024-04-15T15:20:40.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.355+0000 7f76ee2d3640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f76e806c590 0x7f76e80725b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:40.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.355+0000 7f76ee2d3640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f76e8072af0 0x7f76e815efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:40.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.355+0000 7f76ee2d3640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76e80d0ae0 con 0x7f76e8159160 2024-04-15T15:20:40.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.355+0000 7f76ee2d3640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f76e8072f60 con 0x7f76e8072af0 2024-04-15T15:20:40.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.355+0000 7f76ee2d3640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f76e806baf0 con 0x7f76e806c590 2024-04-15T15:20:40.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.355+0000 7f76ec870640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f76e8072af0 0x7f76e815efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:40.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.355+0000 7f76e7fff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f76e8159160 0x7f76e80cfdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:43782/0 (socket says 172.21.15.105:43782) 2024-04-15T15:20:40.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.355+0000 7f76ec870640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f76e8072af0 0x7f76e815efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45466/0 (socket says 172.21.15.105:45466) 2024-04-15T15:20:40.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.355+0000 7f76e7fff640 1 -- 172.21.15.105:0/1040619558 learned_addr learned my addr 172.21.15.105:0/1040619558 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:20:40.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.355+0000 7f76e77fe640 1 --2- 172.21.15.105:0/1040619558 >> v2:172.21.15.154:3300/0 conn(0x7f76e806c590 0x7f76e80725b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:40.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.355+0000 7f76ec870640 1 -- 172.21.15.105:0/1040619558 >> v2:172.21.15.154:3300/0 conn(0x7f76e806c590 msgr2=0x7f76e80725b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:40.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.355+0000 7f76ec870640 1 --2- 172.21.15.105:0/1040619558 >> v2:172.21.15.154:3300/0 conn(0x7f76e806c590 0x7f76e80725b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:40.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.355+0000 7f76ec870640 1 -- 172.21.15.105:0/1040619558 >> v1:172.21.15.105:6789/0 conn(0x7f76e8159160 legacy=0x7f76e80cfdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:40.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.355+0000 7f76ec870640 1 -- 172.21.15.105:0/1040619558 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76e80d0800 con 0x7f76e8072af0 2024-04-15T15:20:40.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.355+0000 7f76e6ffd640 1 -- 172.21.15.105:0/1040619558 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1137674320 0 0) 0x7f76e80d0ae0 con 0x7f76e8159160 2024-04-15T15:20:40.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.355+0000 7f76e77fe640 1 --2- 172.21.15.105:0/1040619558 >> v2:172.21.15.154:3300/0 conn(0x7f76e806c590 0x7f76e80725b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:40.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.356+0000 7f76ec870640 1 --2- 172.21.15.105:0/1040619558 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76e8072af0 0x7f76e815efe0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f76e809ef30 tx=0x7f76e002bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6bfa9752721893c server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:40.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.356+0000 7f76e6ffd640 1 -- 172.21.15.105:0/1040619558 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e0039070 con 0x7f76e8072af0 2024-04-15T15:20:40.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.356+0000 7f76e6ffd640 1 -- 172.21.15.105:0/1040619558 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76e002cb60 con 0x7f76e8072af0 2024-04-15T15:20:40.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.356+0000 7f76e6ffd640 1 -- 172.21.15.105:0/1040619558 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e0034440 con 0x7f76e8072af0 2024-04-15T15:20:40.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.357+0000 7f76ee2d3640 1 -- 172.21.15.105:0/1040619558 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76e8072af0 msgr2=0x7f76e815efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:40.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.357+0000 7f76ee2d3640 1 --2- 172.21.15.105:0/1040619558 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76e8072af0 0x7f76e815efe0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f76e809ef30 tx=0x7f76e002bdb0 comp rx=0 tx=0).stop 2024-04-15T15:20:40.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.357+0000 7f76ee2d3640 1 -- 172.21.15.105:0/1040619558 shutdown_connections 2024-04-15T15:20:40.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.357+0000 7f76ee2d3640 1 --2- 172.21.15.105:0/1040619558 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76e8072af0 0x7f76e815efe0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:40.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.357+0000 7f76ee2d3640 1 --2- 172.21.15.105:0/1040619558 >> v2:172.21.15.154:3300/0 conn(0x7f76e806c590 0x7f76e80725b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:40.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.357+0000 7f76ee2d3640 1 -- 172.21.15.105:0/1040619558 >> 172.21.15.105:0/1040619558 conn(0x7f76e80d0cd0 msgr2=0x7f76e80d10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:40.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.357+0000 7f76ee2d3640 1 -- 172.21.15.105:0/1040619558 shutdown_connections 2024-04-15T15:20:40.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.357+0000 7f76ee2d3640 1 -- 172.21.15.105:0/1040619558 wait complete. 2024-04-15T15:20:40.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.358+0000 7f76ee2d3640 1 Processor -- start 2024-04-15T15:20:40.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.358+0000 7f76ee2d3640 1 -- start start 2024-04-15T15:20:40.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.359+0000 7f76ee2d3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76e806c590 0x7f76e815a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:40.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.359+0000 7f76ee2d3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f76e8072af0 0x7f76e81f4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:40.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.359+0000 7f76ee2d3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76e8159160 0x7f76e80af2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:40.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.359+0000 7f76ee2d3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f76e80d0500 con 0x7f76e8159160 2024-04-15T15:20:40.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.359+0000 7f76ee2d3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f76e806c070 con 0x7f76e806c590 2024-04-15T15:20:40.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.359+0000 7f76ee2d3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f76e8070320 con 0x7f76e8072af0 2024-04-15T15:20:40.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.359+0000 7f76ec870640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76e8159160 0x7f76e80af2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:40.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.359+0000 7f76ec870640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76e8159160 0x7f76e80af2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45482/0 (socket says 172.21.15.105:45482) 2024-04-15T15:20:40.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.359+0000 7f76ec870640 1 -- 172.21.15.105:0/1250208885 learned_addr learned my addr 172.21.15.105:0/1250208885 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:40.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.359+0000 7f76e7fff640 1 --2- 172.21.15.105:0/1250208885 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76e806c590 0x7f76e815a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:40.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.359+0000 7f76e77fe640 1 --2- 172.21.15.105:0/1250208885 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f76e8072af0 0x7f76e81f4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:40.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.359+0000 7f76ec870640 1 -- 172.21.15.105:0/1250208885 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f76e8072af0 msgr2=0x7f76e81f4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:40.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.359+0000 7f76ec870640 1 --2- 172.21.15.105:0/1250208885 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f76e8072af0 0x7f76e81f4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:40.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.359+0000 7f76ec870640 1 -- 172.21.15.105:0/1250208885 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76e806c590 msgr2=0x7f76e815a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:40.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.359+0000 7f76ec870640 1 --2- 172.21.15.105:0/1250208885 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76e806c590 0x7f76e815a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:40.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.359+0000 7f76ec870640 1 -- 172.21.15.105:0/1250208885 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76e80af800 con 0x7f76e8159160 2024-04-15T15:20:40.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.359+0000 7f76ec870640 1 --2- 172.21.15.105:0/1250208885 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76e8159160 0x7f76e80af2c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f76e0037040 tx=0x7f76e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:40.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.359+0000 7f76e4ff9640 1 -- 172.21.15.105:0/1250208885 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e0039050 con 0x7f76e8159160 2024-04-15T15:20:40.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.359+0000 7f76e4ff9640 1 -- 172.21.15.105:0/1250208885 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76e003a040 con 0x7f76e8159160 2024-04-15T15:20:40.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.359+0000 7f76ee2d3640 1 -- 172.21.15.105:0/1250208885 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76e81f4160 con 0x7f76e8159160 2024-04-15T15:20:40.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.360+0000 7f76e4ff9640 1 -- 172.21.15.105:0/1250208885 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e003e850 con 0x7f76e8159160 2024-04-15T15:20:40.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.360+0000 7f76ee2d3640 1 -- 172.21.15.105:0/1250208885 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76e815ab70 con 0x7f76e8159160 2024-04-15T15:20:40.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.361+0000 7f76ee2d3640 1 -- 172.21.15.105:0/1250208885 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76e8068a40 con 0x7f76e8159160 2024-04-15T15:20:40.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.361+0000 7f76e4ff9640 1 -- 172.21.15.105:0/1250208885 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f76e0047020 con 0x7f76e8159160 2024-04-15T15:20:40.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.361+0000 7f76e4ff9640 1 --2- 172.21.15.105:0/1250208885 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f76d406c8f0 0x7f76d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:40.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.362+0000 7f76e4ff9640 1 -- 172.21.15.105:0/1250208885 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f76e00ae0a0 con 0x7f76e8159160 2024-04-15T15:20:40.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.362+0000 7f76e7fff640 1 --2- 172.21.15.105:0/1250208885 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f76d406c8f0 0x7f76d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:40.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.362+0000 7f76e7fff640 1 --2- 172.21.15.105:0/1250208885 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f76d406c8f0 0x7f76d406edb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f76d8000f60 tx=0x7f76d8007c00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:40.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.366+0000 7f76e4ff9640 1 -- 172.21.15.105:0/1250208885 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f76e004e020 con 0x7f76e8159160 2024-04-15T15:20:40.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.579+0000 7f76ee2d3640 1 -- 172.21.15.105:0/1250208885 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f76e80cff30 con 0x7f76e8159160 2024-04-15T15:20:40.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.579+0000 7f76e4ff9640 1 -- 172.21.15.105:0/1250208885 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v14) ==== 53+0+73 (secure 0 0 0) 0x7f76e007c4d0 con 0x7f76e8159160 2024-04-15T15:20:40.582 INFO:teuthology.orchestra.run.smithi105.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-15T15:20:40.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.582+0000 7f76ee2d3640 1 -- 172.21.15.105:0/1250208885 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f76d406c8f0 msgr2=0x7f76d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:40.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.582+0000 7f76ee2d3640 1 --2- 172.21.15.105:0/1250208885 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f76d406c8f0 0x7f76d406edb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f76d8000f60 tx=0x7f76d8007c00 comp rx=0 tx=0).stop 2024-04-15T15:20:40.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.582+0000 7f76ee2d3640 1 -- 172.21.15.105:0/1250208885 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76e8159160 msgr2=0x7f76e80af2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:40.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.582+0000 7f76ee2d3640 1 --2- 172.21.15.105:0/1250208885 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76e8159160 0x7f76e80af2c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f76e0037040 tx=0x7f76e0004a40 comp rx=0 tx=0).stop 2024-04-15T15:20:40.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.582+0000 7f76ee2d3640 1 -- 172.21.15.105:0/1250208885 shutdown_connections 2024-04-15T15:20:40.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.582+0000 7f76ee2d3640 1 --2- 172.21.15.105:0/1250208885 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f76e8072af0 0x7f76e81f4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:40.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.582+0000 7f76ee2d3640 1 --2- 172.21.15.105:0/1250208885 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f76d406c8f0 0x7f76d406edb0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:40.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.582+0000 7f76ee2d3640 1 --2- 172.21.15.105:0/1250208885 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76e8159160 0x7f76e80af2c0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:40.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.582+0000 7f76ee2d3640 1 --2- 172.21.15.105:0/1250208885 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76e806c590 0x7f76e815a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:40.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.582+0000 7f76ee2d3640 1 -- 172.21.15.105:0/1250208885 >> 172.21.15.105:0/1250208885 conn(0x7f76e80d0cd0 msgr2=0x7f76e80d10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:40.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.582+0000 7f76ee2d3640 1 -- 172.21.15.105:0/1250208885 shutdown_connections 2024-04-15T15:20:40.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:40.582+0000 7f76ee2d3640 1 -- 172.21.15.105:0/1250208885 wait complete. 2024-04-15T15:20:40.595 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-15T15:20:40.595 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-15T15:20:40.595 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi154.front.sepia.ceph.com 2024-04-15T15:20:40.596 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-15T15:20:40.596 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-15T15:20:40.596 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-15T15:20:40.596 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-15T15:20:40.596 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-15T15:20:40.596 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-15T15:20:40.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:40.596 DEBUG:teuthology.orchestra.run.smithi154:> ip addr 2024-04-15T15:20:40.616 INFO:teuthology.orchestra.run.smithi154.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-15T15:20:40.616 INFO:teuthology.orchestra.run.smithi154.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-15T15:20:40.617 INFO:teuthology.orchestra.run.smithi154.stdout: inet 127.0.0.1/8 scope host lo 2024-04-15T15:20:40.617 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:40.617 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 ::1/128 scope host 2024-04-15T15:20:40.617 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:40.617 INFO:teuthology.orchestra.run.smithi154.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:20:40.617 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:54 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:20:40.617 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f0 2024-04-15T15:20:40.617 INFO:teuthology.orchestra.run.smithi154.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:20:40.617 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:55 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:20:40.617 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f1 2024-04-15T15:20:40.617 INFO:teuthology.orchestra.run.smithi154.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:20:40.617 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:60 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:20:40.617 INFO:teuthology.orchestra.run.smithi154.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-15T15:20:40.618 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:61 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:20:40.618 INFO:teuthology.orchestra.run.smithi154.stdout: inet 172.21.15.154/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-15T15:20:40.618 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft 42635sec preferred_lft 42635sec 2024-04-15T15:20:40.618 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::ec4:7aff:fe88:7761/64 scope link 2024-04-15T15:20:40.618 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:40.618 INFO:teuthology.orchestra.run.smithi154.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-15T15:20:40.618 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether a2:e6:6b:6d:26:a4 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:20:40.618 INFO:teuthology.orchestra.run.smithi154.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-15T15:20:40.618 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:40.618 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::1cc6:e4ff:fe35:1850/64 scope link 2024-04-15T15:20:40.618 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:40.618 INFO:teuthology.orchestra.run.smithi154.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-15T15:20:40.618 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether ce:c3:4d:70:5f:15 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-15T15:20:40.619 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::ccc3:4dff:fe70:5f15/64 scope link 2024-04-15T15:20:40.619 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:40.619 INFO:teuthology.orchestra.run.smithi154.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-15T15:20:40.619 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether a2:e6:6b:6d:26:a4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-15T15:20:40.619 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::a0e6:6bff:fe6d:26a4/64 scope link 2024-04-15T15:20:40.619 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:40.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:40.619 DEBUG:teuthology.orchestra.run.smithi154:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-15T15:20:40.688 INFO:teuthology.orchestra.run.smithi154.stdout:1 2024-04-15T15:20:40.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:40.690 DEBUG:teuthology.orchestra.run.smithi154:> route 2024-04-15T15:20:40.750 INFO:teuthology.orchestra.run.smithi154.stdout:Kernel IP routing table 2024-04-15T15:20:40.750 INFO:teuthology.orchestra.run.smithi154.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-15T15:20:40.750 INFO:teuthology.orchestra.run.smithi154.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-15T15:20:40.750 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-15T15:20:40.750 INFO:teuthology.orchestra.run.smithi154.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-15T15:20:40.751 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:20:40.751 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2024-04-15T15:20:40.751 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-15T15:20:40.751 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-15T15:20:40.751 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-15T15:20:40.751 DEBUG:teuthology.orchestra.run.smithi154:> ') 2024-04-15T15:20:40.844 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:20:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:20:40.890 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:20:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:20:40.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:40.896 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list 2024-04-15T15:20:40.913 INFO:teuthology.orchestra.run.smithi154.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-15T15:20:40.914 INFO:teuthology.orchestra.run.smithi154.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-15T15:20:40.914 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-15T15:20:40.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:40.914 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link set brx.0 up 2024-04-15T15:20:40.983 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-15T15:20:40.983 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-15T15:20:40.983 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-15T15:20:41.037 INFO:teuthology.orchestra.run.smithi154.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-15T15:20:41.038 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-15T15:20:41.038 DEBUG:teuthology.orchestra.run.smithi154:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-15T15:20:41.092 DEBUG:teuthology.orchestra.run.smithi154:> sudo modprobe fuse 2024-04-15T15:20:41.162 DEBUG:teuthology.orchestra.run.smithi154:> cat /proc/self/mounts | awk '{print $2}' 2024-04-15T15:20:41.217 INFO:teuthology.orchestra.run.smithi154.stdout:/proc 2024-04-15T15:20:41.217 INFO:teuthology.orchestra.run.smithi154.stdout:/sys 2024-04-15T15:20:41.217 INFO:teuthology.orchestra.run.smithi154.stdout:/dev 2024-04-15T15:20:41.217 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/security 2024-04-15T15:20:41.217 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/shm 2024-04-15T15:20:41.218 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/pts 2024-04-15T15:20:41.218 INFO:teuthology.orchestra.run.smithi154.stdout:/run 2024-04-15T15:20:41.218 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/cgroup 2024-04-15T15:20:41.218 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/pstore 2024-04-15T15:20:41.218 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/bpf 2024-04-15T15:20:41.218 INFO:teuthology.orchestra.run.smithi154.stdout:/ 2024-04-15T15:20:41.218 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/selinux 2024-04-15T15:20:41.218 INFO:teuthology.orchestra.run.smithi154.stdout:/proc/sys/fs/binfmt_misc 2024-04-15T15:20:41.218 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/mqueue 2024-04-15T15:20:41.218 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/hugepages 2024-04-15T15:20:41.218 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/debug 2024-04-15T15:20:41.218 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/tracing 2024-04-15T15:20:41.218 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-15T15:20:41.219 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/fuse/connections 2024-04-15T15:20:41.219 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-sysctl.service 2024-04-15T15:20:41.219 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/config 2024-04-15T15:20:41.219 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-15T15:20:41.219 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/nfs/rpc_pipefs 2024-04-15T15:20:41.219 INFO:teuthology.orchestra.run.smithi154.stdout:/run/user/1000 2024-04-15T15:20:41.219 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph 2024-04-15T15:20:41.219 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-4 2024-04-15T15:20:41.219 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-5 2024-04-15T15:20:41.219 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-6 2024-04-15T15:20:41.219 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-7 2024-04-15T15:20:41.219 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns 2024-04-15T15:20:41.219 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-15T15:20:41.219 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-15T15:20:41.219 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-15T15:20:41.220 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-15T15:20:41.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:41.221 DEBUG:teuthology.orchestra.run.smithi154:> ls /sys/fs/fuse/connections 2024-04-15T15:20:41.275 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-15T15:20:41.276 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-15T15:20:41.318 DEBUG:teuthology.orchestra.run.smithi154:> sudo modprobe fuse 2024-04-15T15:20:41.346 DEBUG:teuthology.orchestra.run.smithi154:> cat /proc/self/mounts | awk '{print $2}' 2024-04-15T15:20:41.403 INFO:teuthology.orchestra.run.smithi154.stdout:/proc 2024-04-15T15:20:41.403 INFO:teuthology.orchestra.run.smithi154.stdout:/sys 2024-04-15T15:20:41.403 INFO:teuthology.orchestra.run.smithi154.stdout:/dev 2024-04-15T15:20:41.403 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/security 2024-04-15T15:20:41.403 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/shm 2024-04-15T15:20:41.404 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/pts 2024-04-15T15:20:41.404 INFO:teuthology.orchestra.run.smithi154.stdout:/run 2024-04-15T15:20:41.404 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/cgroup 2024-04-15T15:20:41.404 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/pstore 2024-04-15T15:20:41.404 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/bpf 2024-04-15T15:20:41.404 INFO:teuthology.orchestra.run.smithi154.stdout:/ 2024-04-15T15:20:41.404 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/selinux 2024-04-15T15:20:41.404 INFO:teuthology.orchestra.run.smithi154.stdout:/proc/sys/fs/binfmt_misc 2024-04-15T15:20:41.404 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/mqueue 2024-04-15T15:20:41.404 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/hugepages 2024-04-15T15:20:41.404 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/debug 2024-04-15T15:20:41.404 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/tracing 2024-04-15T15:20:41.404 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-15T15:20:41.404 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/fuse/connections 2024-04-15T15:20:41.404 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-sysctl.service 2024-04-15T15:20:41.405 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/config 2024-04-15T15:20:41.405 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-15T15:20:41.405 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/nfs/rpc_pipefs 2024-04-15T15:20:41.405 INFO:teuthology.orchestra.run.smithi154.stdout:/run/user/1000 2024-04-15T15:20:41.405 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph 2024-04-15T15:20:41.405 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-4 2024-04-15T15:20:41.405 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-5 2024-04-15T15:20:41.405 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-6 2024-04-15T15:20:41.405 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-7 2024-04-15T15:20:41.405 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns 2024-04-15T15:20:41.405 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-15T15:20:41.405 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-15T15:20:41.405 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-15T15:20:41.405 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-15T15:20:41.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:41.406 DEBUG:teuthology.orchestra.run.smithi154:> ls /sys/fs/fuse/connections 2024-04-15T15:20:41.408 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi154.stderr:2024-04-15T15:20:41.407+0000 7f160543a480 -1 init, newargv = 0x561b1f6b7500 newargc=15 2024-04-15T15:20:41.408 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi154.stderr:ceph-fuse[45121]: starting ceph client 2024-04-15T15:20:41.413 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi154.stderr:ceph-fuse[45121]: starting fuse 2024-04-15T15:20:41.423 INFO:teuthology.orchestra.run.smithi154.stdout:43 2024-04-15T15:20:41.423 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-04-15T15:20:41.424 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2024-04-15T15:20:41.424 DEBUG:teuthology.orchestra.run.smithi154:> import glob 2024-04-15T15:20:41.424 DEBUG:teuthology.orchestra.run.smithi154:> import re 2024-04-15T15:20:41.424 DEBUG:teuthology.orchestra.run.smithi154:> import os 2024-04-15T15:20:41.424 DEBUG:teuthology.orchestra.run.smithi154:> import subprocess 2024-04-15T15:20:41.424 DEBUG:teuthology.orchestra.run.smithi154:> 2024-04-15T15:20:41.424 DEBUG:teuthology.orchestra.run.smithi154:> def _find_admin_socket(client_name): 2024-04-15T15:20:41.424 DEBUG:teuthology.orchestra.run.smithi154:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-15T15:20:41.424 DEBUG:teuthology.orchestra.run.smithi154:> files = glob.glob(asok_path) 2024-04-15T15:20:41.424 DEBUG:teuthology.orchestra.run.smithi154:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-15T15:20:41.424 DEBUG:teuthology.orchestra.run.smithi154:> 2024-04-15T15:20:41.424 DEBUG:teuthology.orchestra.run.smithi154:> # Given a non-glob path, it better be there 2024-04-15T15:20:41.424 DEBUG:teuthology.orchestra.run.smithi154:> if "*" not in asok_path: 2024-04-15T15:20:41.424 DEBUG:teuthology.orchestra.run.smithi154:> assert(len(files) == 1) 2024-04-15T15:20:41.424 DEBUG:teuthology.orchestra.run.smithi154:> return files[0] 2024-04-15T15:20:41.425 DEBUG:teuthology.orchestra.run.smithi154:> 2024-04-15T15:20:41.425 DEBUG:teuthology.orchestra.run.smithi154:> for f in files: 2024-04-15T15:20:41.425 DEBUG:teuthology.orchestra.run.smithi154:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-15T15:20:41.425 DEBUG:teuthology.orchestra.run.smithi154:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-15T15:20:41.425 DEBUG:teuthology.orchestra.run.smithi154:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-15T15:20:41.425 DEBUG:teuthology.orchestra.run.smithi154:> contents = proc_f.read() 2024-04-15T15:20:41.425 DEBUG:teuthology.orchestra.run.smithi154:> if mountpoint in contents: 2024-04-15T15:20:41.425 DEBUG:teuthology.orchestra.run.smithi154:> return f 2024-04-15T15:20:41.425 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-15T15:20:41.425 DEBUG:teuthology.orchestra.run.smithi154:> 2024-04-15T15:20:41.425 DEBUG:teuthology.orchestra.run.smithi154:> print(_find_admin_socket("client.1")) 2024-04-15T15:20:41.425 DEBUG:teuthology.orchestra.run.smithi154:> ' 2024-04-15T15:20:41.545 INFO:teuthology.orchestra.run.smithi154.stdout:/var/run/ceph/ceph-client.1.45121.asok 2024-04-15T15:20:41.548 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:20:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:20:41.556 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.45121.asok 2024-04-15T15:20:41.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:41.556 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.45121.asok status 2024-04-15T15:20:41.695 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-04-15T15:20:41.695 INFO:teuthology.orchestra.run.smithi154.stdout: "metadata": { 2024-04-15T15:20:41.695 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_sha1": "9bf1c4ba967dd24077be6e5b87edda4ca87a6e14", 2024-04-15T15:20:41.695 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_version": "ceph version 19.0.0-2861-g9bf1c4ba (9bf1c4ba967dd24077be6e5b87edda4ca87a6e14) squid (dev)", 2024-04-15T15:20:41.695 INFO:teuthology.orchestra.run.smithi154.stdout: "entity_id": "1", 2024-04-15T15:20:41.695 INFO:teuthology.orchestra.run.smithi154.stdout: "hostname": "smithi154", 2024-04-15T15:20:41.695 INFO:teuthology.orchestra.run.smithi154.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-15T15:20:41.695 INFO:teuthology.orchestra.run.smithi154.stdout: "pid": "45121", 2024-04-15T15:20:41.696 INFO:teuthology.orchestra.run.smithi154.stdout: "root": "/" 2024-04-15T15:20:41.696 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-15T15:20:41.696 INFO:teuthology.orchestra.run.smithi154.stdout: "dentry_count": 0, 2024-04-15T15:20:41.696 INFO:teuthology.orchestra.run.smithi154.stdout: "dentry_pinned_count": 0, 2024-04-15T15:20:41.696 INFO:teuthology.orchestra.run.smithi154.stdout: "id": 4910, 2024-04-15T15:20:41.696 INFO:teuthology.orchestra.run.smithi154.stdout: "inst": { 2024-04-15T15:20:41.696 INFO:teuthology.orchestra.run.smithi154.stdout: "name": { 2024-04-15T15:20:41.696 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "client", 2024-04-15T15:20:41.696 INFO:teuthology.orchestra.run.smithi154.stdout: "num": 4910 2024-04-15T15:20:41.696 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-15T15:20:41.696 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": { 2024-04-15T15:20:41.696 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v1", 2024-04-15T15:20:41.696 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "192.168.0.1:0", 2024-04-15T15:20:41.696 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 889270462 2024-04-15T15:20:41.697 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-15T15:20:41.697 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-15T15:20:41.697 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": { 2024-04-15T15:20:41.697 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v1", 2024-04-15T15:20:41.697 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "192.168.0.1:0", 2024-04-15T15:20:41.697 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 889270462 2024-04-15T15:20:41.697 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-15T15:20:41.697 INFO:teuthology.orchestra.run.smithi154.stdout: "inst_str": "client.4910 192.168.0.1:0/889270462", 2024-04-15T15:20:41.697 INFO:teuthology.orchestra.run.smithi154.stdout: "addr_str": "192.168.0.1:0/889270462", 2024-04-15T15:20:41.697 INFO:teuthology.orchestra.run.smithi154.stdout: "inode_count": 1, 2024-04-15T15:20:41.697 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_epoch": 14, 2024-04-15T15:20:41.697 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_epoch": 34, 2024-04-15T15:20:41.697 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_epoch_barrier": 0, 2024-04-15T15:20:41.697 INFO:teuthology.orchestra.run.smithi154.stdout: "blocklisted": false, 2024-04-15T15:20:41.697 INFO:teuthology.orchestra.run.smithi154.stdout: "fs_name": "cephfs" 2024-04-15T15:20:41.698 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-04-15T15:20:41.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:41.704 DEBUG:teuthology.orchestra.run.smithi154:> stat --file-system '--printf=%T 2024-04-15T15:20:41.704 DEBUG:teuthology.orchestra.run.smithi154:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-15T15:20:41.720 INFO:teuthology.orchestra.run.smithi154.stdout:fuseblk 2024-04-15T15:20:41.720 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-15T15:20:41.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:41.720 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-15T15:20:41.789 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds debug_mds 25 2024-04-15T15:20:41.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:41.904+0000 7fdb6bb50640 1 Processor -- start 2024-04-15T15:20:41.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:41.904+0000 7fdb6bb50640 1 -- start start 2024-04-15T15:20:41.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:41.905+0000 7fdb6bb50640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdb6406f860 0x7fdb6406fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:41.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:41.905+0000 7fdb6bb50640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fdb640b3900 0x7fdb640b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:41.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:41.905+0000 7fdb6bb50640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb6406c790 con 0x7fdb64144e80 2024-04-15T15:20:41.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:41.905+0000 7fdb6bb50640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fdb640b3470 con 0x7fdb6406f860 2024-04-15T15:20:41.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:41.905+0000 7fdb6bb50640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fdb640cf7b0 con 0x7fdb640b3900 2024-04-15T15:20:41.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:41.905+0000 7fdb698ec640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fdb64144e80 0x7fdb6406dec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41126/0 (socket says 172.21.15.105:41126) 2024-04-15T15:20:41.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:41.905+0000 7fdb690eb640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdb6406f860 0x7fdb6406fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:41.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:41.905+0000 7fdb698ec640 1 -- 172.21.15.105:0/2820273049 learned_addr learned my addr 172.21.15.105:0/2820273049 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:20:41.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:41.905+0000 7fdb6a0ed640 1 --2- 172.21.15.105:0/2820273049 >> v2:172.21.15.154:3300/0 conn(0x7fdb640b3900 0x7fdb640b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:41.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:41.906+0000 7fdb688ea640 1 -- 172.21.15.105:0/2820273049 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 435094939 0 0) 0x7fdb6406c790 con 0x7fdb64144e80 2024-04-15T15:20:41.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:41.906+0000 7fdb688ea640 1 -- 172.21.15.105:0/2820273049 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdb50003520 con 0x7fdb64144e80 2024-04-15T15:20:41.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:41.906+0000 7fdb690eb640 1 -- 172.21.15.105:0/2820273049 >> v2:172.21.15.154:3300/0 conn(0x7fdb640b3900 msgr2=0x7fdb640b3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:41.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:41.906+0000 7fdb690eb640 1 --2- 172.21.15.105:0/2820273049 >> v2:172.21.15.154:3300/0 conn(0x7fdb640b3900 0x7fdb640b3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:41.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:41.906+0000 7fdb690eb640 1 -- 172.21.15.105:0/2820273049 >> v1:172.21.15.105:6789/0 conn(0x7fdb64144e80 legacy=0x7fdb6406dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:41.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:41.906+0000 7fdb690eb640 1 -- 172.21.15.105:0/2820273049 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb640cfa90 con 0x7fdb6406f860 2024-04-15T15:20:41.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:41.906+0000 7fdb690eb640 1 --2- 172.21.15.105:0/2820273049 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb6406f860 0x7fdb6406fcf0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fdb640ce050 tx=0x7fdb5402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9c13b01130f5d04 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:41.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:41.907+0000 7fdb688ea640 1 -- 172.21.15.105:0/2820273049 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb540315a0 con 0x7fdb6406f860 2024-04-15T15:20:41.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:41.907+0000 7fdb688ea640 1 -- 172.21.15.105:0/2820273049 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb54037070 con 0x7fdb6406f860 2024-04-15T15:20:41.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:41.907+0000 7fdb688ea640 1 -- 172.21.15.105:0/2820273049 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb54031960 con 0x7fdb6406f860 2024-04-15T15:20:41.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:41.907+0000 7fdb6bb50640 1 -- 172.21.15.105:0/2820273049 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb6406f860 msgr2=0x7fdb6406fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:41.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:41.907+0000 7fdb6bb50640 1 --2- 172.21.15.105:0/2820273049 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb6406f860 0x7fdb6406fcf0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fdb640ce050 tx=0x7fdb5402b840 comp rx=0 tx=0).stop 2024-04-15T15:20:41.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:41.907+0000 7fdb6bb50640 1 -- 172.21.15.105:0/2820273049 shutdown_connections 2024-04-15T15:20:41.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:41.907+0000 7fdb6bb50640 1 --2- 172.21.15.105:0/2820273049 >> v2:172.21.15.154:3300/0 conn(0x7fdb640b3900 0x7fdb640b3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:41.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:41.907+0000 7fdb6bb50640 1 --2- 172.21.15.105:0/2820273049 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb6406f860 0x7fdb6406fcf0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:41.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:41.907+0000 7fdb6bb50640 1 -- 172.21.15.105:0/2820273049 >> 172.21.15.105:0/2820273049 conn(0x7fdb6406b9c0 msgr2=0x7fdb6406bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:41.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:41.907+0000 7fdb6bb50640 1 -- 172.21.15.105:0/2820273049 shutdown_connections 2024-04-15T15:20:41.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:41.907+0000 7fdb6bb50640 1 -- 172.21.15.105:0/2820273049 wait complete. 2024-04-15T15:20:41.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:41.908+0000 7fdb6bb50640 1 Processor -- start 2024-04-15T15:20:41.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:41.909+0000 7fdb6bb50640 1 -- start start 2024-04-15T15:20:41.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:41.909+0000 7fdb6bb50640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb6406f860 0x7fdb64140590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:41.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:41.909+0000 7fdb6bb50640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdb640b3900 0x7fdb64140ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:41.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:41.909+0000 7fdb6bb50640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb64144e80 0x7fdb64141010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:41.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:41.909+0000 7fdb6bb50640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdb64145260 con 0x7fdb6406f860 2024-04-15T15:20:41.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:41.909+0000 7fdb6bb50640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdb640d0050 con 0x7fdb64144e80 2024-04-15T15:20:41.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:41.909+0000 7fdb6bb50640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fdb641453e0 con 0x7fdb640b3900 2024-04-15T15:20:41.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:41.909+0000 7fdb698ec640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb6406f860 0x7fdb64140590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:41.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:41.909+0000 7fdb698ec640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb6406f860 0x7fdb64140590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58612/0 (socket says 172.21.15.105:58612) 2024-04-15T15:20:41.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:41.909+0000 7fdb698ec640 1 -- 172.21.15.105:0/3437687039 learned_addr learned my addr 172.21.15.105:0/3437687039 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:41.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:41.910+0000 7fdb6a0ed640 1 --2- 172.21.15.105:0/3437687039 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb64144e80 0x7fdb64141010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:41.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:41.910+0000 7fdb698ec640 1 -- 172.21.15.105:0/3437687039 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdb640b3900 msgr2=0x7fdb64140ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:41.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:41.910+0000 7fdb690eb640 1 --2- 172.21.15.105:0/3437687039 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdb640b3900 0x7fdb64140ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:41.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:41.910+0000 7fdb698ec640 1 --2- 172.21.15.105:0/3437687039 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdb640b3900 0x7fdb64140ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:41.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:41.910+0000 7fdb698ec640 1 -- 172.21.15.105:0/3437687039 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb64144e80 msgr2=0x7fdb64141010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:41.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:41.910+0000 7fdb698ec640 1 --2- 172.21.15.105:0/3437687039 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb64144e80 0x7fdb64141010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:41.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:41.910+0000 7fdb698ec640 1 -- 172.21.15.105:0/3437687039 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb6413d4f0 con 0x7fdb6406f860 2024-04-15T15:20:41.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:41.910+0000 7fdb6a0ed640 1 --2- 172.21.15.105:0/3437687039 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb64144e80 0x7fdb64141010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:41.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:41.910+0000 7fdb698ec640 1 --2- 172.21.15.105:0/3437687039 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb6406f860 0x7fdb64140590 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fdb6000cad0 tx=0x7fdb6000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:41.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:41.910+0000 7fdb5a7fc640 1 -- 172.21.15.105:0/3437687039 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb60017070 con 0x7fdb6406f860 2024-04-15T15:20:41.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:41.910+0000 7fdb5a7fc640 1 -- 172.21.15.105:0/3437687039 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb6000ac30 con 0x7fdb6406f860 2024-04-15T15:20:41.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:41.910+0000 7fdb6bb50640 1 -- 172.21.15.105:0/3437687039 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb6413d7d0 con 0x7fdb6406f860 2024-04-15T15:20:41.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:41.910+0000 7fdb5a7fc640 1 -- 172.21.15.105:0/3437687039 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb60012400 con 0x7fdb6406f860 2024-04-15T15:20:41.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:41.910+0000 7fdb6bb50640 1 -- 172.21.15.105:0/3437687039 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb6413dd90 con 0x7fdb6406f860 2024-04-15T15:20:41.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:41.911+0000 7fdb6bb50640 1 -- 172.21.15.105:0/3437687039 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb64068a40 con 0x7fdb6406f860 2024-04-15T15:20:41.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:41.912+0000 7fdb5a7fc640 1 -- 172.21.15.105:0/3437687039 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdb6001a070 con 0x7fdb6406f860 2024-04-15T15:20:41.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:41.912+0000 7fdb5a7fc640 1 --2- 172.21.15.105:0/3437687039 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fdb4806c8f0 0x7fdb4806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:41.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:41.912+0000 7fdb5a7fc640 1 -- 172.21.15.105:0/3437687039 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fdb6008b910 con 0x7fdb6406f860 2024-04-15T15:20:41.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:41.912+0000 7fdb690eb640 1 --2- 172.21.15.105:0/3437687039 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fdb4806c8f0 0x7fdb4806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:41.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:41.913+0000 7fdb690eb640 1 --2- 172.21.15.105:0/3437687039 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fdb4806c8f0 0x7fdb4806edb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fdb54036000 tx=0x7fdb54008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:41.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:41.917+0000 7fdb5a7fc640 1 -- 172.21.15.105:0/3437687039 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdb60059d40 con 0x7fdb6406f860 2024-04-15T15:20:42.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.092+0000 7fdb6bb50640 1 -- 172.21.15.105:0/3437687039 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7fdb640cd0c0 con 0x7fdb6406f860 2024-04-15T15:20:42.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.095+0000 7fdb5a7fc640 1 -- 172.21.15.105:0/3437687039 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v11)=0 v11) ==== 108+0+0 (secure 0 0 0) 0x7fdb60059640 con 0x7fdb6406f860 2024-04-15T15:20:42.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.097+0000 7fdb6bb50640 1 -- 172.21.15.105:0/3437687039 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fdb4806c8f0 msgr2=0x7fdb4806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:42.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.097+0000 7fdb6bb50640 1 --2- 172.21.15.105:0/3437687039 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fdb4806c8f0 0x7fdb4806edb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fdb54036000 tx=0x7fdb54008030 comp rx=0 tx=0).stop 2024-04-15T15:20:42.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.097+0000 7fdb6bb50640 1 -- 172.21.15.105:0/3437687039 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb6406f860 msgr2=0x7fdb64140590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:42.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.097+0000 7fdb6bb50640 1 --2- 172.21.15.105:0/3437687039 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb6406f860 0x7fdb64140590 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fdb6000cad0 tx=0x7fdb6000cf90 comp rx=0 tx=0).stop 2024-04-15T15:20:42.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.097+0000 7fdb6bb50640 1 -- 172.21.15.105:0/3437687039 shutdown_connections 2024-04-15T15:20:42.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.097+0000 7fdb6bb50640 1 --2- 172.21.15.105:0/3437687039 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdb640b3900 0x7fdb64140ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:42.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.097+0000 7fdb6bb50640 1 --2- 172.21.15.105:0/3437687039 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fdb4806c8f0 0x7fdb4806edb0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:42.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.097+0000 7fdb6bb50640 1 --2- 172.21.15.105:0/3437687039 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb64144e80 0x7fdb64141010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:42.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.097+0000 7fdb6bb50640 1 --2- 172.21.15.105:0/3437687039 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb6406f860 0x7fdb64140590 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:42.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.097+0000 7fdb6bb50640 1 -- 172.21.15.105:0/3437687039 >> 172.21.15.105:0/3437687039 conn(0x7fdb6406b9c0 msgr2=0x7fdb6406f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:42.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.097+0000 7fdb6bb50640 1 -- 172.21.15.105:0/3437687039 shutdown_connections 2024-04-15T15:20:42.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.097+0000 7fdb6bb50640 1 -- 172.21.15.105:0/3437687039 wait complete. 2024-04-15T15:20:42.111 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_cache_quiesce_splitauth true 2024-04-15T15:20:42.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.213+0000 7f5547bf4640 1 Processor -- start 2024-04-15T15:20:42.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.213+0000 7f5547bf4640 1 -- start start 2024-04-15T15:20:42.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.214+0000 7f5547bf4640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5540144c90 0x7f55400b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:42.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.214+0000 7f5547bf4640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55400b40a0 con 0x7f554006ef90 2024-04-15T15:20:42.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.214+0000 7f5547bf4640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55400b4380 con 0x7f55400cd8e0 2024-04-15T15:20:42.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.214+0000 7f5547bf4640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f55400b3e50 con 0x7f5540144c90 2024-04-15T15:20:42.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.214+0000 7f554518f640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f554006ef90 0x7f55400cd1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41128/0 (socket says 172.21.15.105:41128) 2024-04-15T15:20:42.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.214+0000 7f554518f640 1 -- 172.21.15.105:0/548558835 learned_addr learned my addr 172.21.15.105:0/548558835 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:20:42.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.214+0000 7f5545990640 1 --2- 172.21.15.105:0/548558835 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5540144c90 0x7f55400b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:42.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.214+0000 7f554498e640 1 -- 172.21.15.105:0/548558835 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3692550526 0 0) 0x7f55400b40a0 con 0x7f554006ef90 2024-04-15T15:20:42.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.214+0000 7f554498e640 1 -- 172.21.15.105:0/548558835 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f552c003520 con 0x7f554006ef90 2024-04-15T15:20:42.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.214+0000 7f554498e640 1 -- 172.21.15.105:0/548558835 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1170577823 0 0) 0x7f55400b4380 con 0x7f55400cd8e0 2024-04-15T15:20:42.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.214+0000 7f554498e640 1 -- 172.21.15.105:0/548558835 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f55400b40a0 con 0x7f55400cd8e0 2024-04-15T15:20:42.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.215+0000 7f5545990640 1 -- 172.21.15.105:0/548558835 >> v1:172.21.15.154:6789/0 conn(0x7f55400cd8e0 legacy=0x7f554015bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:42.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.215+0000 7f5545990640 1 -- 172.21.15.105:0/548558835 >> v1:172.21.15.105:6789/0 conn(0x7f554006ef90 legacy=0x7f55400cd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:42.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.215+0000 7f554498e640 1 -- 172.21.15.105:0/548558835 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2256176248 0 0) 0x7f552c003520 con 0x7f554006ef90 2024-04-15T15:20:42.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.215+0000 7f5545990640 1 -- 172.21.15.105:0/548558835 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55400b4660 con 0x7f5540144c90 2024-04-15T15:20:42.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.215+0000 7f5545990640 1 --2- 172.21.15.105:0/548558835 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5540144c90 0x7f55400b4e60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f553c0097d0 tx=0x7f553c02b770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b83875d4a4a0d8ba server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:42.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.216+0000 7f554498e640 1 -- 172.21.15.105:0/548558835 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553c037070 con 0x7f5540144c90 2024-04-15T15:20:42.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.216+0000 7f554498e640 1 -- 172.21.15.105:0/548558835 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f553c030510 con 0x7f5540144c90 2024-04-15T15:20:42.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.216+0000 7f554498e640 1 -- 172.21.15.105:0/548558835 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5530001670 con 0x7f5540144c90 2024-04-15T15:20:42.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.216+0000 7f5547bf4640 1 -- 172.21.15.105:0/548558835 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5540144c90 msgr2=0x7f55400b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:42.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.216+0000 7f5547bf4640 1 --2- 172.21.15.105:0/548558835 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5540144c90 0x7f55400b4e60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f553c0097d0 tx=0x7f553c02b770 comp rx=0 tx=0).stop 2024-04-15T15:20:42.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.216+0000 7f5547bf4640 1 -- 172.21.15.105:0/548558835 shutdown_connections 2024-04-15T15:20:42.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.216+0000 7f5547bf4640 1 --2- 172.21.15.105:0/548558835 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5540144c90 0x7f55400b4e60 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:42.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.216+0000 7f5547bf4640 1 -- 172.21.15.105:0/548558835 >> 172.21.15.105:0/548558835 conn(0x7f5540067d50 msgr2=0x7f554006b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:42.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.216+0000 7f5547bf4640 1 -- 172.21.15.105:0/548558835 shutdown_connections 2024-04-15T15:20:42.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.216+0000 7f5547bf4640 1 -- 172.21.15.105:0/548558835 wait complete. 2024-04-15T15:20:42.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.217+0000 7f5547bf4640 1 Processor -- start 2024-04-15T15:20:42.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.217+0000 7f5547bf4640 1 -- start start 2024-04-15T15:20:42.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.218+0000 7f5547bf4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f554006ef90 0x7f554014ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:42.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.218+0000 7f5547bf4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55400cd8e0 0x7f55400cbf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:42.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.218+0000 7f5547bf4640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5540144c90 0x7f554015e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:42.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.218+0000 7f5547bf4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f55401451f0 con 0x7f55400cd8e0 2024-04-15T15:20:42.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.218+0000 7f5547bf4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5540145070 con 0x7f554006ef90 2024-04-15T15:20:42.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.218+0000 7f5547bf4640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f554006f3d0 con 0x7f5540144c90 2024-04-15T15:20:42.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.218+0000 7f554518f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55400cd8e0 0x7f55400cbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:42.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.218+0000 7f554518f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55400cd8e0 0x7f55400cbf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58620/0 (socket says 172.21.15.105:58620) 2024-04-15T15:20:42.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.218+0000 7f554518f640 1 -- 172.21.15.105:0/2180669769 learned_addr learned my addr 172.21.15.105:0/2180669769 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:42.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.218+0000 7f5546191640 1 --2- 172.21.15.105:0/2180669769 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5540144c90 0x7f554015e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:42.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.218+0000 7f5545990640 1 --2- 172.21.15.105:0/2180669769 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f554006ef90 0x7f554014ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:42.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.218+0000 7f554518f640 1 -- 172.21.15.105:0/2180669769 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5540144c90 msgr2=0x7f554015e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:42.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.218+0000 7f554518f640 1 --2- 172.21.15.105:0/2180669769 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5540144c90 0x7f554015e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:42.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.219+0000 7f554518f640 1 -- 172.21.15.105:0/2180669769 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f554006ef90 msgr2=0x7f554014ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:42.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.219+0000 7f554518f640 1 --2- 172.21.15.105:0/2180669769 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f554006ef90 0x7f554014ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:42.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.219+0000 7f554518f640 1 -- 172.21.15.105:0/2180669769 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f554015eb30 con 0x7f55400cd8e0 2024-04-15T15:20:42.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.219+0000 7f554518f640 1 --2- 172.21.15.105:0/2180669769 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55400cd8e0 0x7f55400cbf10 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f553000c780 tx=0x7f553000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:42.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.219+0000 7f55367fc640 1 -- 172.21.15.105:0/2180669769 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5530017070 con 0x7f55400cd8e0 2024-04-15T15:20:42.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.219+0000 7f55367fc640 1 -- 172.21.15.105:0/2180669769 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5530005af0 con 0x7f55400cd8e0 2024-04-15T15:20:42.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.219+0000 7f5547bf4640 1 -- 172.21.15.105:0/2180669769 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5540138bc0 con 0x7f55400cd8e0 2024-04-15T15:20:42.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.219+0000 7f55367fc640 1 -- 172.21.15.105:0/2180669769 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5530007070 con 0x7f55400cd8e0 2024-04-15T15:20:42.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.219+0000 7f5547bf4640 1 -- 172.21.15.105:0/2180669769 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f554015e790 con 0x7f55400cd8e0 2024-04-15T15:20:42.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.220+0000 7f5547bf4640 1 -- 172.21.15.105:0/2180669769 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f554015ee10 con 0x7f55400cd8e0 2024-04-15T15:20:42.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.220+0000 7f55367fc640 1 -- 172.21.15.105:0/2180669769 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f553001a070 con 0x7f55400cd8e0 2024-04-15T15:20:42.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.221+0000 7f55367fc640 1 --2- 172.21.15.105:0/2180669769 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f552406c8f0 0x7f552406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:42.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.221+0000 7f55367fc640 1 -- 172.21.15.105:0/2180669769 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f553008b060 con 0x7f55400cd8e0 2024-04-15T15:20:42.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.221+0000 7f5545990640 1 --2- 172.21.15.105:0/2180669769 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f552406c8f0 0x7f552406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:42.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.221+0000 7f5545990640 1 --2- 172.21.15.105:0/2180669769 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f552406c8f0 0x7f552406edb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f553c034000 tx=0x7f553c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:42.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.225+0000 7f55367fc640 1 -- 172.21.15.105:0/2180669769 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5530059520 con 0x7f55400cd8e0 2024-04-15T15:20:42.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.400+0000 7f5547bf4640 1 -- 172.21.15.105:0/2180669769 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=mds_cache_quiesce_splitauth}] v 0) -- 0x7f5540068ab0 con 0x7f55400cd8e0 2024-04-15T15:20:42.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.403+0000 7f55367fc640 1 -- 172.21.15.105:0/2180669769 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_cache_quiesce_splitauth}]=0 v12)=0 v12) ==== 128+0+0 (secure 0 0 0) 0x7f5530058e20 con 0x7f55400cd8e0 2024-04-15T15:20:42.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.406+0000 7f5547bf4640 1 -- 172.21.15.105:0/2180669769 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f552406c8f0 msgr2=0x7f552406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:42.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.406+0000 7f5547bf4640 1 --2- 172.21.15.105:0/2180669769 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f552406c8f0 0x7f552406edb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f553c034000 tx=0x7f553c008030 comp rx=0 tx=0).stop 2024-04-15T15:20:42.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.406+0000 7f5547bf4640 1 -- 172.21.15.105:0/2180669769 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55400cd8e0 msgr2=0x7f55400cbf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:42.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.406+0000 7f5547bf4640 1 --2- 172.21.15.105:0/2180669769 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55400cd8e0 0x7f55400cbf10 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f553000c780 tx=0x7f553000cc40 comp rx=0 tx=0).stop 2024-04-15T15:20:42.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.406+0000 7f5547bf4640 1 -- 172.21.15.105:0/2180669769 shutdown_connections 2024-04-15T15:20:42.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.406+0000 7f5547bf4640 1 --2- 172.21.15.105:0/2180669769 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5540144c90 0x7f554015e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:42.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.406+0000 7f5547bf4640 1 --2- 172.21.15.105:0/2180669769 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f552406c8f0 0x7f552406edb0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:42.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.406+0000 7f5547bf4640 1 --2- 172.21.15.105:0/2180669769 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55400cd8e0 0x7f55400cbf10 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:42.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.406+0000 7f5547bf4640 1 --2- 172.21.15.105:0/2180669769 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f554006ef90 0x7f554014ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:42.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.406+0000 7f5547bf4640 1 -- 172.21.15.105:0/2180669769 >> 172.21.15.105:0/2180669769 conn(0x7f5540067d50 msgr2=0x7f55400b4e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:42.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.406+0000 7f5547bf4640 1 -- 172.21.15.105:0/2180669769 shutdown_connections 2024-04-15T15:20:42.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.406+0000 7f5547bf4640 1 -- 172.21.15.105:0/2180669769 wait complete. 2024-04-15T15:20:42.420 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol_quiesce --mode=777 2024-04-15T15:20:42.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.524+0000 7f8c15b72640 1 Processor -- start 2024-04-15T15:20:42.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.524+0000 7f8c15b72640 1 -- start start 2024-04-15T15:20:42.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.525+0000 7f8c15b72640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8c1015f110 0x7f8c100af780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:42.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.525+0000 7f8c15b72640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8c100b56f0 0x7f8c10168450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:42.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.525+0000 7f8c15b72640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c100b3f20 con 0x7f8c100afcc0 2024-04-15T15:20:42.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.525+0000 7f8c15b72640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f8c100b4250 con 0x7f8c100b56f0 2024-04-15T15:20:42.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.525+0000 7f8c15b72640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f8c100b0750 con 0x7f8c1015f110 2024-04-15T15:20:42.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.525+0000 7f8c0ffff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8c100b56f0 0x7f8c10168450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:42.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.525+0000 7f8c0ffff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8c100b56f0 0x7f8c10168450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58632/0 (socket says 172.21.15.105:58632) 2024-04-15T15:20:42.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.525+0000 7f8c0ffff640 1 -- 172.21.15.105:0/3814641738 learned_addr learned my addr 172.21.15.105:0/3814641738 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:42.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.526+0000 7f8c0ffff640 1 -- 172.21.15.105:0/3814641738 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8c1015f110 msgr2=0x7f8c100af780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:42.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.526+0000 7f8c0ffff640 1 --2- 172.21.15.105:0/3814641738 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8c1015f110 0x7f8c100af780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:42.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.526+0000 7f8c0ffff640 1 -- 172.21.15.105:0/3814641738 >> v1:172.21.15.154:6789/0 conn(0x7f8c100afcc0 legacy=0x7f8c1015f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:42.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.526+0000 7f8c0ffff640 1 -- 172.21.15.105:0/3814641738 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c100b0190 con 0x7f8c100b56f0 2024-04-15T15:20:42.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.526+0000 7f8c0ffff640 1 --2- 172.21.15.105:0/3814641738 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c100b56f0 0x7f8c10168450 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8c08009870 tx=0x7f8c0802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3deef6692f56cd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:42.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.526+0000 7f8c0e7fc640 1 -- 172.21.15.105:0/3814641738 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c080317c0 con 0x7f8c100b56f0 2024-04-15T15:20:42.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.526+0000 7f8c0e7fc640 1 -- 172.21.15.105:0/3814641738 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c08037070 con 0x7f8c100b56f0 2024-04-15T15:20:42.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.527+0000 7f8c0e7fc640 1 -- 172.21.15.105:0/3814641738 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c08031b80 con 0x7f8c100b56f0 2024-04-15T15:20:42.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.527+0000 7f8c15b72640 1 -- 172.21.15.105:0/3814641738 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c100b56f0 msgr2=0x7f8c10168450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:42.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.527+0000 7f8c15b72640 1 --2- 172.21.15.105:0/3814641738 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c100b56f0 0x7f8c10168450 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8c08009870 tx=0x7f8c0802b860 comp rx=0 tx=0).stop 2024-04-15T15:20:42.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.527+0000 7f8c15b72640 1 -- 172.21.15.105:0/3814641738 shutdown_connections 2024-04-15T15:20:42.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.527+0000 7f8c15b72640 1 --2- 172.21.15.105:0/3814641738 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c100b56f0 0x7f8c10168450 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:42.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.527+0000 7f8c15b72640 1 --2- 172.21.15.105:0/3814641738 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8c1015f110 0x7f8c100af780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:42.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.527+0000 7f8c15b72640 1 -- 172.21.15.105:0/3814641738 >> 172.21.15.105:0/3814641738 conn(0x7f8c10073550 msgr2=0x7f8c1006b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:42.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.527+0000 7f8c15b72640 1 -- 172.21.15.105:0/3814641738 shutdown_connections 2024-04-15T15:20:42.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.527+0000 7f8c15b72640 1 -- 172.21.15.105:0/3814641738 wait complete. 2024-04-15T15:20:42.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.528+0000 7f8c15b72640 1 Processor -- start 2024-04-15T15:20:42.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.528+0000 7f8c15b72640 1 -- start start 2024-04-15T15:20:42.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.529+0000 7f8c15b72640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c100afcc0 0x7f8c10155d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:42.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.529+0000 7f8c15b72640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8c100b56f0 0x7f8c101f4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:42.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.529+0000 7f8c15b72640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8c1015f110 0x7f8c101f86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:42.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.529+0000 7f8c15b72640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8c100678e0 con 0x7f8c100afcc0 2024-04-15T15:20:42.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.529+0000 7f8c15b72640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1015fc00 con 0x7f8c1015f110 2024-04-15T15:20:42.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.529+0000 7f8c15b72640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f8c10067a60 con 0x7f8c100b56f0 2024-04-15T15:20:42.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.529+0000 7f8c0f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c100afcc0 0x7f8c10155d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:42.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.529+0000 7f8c0f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c100afcc0 0x7f8c10155d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58644/0 (socket says 172.21.15.105:58644) 2024-04-15T15:20:42.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.529+0000 7f8c0f7fe640 1 -- 172.21.15.105:0/404699696 learned_addr learned my addr 172.21.15.105:0/404699696 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:42.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.529+0000 7f8c0f7fe640 1 -- 172.21.15.105:0/404699696 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8c100b56f0 msgr2=0x7f8c101f4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:42.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.529+0000 7f8c0ffff640 1 --2- 172.21.15.105:0/404699696 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8c1015f110 0x7f8c101f86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:42.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.529+0000 7f8c0effd640 1 --2- 172.21.15.105:0/404699696 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8c100b56f0 0x7f8c101f4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:42.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.530+0000 7f8c0f7fe640 1 --2- 172.21.15.105:0/404699696 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8c100b56f0 0x7f8c101f4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:42.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.530+0000 7f8c0f7fe640 1 -- 172.21.15.105:0/404699696 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8c1015f110 msgr2=0x7f8c101f86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:42.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.530+0000 7f8c0f7fe640 1 --2- 172.21.15.105:0/404699696 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8c1015f110 0x7f8c101f86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:42.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.530+0000 7f8c0f7fe640 1 -- 172.21.15.105:0/404699696 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c101552c0 con 0x7f8c100afcc0 2024-04-15T15:20:42.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.530+0000 7f8c0ffff640 1 --2- 172.21.15.105:0/404699696 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8c1015f110 0x7f8c101f86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:42.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.530+0000 7f8c0effd640 1 --2- 172.21.15.105:0/404699696 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8c100b56f0 0x7f8c101f4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:42.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.530+0000 7f8c0f7fe640 1 --2- 172.21.15.105:0/404699696 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c100afcc0 0x7f8c10155d30 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8c0000ba50 tx=0x7f8c0000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:42.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.530+0000 7f8bf7fff640 1 -- 172.21.15.105:0/404699696 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c00019070 con 0x7f8c100afcc0 2024-04-15T15:20:42.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.530+0000 7f8bf7fff640 1 -- 172.21.15.105:0/404699696 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c0000cba0 con 0x7f8c100afcc0 2024-04-15T15:20:42.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.530+0000 7f8bf7fff640 1 -- 172.21.15.105:0/404699696 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c00014440 con 0x7f8c100afcc0 2024-04-15T15:20:42.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.530+0000 7f8c15b72640 1 -- 172.21.15.105:0/404699696 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c101f8c10 con 0x7f8c100afcc0 2024-04-15T15:20:42.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.530+0000 7f8c15b72640 1 -- 172.21.15.105:0/404699696 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c10155530 con 0x7f8c100afcc0 2024-04-15T15:20:42.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.531+0000 7f8c15b72640 1 -- 172.21.15.105:0/404699696 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c10068ab0 con 0x7f8c100afcc0 2024-04-15T15:20:42.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.531+0000 7f8bf7fff640 1 -- 172.21.15.105:0/404699696 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8c0001c070 con 0x7f8c100afcc0 2024-04-15T15:20:42.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.532+0000 7f8bf7fff640 1 --2- 172.21.15.105:0/404699696 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f8bfc06c8f0 0x7f8bfc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:42.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.532+0000 7f8bf7fff640 1 -- 172.21.15.105:0/404699696 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f8c0008b820 con 0x7f8c100afcc0 2024-04-15T15:20:42.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.532+0000 7f8c0effd640 1 --2- 172.21.15.105:0/404699696 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f8bfc06c8f0 0x7f8bfc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:42.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.532+0000 7f8c0effd640 1 --2- 172.21.15.105:0/404699696 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f8bfc06c8f0 0x7f8bfc06edb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8c100cff80 tx=0x7f8c04008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:42.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.537+0000 7f8bf7fff640 1 -- 172.21.15.105:0/404699696 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8c00059ce0 con 0x7f8c100afcc0 2024-04-15T15:20:42.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.738+0000 7f8c15b72640 1 -- 172.21.15.105:0/404699696 --> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol_quiesce", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f8c1006c990 con 0x7f8bfc06c8f0 2024-04-15T15:20:42.745 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-04-15T15:20:42.742+0000 7ff0f7579640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-15T15:20:42.745 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-04-15T15:20:42.742+0000 7ff0f7579640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-15T15:20:42.745 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-04-15T15:20:42.742+0000 7ff0f7579640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-15T15:20:42.745 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-04-15T15:20:42.742+0000 7ff0f7579640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-15T15:20:42.745 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-04-15T15:20:42.742+0000 7ff0f7579640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-15T15:20:42.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.774+0000 7f8bf7fff640 1 -- 172.21.15.105:0/404699696 <== mgr.4107 v2:172.21.15.105:6832/3506549322 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8c1006c990 con 0x7f8bfc06c8f0 2024-04-15T15:20:42.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.776+0000 7f8c15b72640 1 -- 172.21.15.105:0/404699696 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f8bfc06c8f0 msgr2=0x7f8bfc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:42.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.776+0000 7f8c15b72640 1 --2- 172.21.15.105:0/404699696 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f8bfc06c8f0 0x7f8bfc06edb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8c100cff80 tx=0x7f8c04008040 comp rx=0 tx=0).stop 2024-04-15T15:20:42.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.776+0000 7f8c15b72640 1 -- 172.21.15.105:0/404699696 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c100afcc0 msgr2=0x7f8c10155d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:42.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.776+0000 7f8c15b72640 1 --2- 172.21.15.105:0/404699696 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c100afcc0 0x7f8c10155d30 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8c0000ba50 tx=0x7f8c0000bf10 comp rx=0 tx=0).stop 2024-04-15T15:20:42.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.776+0000 7f8c15b72640 1 -- 172.21.15.105:0/404699696 shutdown_connections 2024-04-15T15:20:42.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.776+0000 7f8c15b72640 1 --2- 172.21.15.105:0/404699696 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8c100b56f0 0x7f8c101f4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:42.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.776+0000 7f8c15b72640 1 --2- 172.21.15.105:0/404699696 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f8bfc06c8f0 0x7f8bfc06edb0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:42.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.776+0000 7f8c15b72640 1 --2- 172.21.15.105:0/404699696 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8c1015f110 0x7f8c101f86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:42.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.776+0000 7f8c15b72640 1 --2- 172.21.15.105:0/404699696 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c100afcc0 0x7f8c10155d30 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:42.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.776+0000 7f8c15b72640 1 -- 172.21.15.105:0/404699696 >> 172.21.15.105:0/404699696 conn(0x7f8c10073550 msgr2=0x7f8c10154900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:42.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.776+0000 7f8c15b72640 1 -- 172.21.15.105:0/404699696 shutdown_connections 2024-04-15T15:20:42.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.776+0000 7f8c15b72640 1 -- 172.21.15.105:0/404699696 wait complete. 2024-04-15T15:20:42.790 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol_quiesce 2024-04-15T15:20:42.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.893+0000 7fc5bf2ba640 1 Processor -- start 2024-04-15T15:20:42.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.893+0000 7fc5bf2ba640 1 -- start start 2024-04-15T15:20:42.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.894+0000 7fc5bf2ba640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc5b80b4890 0x7fc5b80b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:42.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.894+0000 7fc5bf2ba640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc5b80b52d0 0x7fc5b80b56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:42.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.894+0000 7fc5bf2ba640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5b806ecd0 con 0x7fc5b815e9f0 2024-04-15T15:20:42.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.894+0000 7fc5bf2ba640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fc5b80b5c20 con 0x7fc5b80b4890 2024-04-15T15:20:42.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.894+0000 7fc5bf2ba640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fc5b80aee60 con 0x7fc5b80b52d0 2024-04-15T15:20:42.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.894+0000 7fc5bc855640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc5b80b4890 0x7fc5b80b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:42.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.894+0000 7fc5bc855640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc5b80b4890 0x7fc5b80b4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58668/0 (socket says 172.21.15.105:58668) 2024-04-15T15:20:42.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.894+0000 7fc5bc855640 1 -- 172.21.15.105:0/4116798258 learned_addr learned my addr 172.21.15.105:0/4116798258 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:42.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.895+0000 7fc5bc855640 1 -- 172.21.15.105:0/4116798258 >> v2:172.21.15.154:3300/0 conn(0x7fc5b80b52d0 msgr2=0x7fc5b80b56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:42.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.895+0000 7fc5bd857640 1 --2- 172.21.15.105:0/4116798258 >> v2:172.21.15.154:3300/0 conn(0x7fc5b80b52d0 0x7fc5b80b56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:42.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.895+0000 7fc5bc855640 1 --2- 172.21.15.105:0/4116798258 >> v2:172.21.15.154:3300/0 conn(0x7fc5b80b52d0 0x7fc5b80b56e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:42.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.895+0000 7fc5bc855640 1 -- 172.21.15.105:0/4116798258 >> v1:172.21.15.154:6789/0 conn(0x7fc5b815e9f0 legacy=0x7fc5b806e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:42.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.895+0000 7fc5bc855640 1 -- 172.21.15.105:0/4116798258 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5b80af140 con 0x7fc5b80b4890 2024-04-15T15:20:42.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.895+0000 7fc5bc855640 1 --2- 172.21.15.105:0/4116798258 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5b80b4890 0x7fc5b80b4d20 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fc5b80b6550 tx=0x7fc5a802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eb02ae164e02797 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:42.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.896+0000 7fc5affff640 1 -- 172.21.15.105:0/4116798258 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a802e070 con 0x7fc5b80b4890 2024-04-15T15:20:42.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.896+0000 7fc5affff640 1 -- 172.21.15.105:0/4116798258 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5a8037070 con 0x7fc5b80b4890 2024-04-15T15:20:42.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.896+0000 7fc5affff640 1 -- 172.21.15.105:0/4116798258 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a8031a00 con 0x7fc5b80b4890 2024-04-15T15:20:42.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.896+0000 7fc5bf2ba640 1 -- 172.21.15.105:0/4116798258 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5b80b4890 msgr2=0x7fc5b80b4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:42.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.896+0000 7fc5bf2ba640 1 --2- 172.21.15.105:0/4116798258 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5b80b4890 0x7fc5b80b4d20 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fc5b80b6550 tx=0x7fc5a802b880 comp rx=0 tx=0).stop 2024-04-15T15:20:42.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.896+0000 7fc5bf2ba640 1 -- 172.21.15.105:0/4116798258 shutdown_connections 2024-04-15T15:20:42.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.896+0000 7fc5bf2ba640 1 --2- 172.21.15.105:0/4116798258 >> v2:172.21.15.154:3300/0 conn(0x7fc5b80b52d0 0x7fc5b80b56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:42.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.896+0000 7fc5bf2ba640 1 --2- 172.21.15.105:0/4116798258 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5b80b4890 0x7fc5b80b4d20 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:42.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.896+0000 7fc5bf2ba640 1 -- 172.21.15.105:0/4116798258 >> 172.21.15.105:0/4116798258 conn(0x7fc5b8073570 msgr2=0x7fc5b806b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:42.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.896+0000 7fc5bf2ba640 1 -- 172.21.15.105:0/4116798258 shutdown_connections 2024-04-15T15:20:42.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.896+0000 7fc5bf2ba640 1 -- 172.21.15.105:0/4116798258 wait complete. 2024-04-15T15:20:42.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.897+0000 7fc5bf2ba640 1 Processor -- start 2024-04-15T15:20:42.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.897+0000 7fc5bf2ba640 1 -- start start 2024-04-15T15:20:42.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.898+0000 7fc5bf2ba640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc5b80b4890 0x7fc5b8153340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:42.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.898+0000 7fc5bf2ba640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5b80b52d0 0x7fc5b8154db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:42.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.898+0000 7fc5bf2ba640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc5b815e9f0 0x7fc5b815bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:42.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.898+0000 7fc5bf2ba640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b8067940 con 0x7fc5b80b52d0 2024-04-15T15:20:42.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.898+0000 7fc5bf2ba640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b80af700 con 0x7fc5b80b4890 2024-04-15T15:20:42.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.898+0000 7fc5bf2ba640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fc5b80aff00 con 0x7fc5b815e9f0 2024-04-15T15:20:42.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.898+0000 7fc5bc855640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5b80b52d0 0x7fc5b8154db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:42.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.898+0000 7fc5bc855640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5b80b52d0 0x7fc5b8154db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58676/0 (socket says 172.21.15.105:58676) 2024-04-15T15:20:42.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.898+0000 7fc5bc855640 1 -- 172.21.15.105:0/3991751777 learned_addr learned my addr 172.21.15.105:0/3991751777 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:42.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.898+0000 7fc5bc855640 1 -- 172.21.15.105:0/3991751777 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc5b815e9f0 msgr2=0x7fc5b815bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:42.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.898+0000 7fc5bc855640 1 --2- 172.21.15.105:0/3991751777 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc5b815e9f0 0x7fc5b815bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:42.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.899+0000 7fc5bc855640 1 -- 172.21.15.105:0/3991751777 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc5b80b4890 msgr2=0x7fc5b8153340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:42.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.899+0000 7fc5bd056640 1 --2- 172.21.15.105:0/3991751777 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc5b80b4890 0x7fc5b8153340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:42.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.899+0000 7fc5bc855640 1 --2- 172.21.15.105:0/3991751777 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc5b80b4890 0x7fc5b8153340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:42.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.899+0000 7fc5bc855640 1 -- 172.21.15.105:0/3991751777 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5b81522a0 con 0x7fc5b80b52d0 2024-04-15T15:20:42.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.899+0000 7fc5bc855640 1 --2- 172.21.15.105:0/3991751777 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5b80b52d0 0x7fc5b8154db0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fc5a802b8c0 tx=0x7fc5a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:42.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.899+0000 7fc5adffb640 1 -- 172.21.15.105:0/3991751777 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a8032880 con 0x7fc5b80b52d0 2024-04-15T15:20:42.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.899+0000 7fc5adffb640 1 -- 172.21.15.105:0/3991751777 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5a8032a20 con 0x7fc5b80b52d0 2024-04-15T15:20:42.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.899+0000 7fc5bf2ba640 1 -- 172.21.15.105:0/3991751777 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5b8152580 con 0x7fc5b80b52d0 2024-04-15T15:20:42.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.899+0000 7fc5adffb640 1 -- 172.21.15.105:0/3991751777 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a8032e10 con 0x7fc5b80b52d0 2024-04-15T15:20:42.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.899+0000 7fc5bf2ba640 1 -- 172.21.15.105:0/3991751777 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5b8152b40 con 0x7fc5b80b52d0 2024-04-15T15:20:42.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.900+0000 7fc5bf2ba640 1 -- 172.21.15.105:0/3991751777 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5b806cf90 con 0x7fc5b80b52d0 2024-04-15T15:20:42.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.901+0000 7fc5adffb640 1 -- 172.21.15.105:0/3991751777 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc5a80453f0 con 0x7fc5b80b52d0 2024-04-15T15:20:42.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.901+0000 7fc5adffb640 1 --2- 172.21.15.105:0/3991751777 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fc59c06c8f0 0x7fc59c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:42.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.901+0000 7fc5adffb640 1 -- 172.21.15.105:0/3991751777 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fc5a80adb30 con 0x7fc5b80b52d0 2024-04-15T15:20:42.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.901+0000 7fc5bd056640 1 --2- 172.21.15.105:0/3991751777 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fc59c06c8f0 0x7fc59c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:42.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.902+0000 7fc5bd056640 1 --2- 172.21.15.105:0/3991751777 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fc59c06c8f0 0x7fc59c06edb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc5b4007dc0 tx=0x7fc5b40072f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:42.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:42.905+0000 7fc5adffb640 1 -- 172.21.15.105:0/3991751777 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc5b806cf90 con 0x7fc5b80b52d0 2024-04-15T15:20:43.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.110+0000 7fc5bf2ba640 1 -- 172.21.15.105:0/3991751777 --> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol_quiesce", "target": ["mon-mgr", ""]}) -- 0x7fc5b806d1b0 con 0x7fc59c06c8f0 2024-04-15T15:20:43.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.114+0000 7fc5adffb640 1 -- 172.21.15.105:0/3991751777 <== mgr.4107 v2:172.21.15.105:6832/3506549322 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+70 (secure 0 0 0) 0x7fc5b806d1b0 con 0x7fc59c06c8f0 2024-04-15T15:20:43.117 INFO:teuthology.orchestra.run.smithi105.stdout:/volumes/_nogroup/subvol_quiesce/3b88d86e-6ee6-481f-8fa4-d97e9f88b65d 2024-04-15T15:20:43.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.116+0000 7fc5bf2ba640 1 -- 172.21.15.105:0/3991751777 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fc59c06c8f0 msgr2=0x7fc59c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:43.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.116+0000 7fc5bf2ba640 1 --2- 172.21.15.105:0/3991751777 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fc59c06c8f0 0x7fc59c06edb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc5b4007dc0 tx=0x7fc5b40072f0 comp rx=0 tx=0).stop 2024-04-15T15:20:43.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.116+0000 7fc5bf2ba640 1 -- 172.21.15.105:0/3991751777 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5b80b52d0 msgr2=0x7fc5b8154db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:43.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.116+0000 7fc5bf2ba640 1 --2- 172.21.15.105:0/3991751777 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5b80b52d0 0x7fc5b8154db0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fc5a802b8c0 tx=0x7fc5a80046c0 comp rx=0 tx=0).stop 2024-04-15T15:20:43.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.116+0000 7fc5bf2ba640 1 -- 172.21.15.105:0/3991751777 shutdown_connections 2024-04-15T15:20:43.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.116+0000 7fc5bf2ba640 1 --2- 172.21.15.105:0/3991751777 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc5b815e9f0 0x7fc5b815bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:43.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.116+0000 7fc5bf2ba640 1 --2- 172.21.15.105:0/3991751777 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fc59c06c8f0 0x7fc59c06edb0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:43.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.116+0000 7fc5bf2ba640 1 --2- 172.21.15.105:0/3991751777 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5b80b52d0 0x7fc5b8154db0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:43.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.116+0000 7fc5bf2ba640 1 --2- 172.21.15.105:0/3991751777 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc5b80b4890 0x7fc5b8153340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:43.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.116+0000 7fc5bf2ba640 1 -- 172.21.15.105:0/3991751777 >> 172.21.15.105:0/3991751777 conn(0x7fc5b8073570 msgr2=0x7fc5b80b4120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:43.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.116+0000 7fc5bf2ba640 1 -- 172.21.15.105:0/3991751777 shutdown_connections 2024-04-15T15:20:43.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.116+0000 7fc5bf2ba640 1 -- 172.21.15.105:0/3991751777 wait complete. 2024-04-15T15:20:43.130 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-04-15T15:20:43.130 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:20:43.147 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:20:43.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.287+0000 7f2b4bbe2640 1 Processor -- start 2024-04-15T15:20:43.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.287+0000 7f2b4bbe2640 1 -- start start 2024-04-15T15:20:43.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.288+0000 7f2b4bbe2640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2b440725f0 0x7f2b4406c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:43.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.288+0000 7f2b4bbe2640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2b440b5ca0 0x7f2b4415cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:43.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.288+0000 7f2b4bbe2640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b440d0260 con 0x7f2b440cfe00 2024-04-15T15:20:43.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.288+0000 7f2b4bbe2640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f2b440d0ad0 con 0x7f2b440725f0 2024-04-15T15:20:43.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.288+0000 7f2b4bbe2640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f2b440d0510 con 0x7f2b440b5ca0 2024-04-15T15:20:43.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.288+0000 7f2b4997e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2b440725f0 0x7f2b4406c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:43.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.288+0000 7f2b4917d640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f2b440cfe00 0x7f2b440729d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41134/0 (socket says 172.21.15.105:41134) 2024-04-15T15:20:43.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.288+0000 7f2b4917d640 1 -- 172.21.15.105:0/3507680817 learned_addr learned my addr 172.21.15.105:0/3507680817 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:20:43.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.288+0000 7f2b4a17f640 1 --2- 172.21.15.105:0/3507680817 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2b440b5ca0 0x7f2b4415cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:43.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.288+0000 7f2b4997e640 1 -- 172.21.15.105:0/3507680817 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2b440b5ca0 msgr2=0x7f2b4415cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:43.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.288+0000 7f2b4997e640 1 --2- 172.21.15.105:0/3507680817 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2b440b5ca0 0x7f2b4415cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:43.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.288+0000 7f2b4997e640 1 -- 172.21.15.105:0/3507680817 >> v1:172.21.15.105:6789/0 conn(0x7f2b440cfe00 legacy=0x7f2b440729d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:43.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.288+0000 7f2b4897c640 1 -- 172.21.15.105:0/3507680817 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3370750311 0 0) 0x7f2b440d0260 con 0x7f2b440cfe00 2024-04-15T15:20:43.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.288+0000 7f2b4997e640 1 -- 172.21.15.105:0/3507680817 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b440d07f0 con 0x7f2b440725f0 2024-04-15T15:20:43.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.288+0000 7f2b4a17f640 1 --2- 172.21.15.105:0/3507680817 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2b440b5ca0 0x7f2b4415cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:43.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.289+0000 7f2b4997e640 1 --2- 172.21.15.105:0/3507680817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2b440725f0 0x7f2b4406c3d0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f2b400097a0 tx=0x7f2b4002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5591a47c43469fdc server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:43.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.289+0000 7f2b4897c640 1 -- 172.21.15.105:0/3507680817 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b40035020 con 0x7f2b440725f0 2024-04-15T15:20:43.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.289+0000 7f2b4897c640 1 -- 172.21.15.105:0/3507680817 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b40031830 con 0x7f2b440725f0 2024-04-15T15:20:43.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.289+0000 7f2b4897c640 1 -- 172.21.15.105:0/3507680817 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b40031c00 con 0x7f2b440725f0 2024-04-15T15:20:43.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.290+0000 7f2b4bbe2640 1 -- 172.21.15.105:0/3507680817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2b440725f0 msgr2=0x7f2b4406c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:43.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.290+0000 7f2b4bbe2640 1 --2- 172.21.15.105:0/3507680817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2b440725f0 0x7f2b4406c3d0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f2b400097a0 tx=0x7f2b4002b8d0 comp rx=0 tx=0).stop 2024-04-15T15:20:43.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.290+0000 7f2b4bbe2640 1 -- 172.21.15.105:0/3507680817 shutdown_connections 2024-04-15T15:20:43.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.290+0000 7f2b4bbe2640 1 --2- 172.21.15.105:0/3507680817 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2b440b5ca0 0x7f2b4415cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:43.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.290+0000 7f2b4bbe2640 1 --2- 172.21.15.105:0/3507680817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2b440725f0 0x7f2b4406c3d0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:43.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.290+0000 7f2b4bbe2640 1 -- 172.21.15.105:0/3507680817 >> 172.21.15.105:0/3507680817 conn(0x7f2b440d0d00 msgr2=0x7f2b440d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:43.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.290+0000 7f2b4bbe2640 1 -- 172.21.15.105:0/3507680817 shutdown_connections 2024-04-15T15:20:43.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.290+0000 7f2b4bbe2640 1 -- 172.21.15.105:0/3507680817 wait complete. 2024-04-15T15:20:43.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.291+0000 7f2b4bbe2640 1 Processor -- start 2024-04-15T15:20:43.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.291+0000 7f2b4bbe2640 1 -- start start 2024-04-15T15:20:43.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.292+0000 7f2b4bbe2640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2b440725f0 0x7f2b4414e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:43.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.292+0000 7f2b4bbe2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b440b5ca0 0x7f2b441f33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:43.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.292+0000 7f2b4bbe2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2b440cfe00 0x7f2b4415ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:43.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.292+0000 7f2b4bbe2640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2b440b0680 con 0x7f2b440cfe00 2024-04-15T15:20:43.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.292+0000 7f2b4bbe2640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2b440b5780 con 0x7f2b440b5ca0 2024-04-15T15:20:43.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.292+0000 7f2b4bbe2640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f2b440b0800 con 0x7f2b440725f0 2024-04-15T15:20:43.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.292+0000 7f2b4a17f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2b440cfe00 0x7f2b4415ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:43.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.292+0000 7f2b4a17f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2b440cfe00 0x7f2b4415ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58698/0 (socket says 172.21.15.105:58698) 2024-04-15T15:20:43.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.292+0000 7f2b4a17f640 1 -- 172.21.15.105:0/1759559023 learned_addr learned my addr 172.21.15.105:0/1759559023 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:43.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.292+0000 7f2b4997e640 1 --2- 172.21.15.105:0/1759559023 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2b440725f0 0x7f2b4414e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:43.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.292+0000 7f2b4917d640 1 --2- 172.21.15.105:0/1759559023 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b440b5ca0 0x7f2b441f33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:43.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.292+0000 7f2b4a17f640 1 -- 172.21.15.105:0/1759559023 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2b440725f0 msgr2=0x7f2b4414e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:43.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.292+0000 7f2b4a17f640 1 --2- 172.21.15.105:0/1759559023 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2b440725f0 0x7f2b4414e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:43.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.292+0000 7f2b4a17f640 1 -- 172.21.15.105:0/1759559023 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b440b5ca0 msgr2=0x7f2b441f33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:43.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.292+0000 7f2b4a17f640 1 --2- 172.21.15.105:0/1759559023 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b440b5ca0 0x7f2b441f33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:43.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.292+0000 7f2b4a17f640 1 -- 172.21.15.105:0/1759559023 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b4414eea0 con 0x7f2b440cfe00 2024-04-15T15:20:43.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.292+0000 7f2b4a17f640 1 --2- 172.21.15.105:0/1759559023 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2b440cfe00 0x7f2b4415ea10 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c00cf20 tx=0x7f2b3c00a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:43.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.293+0000 7f2b3a7fc640 1 -- 172.21.15.105:0/1759559023 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b3c01a070 con 0x7f2b440cfe00 2024-04-15T15:20:43.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.293+0000 7f2b3a7fc640 1 -- 172.21.15.105:0/1759559023 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b3c00d840 con 0x7f2b440cfe00 2024-04-15T15:20:43.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.293+0000 7f2b3a7fc640 1 -- 172.21.15.105:0/1759559023 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b3c00dc30 con 0x7f2b440cfe00 2024-04-15T15:20:43.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.293+0000 7f2b4bbe2640 1 -- 172.21.15.105:0/1759559023 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b4414f140 con 0x7f2b440cfe00 2024-04-15T15:20:43.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.293+0000 7f2b4bbe2640 1 -- 172.21.15.105:0/1759559023 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b4414f6c0 con 0x7f2b440cfe00 2024-04-15T15:20:43.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.294+0000 7f2b4bbe2640 1 -- 172.21.15.105:0/1759559023 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b4414f8c0 con 0x7f2b440cfe00 2024-04-15T15:20:43.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.294+0000 7f2b3a7fc640 1 -- 172.21.15.105:0/1759559023 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2b3c01d070 con 0x7f2b440cfe00 2024-04-15T15:20:43.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.294+0000 7f2b3a7fc640 1 --2- 172.21.15.105:0/1759559023 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f2b2806c8f0 0x7f2b2806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:43.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.295+0000 7f2b4997e640 1 --2- 172.21.15.105:0/1759559023 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f2b2806c8f0 0x7f2b2806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:43.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.295+0000 7f2b3a7fc640 1 -- 172.21.15.105:0/1759559023 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f2b3c08c410 con 0x7f2b440cfe00 2024-04-15T15:20:43.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.295+0000 7f2b4997e640 1 --2- 172.21.15.105:0/1759559023 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f2b2806c8f0 0x7f2b2806edb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2b40036000 tx=0x7f2b40008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:43.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.299+0000 7f2b3a7fc640 1 -- 172.21.15.105:0/1759559023 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2b4414f8c0 con 0x7f2b440cfe00 2024-04-15T15:20:43.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.473+0000 7f2b4bbe2640 1 -- 172.21.15.105:0/1759559023 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2b44068a40 con 0x7f2b440cfe00 2024-04-15T15:20:43.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.475+0000 7f2b3a7fc640 1 -- 172.21.15.105:0/1759559023 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16717 (secure 0 0 0) 0x7f2b4414eea0 con 0x7f2b440cfe00 2024-04-15T15:20:43.478 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:20:43.478 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":34,"fsid":"09e4d2a4-a324-4793-af77-644b25e520e1","created":"2024-04-15T15:19:01.524543+0000","modified":"2024-04-15T15:20:30.645565+0000","last_up_change":"2024-04-15T15:19:12.825815+0000","last_in_change":"2024-04-15T15:19:03.788126+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:19:13.748585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:20:26.983353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-15T15:20:27.961802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6f6292b-2d7e-4aa0-9d1c-6ddd0f97287c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6801","nonce":2873040054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6803","nonce":2873040054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6807","nonce":2873040054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6805","nonce":2873040054}]},"public_addr":"172.21.15.105:6801/2873040054","cluster_addr":"172.21.15.105:6803/2873040054","heartbeat_back_addr":"172.21.15.105:6807/2873040054","heartbeat_front_addr":"172.21.15.105:6805/2873040054","state":["exists","up"]},{"osd":1,"uuid":"8900966a-b393-41e3-8f93-e5a43bd23622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6818","nonce":2494994479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6821","nonce":2494994479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6828","nonce":2494994479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6823","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6825","nonce":2494994479}]},"public_addr":"172.21.15.105:6818/2494994479","cluster_addr":"172.21.15.105:6821/2494994479","heartbeat_back_addr":"172.21.15.105:6828/2494994479","heartbeat_front_addr":"172.21.15.105:6825/2494994479","state":["exists","up"]},{"osd":2,"uuid":"81a5dc7a-9325-4ff5-941d-a2d5fba21f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6809","nonce":3798408699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6811","nonce":3798408699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6815","nonce":3798408699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6813","nonce":3798408699}]},"public_addr":"172.21.15.105:6809/3798408699","cluster_addr":"172.21.15.105:6811/3798408699","heartbeat_back_addr":"172.21.15.105:6815/3798408699","heartbeat_front_addr":"172.21.15.105:6813/3798408699","state":["exists","up"]},{"osd":3,"uuid":"df766ed2-5c71-4805-96f4-3d4970edc960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6819","nonce":1920720186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6824","nonce":1920720186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6831","nonce":1920720186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6827","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6829","nonce":1920720186}]},"public_addr":"172.21.15.105:6819/1920720186","cluster_addr":"172.21.15.105:6824/1920720186","heartbeat_back_addr":"172.21.15.105:6831/1920720186","heartbeat_front_addr":"172.21.15.105:6829/1920720186","state":["exists","up"]},{"osd":4,"uuid":"e1039e03-28b8-49df-b951-c91930b93c9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6810","nonce":3516494456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6813","nonce":3516494456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6820","nonce":3516494456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6817","nonce":3516494456}]},"public_addr":"172.21.15.154:6810/3516494456","cluster_addr":"172.21.15.154:6813/3516494456","heartbeat_back_addr":"172.21.15.154:6820/3516494456","heartbeat_front_addr":"172.21.15.154:6817/3516494456","state":["exists","up"]},{"osd":5,"uuid":"a9f5eaf1-66ff-405a-a9e4-ec3340812a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6801","nonce":540367751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6803","nonce":540367751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6807","nonce":540367751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6805","nonce":540367751}]},"public_addr":"172.21.15.154:6801/540367751","cluster_addr":"172.21.15.154:6803/540367751","heartbeat_back_addr":"172.21.15.154:6807/540367751","heartbeat_front_addr":"172.21.15.154:6805/540367751","state":["exists","up"]},{"osd":6,"uuid":"267d02ea-675f-428f-aae8-e0375e4d4feb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6825","nonce":1550829005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6827","nonce":1550829005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6831","nonce":1550829005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6829","nonce":1550829005}]},"public_addr":"172.21.15.154:6825/1550829005","cluster_addr":"172.21.15.154:6827/1550829005","heartbeat_back_addr":"172.21.15.154:6831/1550829005","heartbeat_front_addr":"172.21.15.154:6829/1550829005","state":["exists","up"]},{"osd":7,"uuid":"759ac6cc-896a-423c-885f-9fe90d8dcb14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6809","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6811","nonce":3920089903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6816","nonce":3920089903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6823","nonce":3920089903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6819","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6821","nonce":3920089903}]},"public_addr":"172.21.15.154:6811/3920089903","cluster_addr":"172.21.15.154:6816/3920089903","heartbeat_back_addr":"172.21.15.154:6823/3920089903","heartbeat_front_addr":"172.21.15.154:6821/3920089903","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.949509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.984072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.907278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.947391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.975447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.998896+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:09.028739+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.894119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:20:43.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.477+0000 7f2b4bbe2640 1 -- 172.21.15.105:0/1759559023 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f2b2806c8f0 msgr2=0x7f2b2806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:43.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.477+0000 7f2b4bbe2640 1 --2- 172.21.15.105:0/1759559023 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f2b2806c8f0 0x7f2b2806edb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2b40036000 tx=0x7f2b40008030 comp rx=0 tx=0).stop 2024-04-15T15:20:43.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.477+0000 7f2b4bbe2640 1 -- 172.21.15.105:0/1759559023 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2b440cfe00 msgr2=0x7f2b4415ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:43.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.477+0000 7f2b4bbe2640 1 --2- 172.21.15.105:0/1759559023 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2b440cfe00 0x7f2b4415ea10 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c00cf20 tx=0x7f2b3c00a7d0 comp rx=0 tx=0).stop 2024-04-15T15:20:43.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.477+0000 7f2b4bbe2640 1 -- 172.21.15.105:0/1759559023 shutdown_connections 2024-04-15T15:20:43.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.477+0000 7f2b4bbe2640 1 --2- 172.21.15.105:0/1759559023 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f2b2806c8f0 0x7f2b2806edb0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:43.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.477+0000 7f2b4bbe2640 1 --2- 172.21.15.105:0/1759559023 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2b440cfe00 0x7f2b4415ea10 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:43.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.477+0000 7f2b4bbe2640 1 --2- 172.21.15.105:0/1759559023 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b440b5ca0 0x7f2b441f33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:43.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.477+0000 7f2b4bbe2640 1 --2- 172.21.15.105:0/1759559023 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2b440725f0 0x7f2b4414e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:43.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.477+0000 7f2b4bbe2640 1 -- 172.21.15.105:0/1759559023 >> 172.21.15.105:0/1759559023 conn(0x7f2b440d0d00 msgr2=0x7f2b440729d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:43.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.477+0000 7f2b4bbe2640 1 -- 172.21.15.105:0/1759559023 shutdown_connections 2024-04-15T15:20:43.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.477+0000 7f2b4bbe2640 1 -- 172.21.15.105:0/1759559023 wait complete. 2024-04-15T15:20:43.491 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:19:13.748585+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-15T15:20:26.983353+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-15T15:20:27.961802+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:20:43.491 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:20:43.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.596+0000 7fde0e4b3640 1 Processor -- start 2024-04-15T15:20:43.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.596+0000 7fde0e4b3640 1 -- start start 2024-04-15T15:20:43.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.596+0000 7fde0e4b3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fde0815e8d0 0x7fde080d2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:43.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.596+0000 7fde0e4b3640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fde080ba480 0x7fde080d1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:43.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.596+0000 7fde0e4b3640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde080d3400 con 0x7fde080d1650 2024-04-15T15:20:43.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.596+0000 7fde0e4b3640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fde080d1ac0 con 0x7fde080ba480 2024-04-15T15:20:43.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.596+0000 7fde0e4b3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fde080d36e0 con 0x7fde0815e8d0 2024-04-15T15:20:43.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.596+0000 7fddfffff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fde080ba480 0x7fde080d1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:43.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.596+0000 7fde0ca50640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fde080d1650 0x7fde080b61d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41148/0 (socket says 172.21.15.105:41148) 2024-04-15T15:20:43.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.596+0000 7fddfffff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fde080ba480 0x7fde080d1110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58700/0 (socket says 172.21.15.105:58700) 2024-04-15T15:20:43.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.596+0000 7fde0ca50640 1 -- 172.21.15.105:0/2074373215 learned_addr learned my addr 172.21.15.105:0/2074373215 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:20:43.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.596+0000 7fde07fff640 1 --2- 172.21.15.105:0/2074373215 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fde0815e8d0 0x7fde080d2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:43.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.596+0000 7fddfffff640 1 -- 172.21.15.105:0/2074373215 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fde0815e8d0 msgr2=0x7fde080d2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:43.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.596+0000 7fddfffff640 1 --2- 172.21.15.105:0/2074373215 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fde0815e8d0 0x7fde080d2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:43.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.596+0000 7fddfffff640 1 -- 172.21.15.105:0/2074373215 >> v1:172.21.15.105:6789/0 conn(0x7fde080d1650 legacy=0x7fde080b61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:43.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.596+0000 7fddfffff640 1 -- 172.21.15.105:0/2074373215 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde080d39c0 con 0x7fde080ba480 2024-04-15T15:20:43.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.596+0000 7fde077fe640 1 -- 172.21.15.105:0/2074373215 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1320359113 0 0) 0x7fde080d3400 con 0x7fde080d1650 2024-04-15T15:20:43.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.597+0000 7fddfffff640 1 --2- 172.21.15.105:0/2074373215 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde080ba480 0x7fde080d1110 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fddf00097a0 tx=0x7fddf002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=509a02849bcfe113 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:43.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.597+0000 7fde077fe640 1 -- 172.21.15.105:0/2074373215 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf0034070 con 0x7fde080ba480 2024-04-15T15:20:43.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.597+0000 7fde077fe640 1 -- 172.21.15.105:0/2074373215 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddf0030610 con 0x7fde080ba480 2024-04-15T15:20:43.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.597+0000 7fde077fe640 1 -- 172.21.15.105:0/2074373215 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf00309e0 con 0x7fde080ba480 2024-04-15T15:20:43.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.597+0000 7fde0e4b3640 1 -- 172.21.15.105:0/2074373215 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde080ba480 msgr2=0x7fde080d1110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:43.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.597+0000 7fde0e4b3640 1 --2- 172.21.15.105:0/2074373215 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde080ba480 0x7fde080d1110 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fddf00097a0 tx=0x7fddf002b960 comp rx=0 tx=0).stop 2024-04-15T15:20:43.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.597+0000 7fde0e4b3640 1 -- 172.21.15.105:0/2074373215 shutdown_connections 2024-04-15T15:20:43.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.597+0000 7fde0e4b3640 1 --2- 172.21.15.105:0/2074373215 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde080ba480 0x7fde080d1110 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:43.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.597+0000 7fde0e4b3640 1 --2- 172.21.15.105:0/2074373215 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fde0815e8d0 0x7fde080d2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:43.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.597+0000 7fde0e4b3640 1 -- 172.21.15.105:0/2074373215 >> 172.21.15.105:0/2074373215 conn(0x7fde0806d460 msgr2=0x7fde08067720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:43.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.598+0000 7fde0e4b3640 1 -- 172.21.15.105:0/2074373215 shutdown_connections 2024-04-15T15:20:43.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.598+0000 7fde0e4b3640 1 -- 172.21.15.105:0/2074373215 wait complete. 2024-04-15T15:20:43.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.599+0000 7fde0e4b3640 1 Processor -- start 2024-04-15T15:20:43.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.599+0000 7fde0e4b3640 1 -- start start 2024-04-15T15:20:43.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.599+0000 7fde0e4b3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fde080ba480 0x7fde080abda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:43.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.599+0000 7fde0e4b3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde080d1650 0x7fde080cc230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:43.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.599+0000 7fde0e4b3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde0815e8d0 0x7fde080cc770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:43.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.599+0000 7fde0e4b3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fde080d44a0 con 0x7fde080d1650 2024-04-15T15:20:43.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.599+0000 7fde0e4b3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fde080d3fc0 con 0x7fde0815e8d0 2024-04-15T15:20:43.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.599+0000 7fde0e4b3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fde080b8890 con 0x7fde080ba480 2024-04-15T15:20:43.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.599+0000 7fddfffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde080d1650 0x7fde080cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:43.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.599+0000 7fddfffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde080d1650 0x7fde080cc230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58716/0 (socket says 172.21.15.105:58716) 2024-04-15T15:20:43.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.599+0000 7fddfffff640 1 -- 172.21.15.105:0/2627008266 learned_addr learned my addr 172.21.15.105:0/2627008266 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:43.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.600+0000 7fde07fff640 1 --2- 172.21.15.105:0/2627008266 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fde080ba480 0x7fde080abda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:43.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.600+0000 7fde0ca50640 1 --2- 172.21.15.105:0/2627008266 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde0815e8d0 0x7fde080cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:43.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.600+0000 7fddfffff640 1 -- 172.21.15.105:0/2627008266 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fde080ba480 msgr2=0x7fde080abda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:43.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.600+0000 7fddfffff640 1 --2- 172.21.15.105:0/2627008266 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fde080ba480 0x7fde080abda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:43.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.600+0000 7fddfffff640 1 -- 172.21.15.105:0/2627008266 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde0815e8d0 msgr2=0x7fde080cc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:43.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.600+0000 7fddfffff640 1 --2- 172.21.15.105:0/2627008266 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde0815e8d0 0x7fde080cc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:43.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.600+0000 7fddfffff640 1 -- 172.21.15.105:0/2627008266 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde080cccb0 con 0x7fde080d1650 2024-04-15T15:20:43.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.600+0000 7fddfffff640 1 --2- 172.21.15.105:0/2627008266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde080d1650 0x7fde080cc230 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fddf0009890 tx=0x7fddf00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:43.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.600+0000 7fde057fa640 1 -- 172.21.15.105:0/2627008266 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf0034040 con 0x7fde080d1650 2024-04-15T15:20:43.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.600+0000 7fde057fa640 1 -- 172.21.15.105:0/2627008266 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddf003acc0 con 0x7fde080d1650 2024-04-15T15:20:43.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.600+0000 7fde0e4b3640 1 -- 172.21.15.105:0/2627008266 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde080ccf60 con 0x7fde080d1650 2024-04-15T15:20:43.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.600+0000 7fde057fa640 1 -- 172.21.15.105:0/2627008266 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf0044630 con 0x7fde080d1650 2024-04-15T15:20:43.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.600+0000 7fde0e4b3640 1 -- 172.21.15.105:0/2627008266 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde080cd980 con 0x7fde080d1650 2024-04-15T15:20:43.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.601+0000 7fde0e4b3640 1 -- 172.21.15.105:0/2627008266 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde08068ab0 con 0x7fde080d1650 2024-04-15T15:20:43.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.602+0000 7fde057fa640 1 -- 172.21.15.105:0/2627008266 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fddf004b060 con 0x7fde080d1650 2024-04-15T15:20:43.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.602+0000 7fde057fa640 1 --2- 172.21.15.105:0/2627008266 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fdde806c8f0 0x7fdde806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:43.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.602+0000 7fde057fa640 1 -- 172.21.15.105:0/2627008266 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fddf00ad8b0 con 0x7fde080d1650 2024-04-15T15:20:43.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.602+0000 7fde07fff640 1 --2- 172.21.15.105:0/2627008266 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fdde806c8f0 0x7fdde806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:43.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.602+0000 7fde07fff640 1 --2- 172.21.15.105:0/2627008266 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fdde806c8f0 0x7fdde806edb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fddf8006fd0 tx=0x7fddf8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:43.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.607+0000 7fde057fa640 1 -- 172.21.15.105:0/2627008266 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fddf003f070 con 0x7fde080d1650 2024-04-15T15:20:43.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.785+0000 7fde0e4b3640 1 -- 172.21.15.105:0/2627008266 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fde080ce150 con 0x7fde080d1650 2024-04-15T15:20:43.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.786+0000 7fde057fa640 1 -- 172.21.15.105:0/2627008266 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) ==== 93+0+10 (secure 0 0 0) 0x7fddf007bd70 con 0x7fde080d1650 2024-04-15T15:20:43.787 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-04-15T15:20:43.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.787+0000 7fde0e4b3640 1 -- 172.21.15.105:0/2627008266 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fdde806c8f0 msgr2=0x7fdde806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:43.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.787+0000 7fde0e4b3640 1 --2- 172.21.15.105:0/2627008266 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fdde806c8f0 0x7fdde806edb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fddf8006fd0 tx=0x7fddf8008040 comp rx=0 tx=0).stop 2024-04-15T15:20:43.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.787+0000 7fde0e4b3640 1 -- 172.21.15.105:0/2627008266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde080d1650 msgr2=0x7fde080cc230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:43.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.787+0000 7fde0e4b3640 1 --2- 172.21.15.105:0/2627008266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde080d1650 0x7fde080cc230 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fddf0009890 tx=0x7fddf00046c0 comp rx=0 tx=0).stop 2024-04-15T15:20:43.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.788+0000 7fde0e4b3640 1 -- 172.21.15.105:0/2627008266 shutdown_connections 2024-04-15T15:20:43.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.788+0000 7fde0e4b3640 1 --2- 172.21.15.105:0/2627008266 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fdde806c8f0 0x7fdde806edb0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:43.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.788+0000 7fde0e4b3640 1 --2- 172.21.15.105:0/2627008266 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde0815e8d0 0x7fde080cc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:43.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.788+0000 7fde0e4b3640 1 --2- 172.21.15.105:0/2627008266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde080d1650 0x7fde080cc230 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:43.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.788+0000 7fde0e4b3640 1 --2- 172.21.15.105:0/2627008266 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fde080ba480 0x7fde080abda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:43.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.788+0000 7fde0e4b3640 1 -- 172.21.15.105:0/2627008266 >> 172.21.15.105:0/2627008266 conn(0x7fde0806d460 msgr2=0x7fde0815dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:43.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.788+0000 7fde0e4b3640 1 -- 172.21.15.105:0/2627008266 shutdown_connections 2024-04-15T15:20:43.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.788+0000 7fde0e4b3640 1 -- 172.21.15.105:0/2627008266 wait complete. 2024-04-15T15:20:43.800 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-15T15:20:43.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.905+0000 7f2816374640 1 Processor -- start 2024-04-15T15:20:43.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.905+0000 7f2816374640 1 -- start start 2024-04-15T15:20:43.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.906+0000 7f2816374640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2810160230 0x7f28100ce710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:43.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.906+0000 7f2816374640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28100cec50 con 0x7f28100ce330 2024-04-15T15:20:43.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.906+0000 7f2816374640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28101607a0 con 0x7f281015e320 2024-04-15T15:20:43.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.906+0000 7f2816374640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f281015de90 con 0x7f2810160230 2024-04-15T15:20:43.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.906+0000 7f280ffff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f28100ce330 0x7f281015d690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41156/0 (socket says 172.21.15.105:41156) 2024-04-15T15:20:43.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.906+0000 7f280ffff640 1 -- 172.21.15.105:0/3665280210 learned_addr learned my addr 172.21.15.105:0/3665280210 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:20:43.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.906+0000 7f2807fff640 1 --2- 172.21.15.105:0/3665280210 >> v2:172.21.15.154:3300/0 conn(0x7f2810160230 0x7f28100ce710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:43.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.906+0000 7f280f7fe640 1 -- 172.21.15.105:0/3665280210 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1929518816 0 0) 0x7f28100cec50 con 0x7f28100ce330 2024-04-15T15:20:43.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.906+0000 7f280f7fe640 1 -- 172.21.15.105:0/3665280210 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f27fc003520 con 0x7f28100ce330 2024-04-15T15:20:43.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.907+0000 7f280f7fe640 1 -- 172.21.15.105:0/3665280210 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1793745907 0 0) 0x7f27fc003520 con 0x7f28100ce330 2024-04-15T15:20:43.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.907+0000 7f280f7fe640 1 -- 172.21.15.105:0/3665280210 >> v2:172.21.15.154:3300/0 conn(0x7f2810160230 msgr2=0x7f28100ce710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:43.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.907+0000 7f280f7fe640 1 --2- 172.21.15.105:0/3665280210 >> v2:172.21.15.154:3300/0 conn(0x7f2810160230 0x7f28100ce710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:43.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.907+0000 7f280f7fe640 1 -- 172.21.15.105:0/3665280210 >> v1:172.21.15.154:6789/0 conn(0x7f281015e320 legacy=0x7f281015e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:43.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.907+0000 7f280f7fe640 1 -- 172.21.15.105:0/3665280210 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2810160a80 con 0x7f28100ce330 2024-04-15T15:20:43.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.907+0000 7f280f7fe640 1 -- 172.21.15.105:0/3665280210 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7f2800001670 con 0x7f28100ce330 2024-04-15T15:20:43.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.907+0000 7f280f7fe640 1 -- 172.21.15.105:0/3665280210 >> v1:172.21.15.105:6789/0 conn(0x7f28100ce330 legacy=0x7f281015d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:43.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.907+0000 7f280f7fe640 1 --2- 172.21.15.105:0/3665280210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27fc0039b0 0x7f27fc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:43.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.907+0000 7f280f7fe640 1 --2- 172.21.15.105:0/3665280210 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27fc004800 0x7f27fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:43.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.907+0000 7f280f7fe640 1 --2- 172.21.15.105:0/3665280210 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f27fc0051c0 0x7f27fc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:43.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.907+0000 7f280f7fe640 1 -- 172.21.15.105:0/3665280210 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f281015de90 con 0x7f27fc0039b0 2024-04-15T15:20:43.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.907+0000 7f280f7fe640 1 -- 172.21.15.105:0/3665280210 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f27fc00c060 con 0x7f27fc004800 2024-04-15T15:20:43.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.907+0000 7f280f7fe640 1 -- 172.21.15.105:0/3665280210 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f27fc00c260 con 0x7f27fc0051c0 2024-04-15T15:20:43.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.907+0000 7f280ffff640 1 --2- 172.21.15.105:0/3665280210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27fc0039b0 0x7f27fc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:43.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.907+0000 7f2807fff640 1 --2- 172.21.15.105:0/3665280210 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27fc004800 0x7f27fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:43.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.907+0000 7f2814911640 1 --2- 172.21.15.105:0/3665280210 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f27fc0051c0 0x7f27fc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:43.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.907+0000 7f280ffff640 1 -- 172.21.15.105:0/3665280210 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f27fc0051c0 msgr2=0x7f27fc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:43.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.907+0000 7f280ffff640 1 --2- 172.21.15.105:0/3665280210 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f27fc0051c0 0x7f27fc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:43.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.907+0000 7f280ffff640 1 -- 172.21.15.105:0/3665280210 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27fc004800 msgr2=0x7f27fc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:43.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.907+0000 7f280ffff640 1 --2- 172.21.15.105:0/3665280210 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27fc004800 0x7f27fc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:43.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.907+0000 7f280ffff640 1 -- 172.21.15.105:0/3665280210 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27fc003520 con 0x7f27fc0039b0 2024-04-15T15:20:43.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.907+0000 7f2814911640 1 -- 172.21.15.105:0/3665280210 reap_dead start 2024-04-15T15:20:43.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.908+0000 7f280ffff640 1 --2- 172.21.15.105:0/3665280210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27fc0039b0 0x7f27fc003dd0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f280000c210 tx=0x7f27fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f37ff7f15ee20c37 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:43.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.908+0000 7f280f7fe640 1 -- 172.21.15.105:0/3665280210 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2800007d30 con 0x7f27fc0039b0 2024-04-15T15:20:43.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.908+0000 7f280f7fe640 1 -- 172.21.15.105:0/3665280210 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2800030070 con 0x7f27fc0039b0 2024-04-15T15:20:43.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.908+0000 7f280f7fe640 1 -- 172.21.15.105:0/3665280210 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280002cda0 con 0x7f27fc0039b0 2024-04-15T15:20:43.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.908+0000 7f2816374640 1 -- 172.21.15.105:0/3665280210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27fc0039b0 msgr2=0x7f27fc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:43.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.908+0000 7f2816374640 1 --2- 172.21.15.105:0/3665280210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27fc0039b0 0x7f27fc003dd0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f280000c210 tx=0x7f27fc005830 comp rx=0 tx=0).stop 2024-04-15T15:20:43.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.909+0000 7f2816374640 1 -- 172.21.15.105:0/3665280210 shutdown_connections 2024-04-15T15:20:43.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.909+0000 7f2816374640 1 --2- 172.21.15.105:0/3665280210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27fc0039b0 0x7f27fc003dd0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:43.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.909+0000 7f2816374640 1 -- 172.21.15.105:0/3665280210 >> 172.21.15.105:0/3665280210 conn(0x7f281006b9c0 msgr2=0x7f281006bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:43.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.909+0000 7f2816374640 1 -- 172.21.15.105:0/3665280210 shutdown_connections 2024-04-15T15:20:43.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.909+0000 7f2816374640 1 -- 172.21.15.105:0/3665280210 wait complete. 2024-04-15T15:20:43.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.910+0000 7f2816374640 1 Processor -- start 2024-04-15T15:20:43.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.910+0000 7f2816374640 1 -- start start 2024-04-15T15:20:43.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.910+0000 7f2816374640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f27fc0039b0 0x7f28100aa330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:43.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.910+0000 7f2816374640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28100ab550 0x7f28100ad280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:43.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.910+0000 7f2816374640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f281015b8e0 0x7f2810139810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:43.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.910+0000 7f2816374640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2810156f90 con 0x7f281015b8e0 2024-04-15T15:20:43.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.910+0000 7f2816374640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2810161040 con 0x7f28100ab550 2024-04-15T15:20:43.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.910+0000 7f2816374640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f2810157110 con 0x7f27fc0039b0 2024-04-15T15:20:43.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.911+0000 7f2814911640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f281015b8e0 0x7f2810139810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:43.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.911+0000 7f2814911640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f281015b8e0 0x7f2810139810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58730/0 (socket says 172.21.15.105:58730) 2024-04-15T15:20:43.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.911+0000 7f2814911640 1 -- 172.21.15.105:0/3651732602 learned_addr learned my addr 172.21.15.105:0/3651732602 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:43.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.911+0000 7f280ffff640 1 --2- 172.21.15.105:0/3651732602 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f27fc0039b0 0x7f28100aa330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:43.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.911+0000 7f2807fff640 1 --2- 172.21.15.105:0/3651732602 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28100ab550 0x7f28100ad280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:43.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.911+0000 7f2814911640 1 -- 172.21.15.105:0/3651732602 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f27fc0039b0 msgr2=0x7f28100aa330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:43.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.911+0000 7f2814911640 1 --2- 172.21.15.105:0/3651732602 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f27fc0039b0 0x7f28100aa330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:43.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.911+0000 7f2814911640 1 -- 172.21.15.105:0/3651732602 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28100ab550 msgr2=0x7f28100ad280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:43.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.911+0000 7f2814911640 1 --2- 172.21.15.105:0/3651732602 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28100ab550 0x7f28100ad280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:43.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.911+0000 7f2814911640 1 -- 172.21.15.105:0/3651732602 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2810159500 con 0x7f281015b8e0 2024-04-15T15:20:43.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.911+0000 7f280ffff640 1 --2- 172.21.15.105:0/3651732602 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f27fc0039b0 0x7f28100aa330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:20:43.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.911+0000 7f2807fff640 1 --2- 172.21.15.105:0/3651732602 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28100ab550 0x7f28100ad280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:20:43.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.911+0000 7f2814911640 1 --2- 172.21.15.105:0/3651732602 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f281015b8e0 0x7f2810139810 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f2808007fd0 tx=0x7f280800eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:43.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.911+0000 7f280d7fa640 1 -- 172.21.15.105:0/3651732602 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2808003970 con 0x7f281015b8e0 2024-04-15T15:20:43.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.911+0000 7f280d7fa640 1 -- 172.21.15.105:0/3651732602 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2808018070 con 0x7f281015b8e0 2024-04-15T15:20:43.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.911+0000 7f2816374640 1 -- 172.21.15.105:0/3651732602 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28101597e0 con 0x7f281015b8e0 2024-04-15T15:20:43.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.911+0000 7f280d7fa640 1 -- 172.21.15.105:0/3651732602 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2808003d70 con 0x7f281015b8e0 2024-04-15T15:20:43.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.911+0000 7f2816374640 1 -- 172.21.15.105:0/3651732602 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2810159da0 con 0x7f281015b8e0 2024-04-15T15:20:43.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.912+0000 7f2816374640 1 -- 172.21.15.105:0/3651732602 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2810068a40 con 0x7f281015b8e0 2024-04-15T15:20:43.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.913+0000 7f280d7fa640 1 -- 172.21.15.105:0/3651732602 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f28080050d0 con 0x7f281015b8e0 2024-04-15T15:20:43.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.913+0000 7f280d7fa640 1 --2- 172.21.15.105:0/3651732602 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f27f006c920 0x7f27f006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:43.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.913+0000 7f280d7fa640 1 -- 172.21.15.105:0/3651732602 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f280808b730 con 0x7f281015b8e0 2024-04-15T15:20:43.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.913+0000 7f280ffff640 1 --2- 172.21.15.105:0/3651732602 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f27f006c920 0x7f27f006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:43.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.914+0000 7f280ffff640 1 --2- 172.21.15.105:0/3651732602 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f27f006c920 0x7f27f006ede0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2800034000 tx=0x7f2800009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:43.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:43.917+0000 7f280d7fa640 1 -- 172.21.15.105:0/3651732602 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2808059b60 con 0x7f281015b8e0 2024-04-15T15:20:44.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.091+0000 7f2816374640 1 -- 172.21.15.105:0/3651732602 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2810159f70 con 0x7f281015b8e0 2024-04-15T15:20:44.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.092+0000 7f280d7fa640 1 -- 172.21.15.105:0/3651732602 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) ==== 104+0+11 (secure 0 0 0) 0x7f2808059460 con 0x7f281015b8e0 2024-04-15T15:20:44.094 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-04-15T15:20:44.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.094+0000 7f2816374640 1 -- 172.21.15.105:0/3651732602 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f27f006c920 msgr2=0x7f27f006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:44.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.094+0000 7f2816374640 1 --2- 172.21.15.105:0/3651732602 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f27f006c920 0x7f27f006ede0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2800034000 tx=0x7f2800009030 comp rx=0 tx=0).stop 2024-04-15T15:20:44.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.094+0000 7f2816374640 1 -- 172.21.15.105:0/3651732602 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f281015b8e0 msgr2=0x7f2810139810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:44.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.094+0000 7f2816374640 1 --2- 172.21.15.105:0/3651732602 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f281015b8e0 0x7f2810139810 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f2808007fd0 tx=0x7f280800eb90 comp rx=0 tx=0).stop 2024-04-15T15:20:44.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.094+0000 7f2816374640 1 -- 172.21.15.105:0/3651732602 shutdown_connections 2024-04-15T15:20:44.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.094+0000 7f2816374640 1 --2- 172.21.15.105:0/3651732602 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f27f006c920 0x7f27f006ede0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:44.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.094+0000 7f2816374640 1 --2- 172.21.15.105:0/3651732602 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f281015b8e0 0x7f2810139810 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:44.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.094+0000 7f2816374640 1 --2- 172.21.15.105:0/3651732602 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28100ab550 0x7f28100ad280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:44.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.094+0000 7f2816374640 1 --2- 172.21.15.105:0/3651732602 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f27fc0039b0 0x7f28100aa330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:44.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.094+0000 7f2816374640 1 -- 172.21.15.105:0/3651732602 >> 172.21.15.105:0/3651732602 conn(0x7f281006b9c0 msgr2=0x7f281015f3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:44.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.094+0000 7f2816374640 1 -- 172.21.15.105:0/3651732602 shutdown_connections 2024-04-15T15:20:44.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.094+0000 7f2816374640 1 -- 172.21.15.105:0/3651732602 wait complete. 2024-04-15T15:20:44.108 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-15T15:20:44.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.212+0000 7f2430037640 1 Processor -- start 2024-04-15T15:20:44.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.212+0000 7f2430037640 1 -- start start 2024-04-15T15:20:44.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.213+0000 7f2430037640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2428072e00 0x7f242806eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:44.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.213+0000 7f2430037640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f242815b1c0 0x7f242815b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:44.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.213+0000 7f2430037640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f242815bfa0 con 0x7f2428151e30 2024-04-15T15:20:44.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.213+0000 7f2430037640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f242806e670 con 0x7f242815b1c0 2024-04-15T15:20:44.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.213+0000 7f2430037640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f242806e020 con 0x7f2428072e00 2024-04-15T15:20:44.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.213+0000 7f242e5d4640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f242815b1c0 0x7f242815b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:44.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.213+0000 7f242ddd3640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f2428151e30 0x7f24280726f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41166/0 (socket says 172.21.15.105:41166) 2024-04-15T15:20:44.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.213+0000 7f242ddd3640 1 -- 172.21.15.105:0/731232954 learned_addr learned my addr 172.21.15.105:0/731232954 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:20:44.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.213+0000 7f242e5d4640 1 -- 172.21.15.105:0/731232954 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2428072e00 msgr2=0x7f242806eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:44.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.213+0000 7f242d5d2640 1 --2- 172.21.15.105:0/731232954 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2428072e00 0x7f242806eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:44.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.213+0000 7f242e5d4640 1 --2- 172.21.15.105:0/731232954 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2428072e00 0x7f242806eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:44.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.213+0000 7f242e5d4640 1 -- 172.21.15.105:0/731232954 >> v1:172.21.15.105:6789/0 conn(0x7f2428151e30 legacy=0x7f24280726f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:44.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.213+0000 7f242cdd1640 1 -- 172.21.15.105:0/731232954 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2373291894 0 0) 0x7f242815bfa0 con 0x7f2428151e30 2024-04-15T15:20:44.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.213+0000 7f242e5d4640 1 -- 172.21.15.105:0/731232954 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f242806e300 con 0x7f242815b1c0 2024-04-15T15:20:44.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.213+0000 7f242d5d2640 1 --2- 172.21.15.105:0/731232954 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2428072e00 0x7f242806eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:44.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.214+0000 7f242e5d4640 1 --2- 172.21.15.105:0/731232954 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f242815b1c0 0x7f242815b630 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f242000ba40 tx=0x7f242002e080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46c002e534fd545d server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:44.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.214+0000 7f242cdd1640 1 -- 172.21.15.105:0/731232954 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f242003b070 con 0x7f242815b1c0 2024-04-15T15:20:44.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.214+0000 7f242cdd1640 1 -- 172.21.15.105:0/731232954 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f242002ee30 con 0x7f242815b1c0 2024-04-15T15:20:44.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.214+0000 7f242cdd1640 1 -- 172.21.15.105:0/731232954 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2420036640 con 0x7f242815b1c0 2024-04-15T15:20:44.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.215+0000 7f2430037640 1 -- 172.21.15.105:0/731232954 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f242815b1c0 msgr2=0x7f242815b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:44.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.215+0000 7f2430037640 1 --2- 172.21.15.105:0/731232954 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f242815b1c0 0x7f242815b630 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f242000ba40 tx=0x7f242002e080 comp rx=0 tx=0).stop 2024-04-15T15:20:44.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.215+0000 7f2430037640 1 -- 172.21.15.105:0/731232954 shutdown_connections 2024-04-15T15:20:44.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.215+0000 7f2430037640 1 --2- 172.21.15.105:0/731232954 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f242815b1c0 0x7f242815b630 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:44.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.215+0000 7f2430037640 1 --2- 172.21.15.105:0/731232954 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2428072e00 0x7f242806eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:44.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.215+0000 7f2430037640 1 -- 172.21.15.105:0/731232954 >> 172.21.15.105:0/731232954 conn(0x7f2428067d50 msgr2=0x7f242806b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:44.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.215+0000 7f2430037640 1 -- 172.21.15.105:0/731232954 shutdown_connections 2024-04-15T15:20:44.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.215+0000 7f2430037640 1 -- 172.21.15.105:0/731232954 wait complete. 2024-04-15T15:20:44.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.216+0000 7f2430037640 1 Processor -- start 2024-04-15T15:20:44.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.216+0000 7f2430037640 1 -- start start 2024-04-15T15:20:44.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.217+0000 7f2430037640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2428072e00 0x7f2428143c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:44.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.217+0000 7f2430037640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2428151e30 0x7f24280ceef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:44.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.217+0000 7f2430037640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f242815b1c0 0x7f24280cf430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:44.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.217+0000 7f2430037640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2428071e60 con 0x7f2428072e00 2024-04-15T15:20:44.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.217+0000 7f2430037640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f242815bd80 con 0x7f2428151e30 2024-04-15T15:20:44.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.217+0000 7f2430037640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f2428071fe0 con 0x7f242815b1c0 2024-04-15T15:20:44.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.217+0000 7f242ddd3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2428072e00 0x7f2428143c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:44.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.217+0000 7f242ddd3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2428072e00 0x7f2428143c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58758/0 (socket says 172.21.15.105:58758) 2024-04-15T15:20:44.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.217+0000 7f242ddd3640 1 -- 172.21.15.105:0/2209265251 learned_addr learned my addr 172.21.15.105:0/2209265251 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:44.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.217+0000 7f242ddd3640 1 -- 172.21.15.105:0/2209265251 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f242815b1c0 msgr2=0x7f24280cf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:44.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.217+0000 7f242ddd3640 1 --2- 172.21.15.105:0/2209265251 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f242815b1c0 0x7f24280cf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:44.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.217+0000 7f242ddd3640 1 -- 172.21.15.105:0/2209265251 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2428151e30 msgr2=0x7f24280ceef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:44.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.217+0000 7f242ddd3640 1 --2- 172.21.15.105:0/2209265251 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2428151e30 0x7f24280ceef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:44.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.217+0000 7f242ddd3640 1 -- 172.21.15.105:0/2209265251 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24281555c0 con 0x7f2428072e00 2024-04-15T15:20:44.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.217+0000 7f242ddd3640 1 --2- 172.21.15.105:0/2209265251 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2428072e00 0x7f2428143c20 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f241800c4d0 tx=0x7f241800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:44.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.217+0000 7f24267fc640 1 -- 172.21.15.105:0/2209265251 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2418018070 con 0x7f2428072e00 2024-04-15T15:20:44.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.217+0000 7f24267fc640 1 -- 172.21.15.105:0/2209265251 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2418007500 con 0x7f2428072e00 2024-04-15T15:20:44.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.217+0000 7f24267fc640 1 -- 172.21.15.105:0/2209265251 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2418004a20 con 0x7f2428072e00 2024-04-15T15:20:44.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.217+0000 7f2430037640 1 -- 172.21.15.105:0/2209265251 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2428154a40 con 0x7f2428072e00 2024-04-15T15:20:44.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.218+0000 7f2430037640 1 -- 172.21.15.105:0/2209265251 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2428155000 con 0x7f2428072e00 2024-04-15T15:20:44.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.219+0000 7f2430037640 1 -- 172.21.15.105:0/2209265251 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2428068ab0 con 0x7f2428072e00 2024-04-15T15:20:44.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.219+0000 7f24267fc640 1 -- 172.21.15.105:0/2209265251 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f241801b070 con 0x7f2428072e00 2024-04-15T15:20:44.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.219+0000 7f24267fc640 1 --2- 172.21.15.105:0/2209265251 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f241406c980 0x7f241406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:44.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.219+0000 7f24267fc640 1 -- 172.21.15.105:0/2209265251 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f241808ae10 con 0x7f2428072e00 2024-04-15T15:20:44.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.219+0000 7f242d5d2640 1 --2- 172.21.15.105:0/2209265251 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f241406c980 0x7f241406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:44.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.220+0000 7f242d5d2640 1 --2- 172.21.15.105:0/2209265251 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f241406c980 0x7f241406ee40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f242815e130 tx=0x7f241c0083d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:44.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.224+0000 7f24267fc640 1 -- 172.21.15.105:0/2209265251 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f24180592e0 con 0x7f2428072e00 2024-04-15T15:20:44.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.397+0000 7f2430037640 1 -- 172.21.15.105:0/2209265251 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2428155d90 con 0x7f2428072e00 2024-04-15T15:20:44.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.397+0000 7f24267fc640 1 -- 172.21.15.105:0/2209265251 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) ==== 100+0+11 (secure 0 0 0) 0x7f2418058be0 con 0x7f2428072e00 2024-04-15T15:20:44.399 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-04-15T15:20:44.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.399+0000 7f2430037640 1 -- 172.21.15.105:0/2209265251 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f241406c980 msgr2=0x7f241406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:44.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.399+0000 7f2430037640 1 --2- 172.21.15.105:0/2209265251 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f241406c980 0x7f241406ee40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f242815e130 tx=0x7f241c0083d0 comp rx=0 tx=0).stop 2024-04-15T15:20:44.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.399+0000 7f2430037640 1 -- 172.21.15.105:0/2209265251 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2428072e00 msgr2=0x7f2428143c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:44.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.399+0000 7f2430037640 1 --2- 172.21.15.105:0/2209265251 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2428072e00 0x7f2428143c20 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f241800c4d0 tx=0x7f241800c990 comp rx=0 tx=0).stop 2024-04-15T15:20:44.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.399+0000 7f2430037640 1 -- 172.21.15.105:0/2209265251 shutdown_connections 2024-04-15T15:20:44.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.399+0000 7f2430037640 1 --2- 172.21.15.105:0/2209265251 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f242815b1c0 0x7f24280cf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:44.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.399+0000 7f2430037640 1 --2- 172.21.15.105:0/2209265251 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f241406c980 0x7f241406ee40 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:44.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.399+0000 7f2430037640 1 --2- 172.21.15.105:0/2209265251 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2428151e30 0x7f24280ceef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:44.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.399+0000 7f2430037640 1 --2- 172.21.15.105:0/2209265251 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2428072e00 0x7f2428143c20 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:44.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.399+0000 7f2430037640 1 -- 172.21.15.105:0/2209265251 >> 172.21.15.105:0/2209265251 conn(0x7f2428067d50 msgr2=0x7f24280b4cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:44.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.399+0000 7f2430037640 1 -- 172.21.15.105:0/2209265251 shutdown_connections 2024-04-15T15:20:44.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.399+0000 7f2430037640 1 -- 172.21.15.105:0/2209265251 wait complete. 2024-04-15T15:20:44.412 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-15T15:20:44.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.514+0000 7f052dfbd640 1 Processor -- start 2024-04-15T15:20:44.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.514+0000 7f052dfbd640 1 -- start start 2024-04-15T15:20:44.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.515+0000 7f052dfbd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f052813ec60 0x7f052806ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:44.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.515+0000 7f052dfbd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f052806f4d0 0x7f052815f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:44.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.515+0000 7f052dfbd640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f052815bc90 con 0x7f052813e680 2024-04-15T15:20:44.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.515+0000 7f052dfbd640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f052806f940 con 0x7f052813ec60 2024-04-15T15:20:44.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.515+0000 7f052dfbd640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f05280d0280 con 0x7f052806f4d0 2024-04-15T15:20:44.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.515+0000 7f0526ffd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f052813ec60 0x7f052806ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:44.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.515+0000 7f05277fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f052813e680 0x7f05280cfb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41178/0 (socket says 172.21.15.105:41178) 2024-04-15T15:20:44.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.515+0000 7f05277fe640 1 -- 172.21.15.105:0/160723423 learned_addr learned my addr 172.21.15.105:0/160723423 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:20:44.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.516+0000 7f0527fff640 1 --2- 172.21.15.105:0/160723423 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f052806f4d0 0x7f052815f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:44.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.516+0000 7f0526ffd640 1 -- 172.21.15.105:0/160723423 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f052806f4d0 msgr2=0x7f052815f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:44.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.516+0000 7f0526ffd640 1 --2- 172.21.15.105:0/160723423 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f052806f4d0 0x7f052815f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:44.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.516+0000 7f0526ffd640 1 -- 172.21.15.105:0/160723423 >> v1:172.21.15.105:6789/0 conn(0x7f052813e680 legacy=0x7f05280cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:44.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.516+0000 7f05267fc640 1 -- 172.21.15.105:0/160723423 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3718553947 0 0) 0x7f052815bc90 con 0x7f052813e680 2024-04-15T15:20:44.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.516+0000 7f0526ffd640 1 -- 172.21.15.105:0/160723423 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f052815bf70 con 0x7f052813ec60 2024-04-15T15:20:44.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.516+0000 7f0526ffd640 1 --2- 172.21.15.105:0/160723423 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f052813ec60 0x7f052806ef90 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f051c0097a0 tx=0x7f051c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3ca21b267aedab5 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:44.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.517+0000 7f05267fc640 1 -- 172.21.15.105:0/160723423 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f051c035020 con 0x7f052813ec60 2024-04-15T15:20:44.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.517+0000 7f05267fc640 1 -- 172.21.15.105:0/160723423 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f051c031830 con 0x7f052813ec60 2024-04-15T15:20:44.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.517+0000 7f05267fc640 1 -- 172.21.15.105:0/160723423 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f051c031c00 con 0x7f052813ec60 2024-04-15T15:20:44.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.517+0000 7f052dfbd640 1 -- 172.21.15.105:0/160723423 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f052813ec60 msgr2=0x7f052806ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:44.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.517+0000 7f052dfbd640 1 --2- 172.21.15.105:0/160723423 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f052813ec60 0x7f052806ef90 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f051c0097a0 tx=0x7f051c02b8d0 comp rx=0 tx=0).stop 2024-04-15T15:20:44.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.517+0000 7f052dfbd640 1 -- 172.21.15.105:0/160723423 shutdown_connections 2024-04-15T15:20:44.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.517+0000 7f052dfbd640 1 --2- 172.21.15.105:0/160723423 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f052806f4d0 0x7f052815f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:44.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.517+0000 7f052dfbd640 1 --2- 172.21.15.105:0/160723423 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f052813ec60 0x7f052806ef90 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:44.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.517+0000 7f052dfbd640 1 -- 172.21.15.105:0/160723423 >> 172.21.15.105:0/160723423 conn(0x7f0528067d50 msgr2=0x7f052806b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:44.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.517+0000 7f052dfbd640 1 -- 172.21.15.105:0/160723423 shutdown_connections 2024-04-15T15:20:44.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.517+0000 7f052dfbd640 1 -- 172.21.15.105:0/160723423 wait complete. 2024-04-15T15:20:44.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.518+0000 7f052dfbd640 1 Processor -- start 2024-04-15T15:20:44.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.519+0000 7f052dfbd640 1 -- start start 2024-04-15T15:20:44.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.519+0000 7f052dfbd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f052806f4d0 0x7f05281514c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:44.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.519+0000 7f052dfbd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f052813e680 0x7f0528135840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:44.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.519+0000 7f052dfbd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f052813ec60 0x7f0528142050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:44.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.519+0000 7f052dfbd640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f052815c810 con 0x7f052813e680 2024-04-15T15:20:44.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.519+0000 7f052dfbd640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f052815c530 con 0x7f052813ec60 2024-04-15T15:20:44.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.519+0000 7f052dfbd640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f05280cf590 con 0x7f052806f4d0 2024-04-15T15:20:44.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.519+0000 7f0526ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f052813e680 0x7f0528135840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:44.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.519+0000 7f0526ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f052813e680 0x7f0528135840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58768/0 (socket says 172.21.15.105:58768) 2024-04-15T15:20:44.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.519+0000 7f0526ffd640 1 -- 172.21.15.105:0/90574425 learned_addr learned my addr 172.21.15.105:0/90574425 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:44.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.520+0000 7f0527fff640 1 --2- 172.21.15.105:0/90574425 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f052813ec60 0x7f0528142050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:44.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.520+0000 7f05277fe640 1 --2- 172.21.15.105:0/90574425 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f052806f4d0 0x7f05281514c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:44.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.520+0000 7f0526ffd640 1 -- 172.21.15.105:0/90574425 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f052806f4d0 msgr2=0x7f05281514c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:44.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.520+0000 7f0526ffd640 1 --2- 172.21.15.105:0/90574425 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f052806f4d0 0x7f05281514c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:44.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.520+0000 7f0526ffd640 1 -- 172.21.15.105:0/90574425 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f052813ec60 msgr2=0x7f0528142050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:44.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.520+0000 7f0526ffd640 1 --2- 172.21.15.105:0/90574425 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f052813ec60 0x7f0528142050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:44.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.520+0000 7f0526ffd640 1 -- 172.21.15.105:0/90574425 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05281554e0 con 0x7f052813e680 2024-04-15T15:20:44.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.520+0000 7f0526ffd640 1 --2- 172.21.15.105:0/90574425 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f052813e680 0x7f0528135840 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f051c0098c0 tx=0x7f051c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:44.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.520+0000 7f050ffff640 1 -- 172.21.15.105:0/90574425 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f051c040070 con 0x7f052813e680 2024-04-15T15:20:44.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.520+0000 7f050ffff640 1 -- 172.21.15.105:0/90574425 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f051c039a20 con 0x7f052813e680 2024-04-15T15:20:44.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.520+0000 7f050ffff640 1 -- 172.21.15.105:0/90574425 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f051c035040 con 0x7f052813e680 2024-04-15T15:20:44.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.520+0000 7f052dfbd640 1 -- 172.21.15.105:0/90574425 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f052813fea0 con 0x7f052813e680 2024-04-15T15:20:44.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.520+0000 7f052dfbd640 1 -- 172.21.15.105:0/90574425 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0528135040 con 0x7f052813e680 2024-04-15T15:20:44.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.521+0000 7f052dfbd640 1 -- 172.21.15.105:0/90574425 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0528068a40 con 0x7f052813e680 2024-04-15T15:20:44.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.521+0000 7f050ffff640 1 -- 172.21.15.105:0/90574425 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f051c038020 con 0x7f052813e680 2024-04-15T15:20:44.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.522+0000 7f050ffff640 1 --2- 172.21.15.105:0/90574425 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f051406c980 0x7f051406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:44.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.522+0000 7f05277fe640 1 --2- 172.21.15.105:0/90574425 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f051406c980 0x7f051406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:44.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.522+0000 7f050ffff640 1 -- 172.21.15.105:0/90574425 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f051c0ad550 con 0x7f052813e680 2024-04-15T15:20:44.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.522+0000 7f05277fe640 1 --2- 172.21.15.105:0/90574425 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f051406c980 0x7f051406ee40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0518000f60 tx=0x7f0518007b70 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:44.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.527+0000 7f050ffff640 1 -- 172.21.15.105:0/90574425 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f051c07ba20 con 0x7f052813e680 2024-04-15T15:20:44.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.715+0000 7f052dfbd640 1 -- 172.21.15.105:0/90574425 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f052806ba60 con 0x7f052813e680 2024-04-15T15:20:44.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.716+0000 7f050ffff640 1 -- 172.21.15.105:0/90574425 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v34) ==== 80+0+0 (secure 0 0 0) 0x7f051c07b320 con 0x7f052813e680 2024-04-15T15:20:44.718 INFO:teuthology.orchestra.run.smithi105.stderr:listed 0 entries 2024-04-15T15:20:44.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.718+0000 7f052dfbd640 1 -- 172.21.15.105:0/90574425 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f051406c980 msgr2=0x7f051406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:44.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.718+0000 7f052dfbd640 1 --2- 172.21.15.105:0/90574425 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f051406c980 0x7f051406ee40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0518000f60 tx=0x7f0518007b70 comp rx=0 tx=0).stop 2024-04-15T15:20:44.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.718+0000 7f052dfbd640 1 -- 172.21.15.105:0/90574425 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f052813e680 msgr2=0x7f0528135840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:44.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.718+0000 7f052dfbd640 1 --2- 172.21.15.105:0/90574425 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f052813e680 0x7f0528135840 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f051c0098c0 tx=0x7f051c0046c0 comp rx=0 tx=0).stop 2024-04-15T15:20:44.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.718+0000 7f052dfbd640 1 -- 172.21.15.105:0/90574425 shutdown_connections 2024-04-15T15:20:44.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.718+0000 7f052dfbd640 1 --2- 172.21.15.105:0/90574425 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f051406c980 0x7f051406ee40 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:44.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.718+0000 7f052dfbd640 1 --2- 172.21.15.105:0/90574425 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f052813ec60 0x7f0528142050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:44.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.718+0000 7f052dfbd640 1 --2- 172.21.15.105:0/90574425 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f052813e680 0x7f0528135840 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:44.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.718+0000 7f052dfbd640 1 --2- 172.21.15.105:0/90574425 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f052806f4d0 0x7f05281514c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:44.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.718+0000 7f052dfbd640 1 -- 172.21.15.105:0/90574425 >> 172.21.15.105:0/90574425 conn(0x7f0528067d50 msgr2=0x7f052806de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:44.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.718+0000 7f052dfbd640 1 -- 172.21.15.105:0/90574425 shutdown_connections 2024-04-15T15:20:44.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.718+0000 7f052dfbd640 1 -- 172.21.15.105:0/90574425 wait complete. 2024-04-15T15:20:44.730 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-04-15T15:20:44.731 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:20:44.747 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:20:44.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.887+0000 7f3c2aba8640 1 Processor -- start 2024-04-15T15:20:44.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.887+0000 7f3c2aba8640 1 -- start start 2024-04-15T15:20:44.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.888+0000 7f3c2aba8640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3c2415eb60 0x7f3c240d11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:44.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.888+0000 7f3c2aba8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3c240d1710 0x7f3c24162630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:44.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.888+0000 7f3c2aba8640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3c24162b70 0x7f3c240b6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:44.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.888+0000 7f3c2aba8640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f3c24163540 con 0x7f3c24162b70 2024-04-15T15:20:44.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.888+0000 7f3c2aba8640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f3c240d2fc0 con 0x7f3c2415eb60 2024-04-15T15:20:44.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.888+0000 7f3c2aba8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f3c240d1b30 con 0x7f3c240d1710 2024-04-15T15:20:44.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.888+0000 7f3c29145640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3c24162b70 0x7f3c240b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:44.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.888+0000 7f3c29145640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3c24162b70 0x7f3c240b6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58772/0 (socket says 172.21.15.105:58772) 2024-04-15T15:20:44.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.889+0000 7f3c29145640 1 -- 172.21.15.105:0/301107720 learned_addr learned my addr 172.21.15.105:0/301107720 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:44.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.889+0000 7f3c28944640 1 --2- 172.21.15.105:0/301107720 >> v2:172.21.15.154:3300/0 conn(0x7f3c2415eb60 0x7f3c240d11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:44.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.889+0000 7f3c23fff640 1 --2- 172.21.15.105:0/301107720 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3c240d1710 0x7f3c24162630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:44.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.889+0000 7f3c29145640 1 -- 172.21.15.105:0/301107720 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3c240d1710 msgr2=0x7f3c24162630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:44.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.889+0000 7f3c29145640 1 --2- 172.21.15.105:0/301107720 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3c240d1710 0x7f3c24162630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:44.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.889+0000 7f3c29145640 1 -- 172.21.15.105:0/301107720 >> v2:172.21.15.154:3300/0 conn(0x7f3c2415eb60 msgr2=0x7f3c240d11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:44.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.889+0000 7f3c29145640 1 --2- 172.21.15.105:0/301107720 >> v2:172.21.15.154:3300/0 conn(0x7f3c2415eb60 0x7f3c240d11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:44.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.889+0000 7f3c29145640 1 -- 172.21.15.105:0/301107720 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c2415ef40 con 0x7f3c24162b70 2024-04-15T15:20:44.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.889+0000 7f3c23fff640 1 --2- 172.21.15.105:0/301107720 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3c240d1710 0x7f3c24162630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:20:44.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.890+0000 7f3c29145640 1 --2- 172.21.15.105:0/301107720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3c24162b70 0x7f3c240b6240 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f3c240d5880 tx=0x7f3c1c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fbf3e9e2a53a7f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:44.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.890+0000 7f3c237fe640 1 -- 172.21.15.105:0/301107720 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c035020 con 0x7f3c24162b70 2024-04-15T15:20:44.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.890+0000 7f3c237fe640 1 -- 172.21.15.105:0/301107720 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c1c031870 con 0x7f3c24162b70 2024-04-15T15:20:44.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.890+0000 7f3c237fe640 1 -- 172.21.15.105:0/301107720 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c031c40 con 0x7f3c24162b70 2024-04-15T15:20:44.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.890+0000 7f3c2aba8640 1 -- 172.21.15.105:0/301107720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3c24162b70 msgr2=0x7f3c240b6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:44.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.890+0000 7f3c2aba8640 1 --2- 172.21.15.105:0/301107720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3c24162b70 0x7f3c240b6240 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f3c240d5880 tx=0x7f3c1c02b910 comp rx=0 tx=0).stop 2024-04-15T15:20:44.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.890+0000 7f3c2aba8640 1 -- 172.21.15.105:0/301107720 shutdown_connections 2024-04-15T15:20:44.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.890+0000 7f3c2aba8640 1 --2- 172.21.15.105:0/301107720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3c24162b70 0x7f3c240b6240 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:44.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.890+0000 7f3c2aba8640 1 --2- 172.21.15.105:0/301107720 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3c240d1710 0x7f3c24162630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:44.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.890+0000 7f3c2aba8640 1 --2- 172.21.15.105:0/301107720 >> v2:172.21.15.154:3300/0 conn(0x7f3c2415eb60 0x7f3c240d11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:44.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.890+0000 7f3c2aba8640 1 -- 172.21.15.105:0/301107720 >> 172.21.15.105:0/301107720 conn(0x7f3c24067720 msgr2=0x7f3c24067b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:44.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.890+0000 7f3c2aba8640 1 -- 172.21.15.105:0/301107720 shutdown_connections 2024-04-15T15:20:44.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.890+0000 7f3c2aba8640 1 -- 172.21.15.105:0/301107720 wait complete. 2024-04-15T15:20:44.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.891+0000 7f3c2aba8640 1 Processor -- start 2024-04-15T15:20:44.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.891+0000 7f3c2aba8640 1 -- start start 2024-04-15T15:20:44.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.892+0000 7f3c2aba8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3c240d1710 0x7f3c2415a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:44.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.892+0000 7f3c2aba8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c2415eb60 0x7f3c240cd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:44.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.892+0000 7f3c2aba8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3c24162b70 0x7f3c240cdf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:44.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.892+0000 7f3c2aba8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3c240d2970 con 0x7f3c24162b70 2024-04-15T15:20:44.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.892+0000 7f3c2aba8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3c24162fc0 con 0x7f3c2415eb60 2024-04-15T15:20:44.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.892+0000 7f3c2aba8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f3c240d2c50 con 0x7f3c240d1710 2024-04-15T15:20:44.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.892+0000 7f3c29145640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3c24162b70 0x7f3c240cdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:44.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.892+0000 7f3c29145640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3c24162b70 0x7f3c240cdf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58780/0 (socket says 172.21.15.105:58780) 2024-04-15T15:20:44.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.892+0000 7f3c29145640 1 -- 172.21.15.105:0/2068333127 learned_addr learned my addr 172.21.15.105:0/2068333127 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:44.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.893+0000 7f3c28944640 1 --2- 172.21.15.105:0/2068333127 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3c240d1710 0x7f3c2415a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:44.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.893+0000 7f3c23fff640 1 --2- 172.21.15.105:0/2068333127 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c2415eb60 0x7f3c240cd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:44.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.893+0000 7f3c29145640 1 -- 172.21.15.105:0/2068333127 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3c240d1710 msgr2=0x7f3c2415a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:44.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.893+0000 7f3c29145640 1 --2- 172.21.15.105:0/2068333127 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3c240d1710 0x7f3c2415a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:44.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.893+0000 7f3c29145640 1 -- 172.21.15.105:0/2068333127 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c2415eb60 msgr2=0x7f3c240cd9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:44.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.893+0000 7f3c29145640 1 --2- 172.21.15.105:0/2068333127 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c2415eb60 0x7f3c240cd9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:44.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.893+0000 7f3c29145640 1 -- 172.21.15.105:0/2068333127 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c240ce550 con 0x7f3c24162b70 2024-04-15T15:20:44.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.893+0000 7f3c29145640 1 --2- 172.21.15.105:0/2068333127 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3c24162b70 0x7f3c240cdf10 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c0090b0 tx=0x7f3c1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:44.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.893+0000 7f3c217fa640 1 -- 172.21.15.105:0/2068333127 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c040070 con 0x7f3c24162b70 2024-04-15T15:20:44.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.893+0000 7f3c217fa640 1 -- 172.21.15.105:0/2068333127 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c1c030d00 con 0x7f3c24162b70 2024-04-15T15:20:44.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.893+0000 7f3c2aba8640 1 -- 172.21.15.105:0/2068333127 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c240ce830 con 0x7f3c24162b70 2024-04-15T15:20:44.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.893+0000 7f3c217fa640 1 -- 172.21.15.105:0/2068333127 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c035040 con 0x7f3c24162b70 2024-04-15T15:20:44.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.893+0000 7f3c2aba8640 1 -- 172.21.15.105:0/2068333127 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c240ceb10 con 0x7f3c24162b70 2024-04-15T15:20:44.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.894+0000 7f3c2aba8640 1 -- 172.21.15.105:0/2068333127 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c24068a40 con 0x7f3c24162b70 2024-04-15T15:20:44.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.895+0000 7f3c217fa640 1 -- 172.21.15.105:0/2068333127 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f3c1c04b060 con 0x7f3c24162b70 2024-04-15T15:20:44.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.895+0000 7f3c217fa640 1 --2- 172.21.15.105:0/2068333127 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f3c0406c980 0x7f3c0406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:44.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.895+0000 7f3c217fa640 1 -- 172.21.15.105:0/2068333127 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f3c1c0ad7e0 con 0x7f3c24162b70 2024-04-15T15:20:44.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.895+0000 7f3c28944640 1 --2- 172.21.15.105:0/2068333127 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f3c0406c980 0x7f3c0406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:44.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.896+0000 7f3c28944640 1 --2- 172.21.15.105:0/2068333127 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f3c0406c980 0x7f3c0406ee40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3c140074a0 tx=0x7f3c1400b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:44.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:44.900+0000 7f3c217fa640 1 -- 172.21.15.105:0/2068333127 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3c1c02f320 con 0x7f3c24162b70 2024-04-15T15:20:45.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.074+0000 7f3c2aba8640 1 -- 172.21.15.105:0/2068333127 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3c240b7210 con 0x7f3c24162b70 2024-04-15T15:20:45.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.075+0000 7f3c217fa640 1 -- 172.21.15.105:0/2068333127 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16717 (secure 0 0 0) 0x7f3c1c07bc20 con 0x7f3c24162b70 2024-04-15T15:20:45.078 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:20:45.078 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":34,"fsid":"09e4d2a4-a324-4793-af77-644b25e520e1","created":"2024-04-15T15:19:01.524543+0000","modified":"2024-04-15T15:20:30.645565+0000","last_up_change":"2024-04-15T15:19:12.825815+0000","last_in_change":"2024-04-15T15:19:03.788126+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:19:13.748585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:20:26.983353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-15T15:20:27.961802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6f6292b-2d7e-4aa0-9d1c-6ddd0f97287c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6801","nonce":2873040054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6803","nonce":2873040054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6807","nonce":2873040054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6805","nonce":2873040054}]},"public_addr":"172.21.15.105:6801/2873040054","cluster_addr":"172.21.15.105:6803/2873040054","heartbeat_back_addr":"172.21.15.105:6807/2873040054","heartbeat_front_addr":"172.21.15.105:6805/2873040054","state":["exists","up"]},{"osd":1,"uuid":"8900966a-b393-41e3-8f93-e5a43bd23622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6818","nonce":2494994479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6821","nonce":2494994479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6828","nonce":2494994479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6823","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6825","nonce":2494994479}]},"public_addr":"172.21.15.105:6818/2494994479","cluster_addr":"172.21.15.105:6821/2494994479","heartbeat_back_addr":"172.21.15.105:6828/2494994479","heartbeat_front_addr":"172.21.15.105:6825/2494994479","state":["exists","up"]},{"osd":2,"uuid":"81a5dc7a-9325-4ff5-941d-a2d5fba21f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6809","nonce":3798408699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6811","nonce":3798408699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6815","nonce":3798408699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6813","nonce":3798408699}]},"public_addr":"172.21.15.105:6809/3798408699","cluster_addr":"172.21.15.105:6811/3798408699","heartbeat_back_addr":"172.21.15.105:6815/3798408699","heartbeat_front_addr":"172.21.15.105:6813/3798408699","state":["exists","up"]},{"osd":3,"uuid":"df766ed2-5c71-4805-96f4-3d4970edc960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6819","nonce":1920720186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6824","nonce":1920720186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6831","nonce":1920720186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6827","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6829","nonce":1920720186}]},"public_addr":"172.21.15.105:6819/1920720186","cluster_addr":"172.21.15.105:6824/1920720186","heartbeat_back_addr":"172.21.15.105:6831/1920720186","heartbeat_front_addr":"172.21.15.105:6829/1920720186","state":["exists","up"]},{"osd":4,"uuid":"e1039e03-28b8-49df-b951-c91930b93c9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6810","nonce":3516494456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6813","nonce":3516494456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6820","nonce":3516494456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6817","nonce":3516494456}]},"public_addr":"172.21.15.154:6810/3516494456","cluster_addr":"172.21.15.154:6813/3516494456","heartbeat_back_addr":"172.21.15.154:6820/3516494456","heartbeat_front_addr":"172.21.15.154:6817/3516494456","state":["exists","up"]},{"osd":5,"uuid":"a9f5eaf1-66ff-405a-a9e4-ec3340812a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6801","nonce":540367751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6803","nonce":540367751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6807","nonce":540367751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6805","nonce":540367751}]},"public_addr":"172.21.15.154:6801/540367751","cluster_addr":"172.21.15.154:6803/540367751","heartbeat_back_addr":"172.21.15.154:6807/540367751","heartbeat_front_addr":"172.21.15.154:6805/540367751","state":["exists","up"]},{"osd":6,"uuid":"267d02ea-675f-428f-aae8-e0375e4d4feb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6825","nonce":1550829005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6827","nonce":1550829005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6831","nonce":1550829005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6829","nonce":1550829005}]},"public_addr":"172.21.15.154:6825/1550829005","cluster_addr":"172.21.15.154:6827/1550829005","heartbeat_back_addr":"172.21.15.154:6831/1550829005","heartbeat_front_addr":"172.21.15.154:6829/1550829005","state":["exists","up"]},{"osd":7,"uuid":"759ac6cc-896a-423c-885f-9fe90d8dcb14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6809","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6811","nonce":3920089903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6816","nonce":3920089903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6823","nonce":3920089903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6819","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6821","nonce":3920089903}]},"public_addr":"172.21.15.154:6811/3920089903","cluster_addr":"172.21.15.154:6816/3920089903","heartbeat_back_addr":"172.21.15.154:6823/3920089903","heartbeat_front_addr":"172.21.15.154:6821/3920089903","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.949509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.984072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.907278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.947391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.975447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.998896+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:09.028739+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.894119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:20:45.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.077+0000 7f3c2aba8640 1 -- 172.21.15.105:0/2068333127 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f3c0406c980 msgr2=0x7f3c0406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:45.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.077+0000 7f3c2aba8640 1 --2- 172.21.15.105:0/2068333127 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f3c0406c980 0x7f3c0406ee40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3c140074a0 tx=0x7f3c1400b040 comp rx=0 tx=0).stop 2024-04-15T15:20:45.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.077+0000 7f3c2aba8640 1 -- 172.21.15.105:0/2068333127 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3c24162b70 msgr2=0x7f3c240cdf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:45.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.077+0000 7f3c2aba8640 1 --2- 172.21.15.105:0/2068333127 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3c24162b70 0x7f3c240cdf10 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c0090b0 tx=0x7f3c1c0046c0 comp rx=0 tx=0).stop 2024-04-15T15:20:45.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.078+0000 7f3c2aba8640 1 -- 172.21.15.105:0/2068333127 shutdown_connections 2024-04-15T15:20:45.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.078+0000 7f3c2aba8640 1 --2- 172.21.15.105:0/2068333127 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f3c0406c980 0x7f3c0406ee40 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:45.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.078+0000 7f3c2aba8640 1 --2- 172.21.15.105:0/2068333127 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3c24162b70 0x7f3c240cdf10 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:45.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.078+0000 7f3c2aba8640 1 --2- 172.21.15.105:0/2068333127 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c2415eb60 0x7f3c240cd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:45.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.078+0000 7f3c2aba8640 1 --2- 172.21.15.105:0/2068333127 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3c240d1710 0x7f3c2415a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:45.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.078+0000 7f3c2aba8640 1 -- 172.21.15.105:0/2068333127 >> 172.21.15.105:0/2068333127 conn(0x7f3c24067720 msgr2=0x7f3c240c8860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:45.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.078+0000 7f3c2aba8640 1 -- 172.21.15.105:0/2068333127 shutdown_connections 2024-04-15T15:20:45.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.078+0000 7f3c2aba8640 1 -- 172.21.15.105:0/2068333127 wait complete. 2024-04-15T15:20:45.091 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:19:13.748585+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-15T15:20:26.983353+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-15T15:20:27.961802+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:20:45.091 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:20:45.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.195+0000 7fd6373d4640 1 Processor -- start 2024-04-15T15:20:45.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.196+0000 7fd6373d4640 1 -- start start 2024-04-15T15:20:45.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.196+0000 7fd6373d4640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd63015e8d0 0x7fd6300d2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:45.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.196+0000 7fd6373d4640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd6300ba480 0x7fd6300d1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:45.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.196+0000 7fd6373d4640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd6300d1650 0x7fd6300b61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:45.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.196+0000 7fd6373d4640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fd6300d1ac0 con 0x7fd6300d1650 2024-04-15T15:20:45.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.196+0000 7fd6373d4640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fd6300d3400 con 0x7fd6300ba480 2024-04-15T15:20:45.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.196+0000 7fd6373d4640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fd6300d36e0 con 0x7fd63015e8d0 2024-04-15T15:20:45.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.197+0000 7fd635971640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd6300d1650 0x7fd6300b61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:45.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.197+0000 7fd635971640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd6300d1650 0x7fd6300b61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58784/0 (socket says 172.21.15.105:58784) 2024-04-15T15:20:45.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.197+0000 7fd635971640 1 -- 172.21.15.105:0/2788624594 learned_addr learned my addr 172.21.15.105:0/2788624594 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:45.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.197+0000 7fd63496f640 1 --2- 172.21.15.105:0/2788624594 >> v2:172.21.15.154:3300/0 conn(0x7fd6300ba480 0x7fd6300d1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:45.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.197+0000 7fd635170640 1 --2- 172.21.15.105:0/2788624594 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd63015e8d0 0x7fd6300d2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:45.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.197+0000 7fd635971640 1 -- 172.21.15.105:0/2788624594 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd63015e8d0 msgr2=0x7fd6300d2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:45.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.197+0000 7fd635971640 1 --2- 172.21.15.105:0/2788624594 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd63015e8d0 0x7fd6300d2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:45.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.197+0000 7fd635971640 1 -- 172.21.15.105:0/2788624594 >> v2:172.21.15.154:3300/0 conn(0x7fd6300ba480 msgr2=0x7fd6300d1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:45.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.197+0000 7fd635971640 1 --2- 172.21.15.105:0/2788624594 >> v2:172.21.15.154:3300/0 conn(0x7fd6300ba480 0x7fd6300d1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:45.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.197+0000 7fd635971640 1 -- 172.21.15.105:0/2788624594 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6300d39c0 con 0x7fd6300d1650 2024-04-15T15:20:45.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.197+0000 7fd63496f640 1 --2- 172.21.15.105:0/2788624594 >> v2:172.21.15.154:3300/0 conn(0x7fd6300ba480 0x7fd6300d1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:20:45.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.198+0000 7fd635971640 1 --2- 172.21.15.105:0/2788624594 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6300d1650 0x7fd6300b61d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fd6300b7340 tx=0x7fd62802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fe1d946a9793a6a server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:45.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.198+0000 7fd623fff640 1 -- 172.21.15.105:0/2788624594 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62802cb00 con 0x7fd6300d1650 2024-04-15T15:20:45.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.198+0000 7fd623fff640 1 -- 172.21.15.105:0/2788624594 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd62802cca0 con 0x7fd6300d1650 2024-04-15T15:20:45.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.198+0000 7fd623fff640 1 -- 172.21.15.105:0/2788624594 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd628034610 con 0x7fd6300d1650 2024-04-15T15:20:45.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.198+0000 7fd6373d4640 1 -- 172.21.15.105:0/2788624594 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6300d1650 msgr2=0x7fd6300b61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:45.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.198+0000 7fd6373d4640 1 --2- 172.21.15.105:0/2788624594 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6300d1650 0x7fd6300b61d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fd6300b7340 tx=0x7fd62802bf60 comp rx=0 tx=0).stop 2024-04-15T15:20:45.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.198+0000 7fd6373d4640 1 -- 172.21.15.105:0/2788624594 shutdown_connections 2024-04-15T15:20:45.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.198+0000 7fd6373d4640 1 --2- 172.21.15.105:0/2788624594 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6300d1650 0x7fd6300b61d0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:45.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.198+0000 7fd6373d4640 1 --2- 172.21.15.105:0/2788624594 >> v2:172.21.15.154:3300/0 conn(0x7fd6300ba480 0x7fd6300d1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:45.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.198+0000 7fd6373d4640 1 --2- 172.21.15.105:0/2788624594 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd63015e8d0 0x7fd6300d2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:45.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.198+0000 7fd6373d4640 1 -- 172.21.15.105:0/2788624594 >> 172.21.15.105:0/2788624594 conn(0x7fd63006d460 msgr2=0x7fd630067720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:45.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.199+0000 7fd6373d4640 1 -- 172.21.15.105:0/2788624594 shutdown_connections 2024-04-15T15:20:45.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.199+0000 7fd6373d4640 1 -- 172.21.15.105:0/2788624594 wait complete. 2024-04-15T15:20:45.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.200+0000 7fd6373d4640 1 Processor -- start 2024-04-15T15:20:45.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.200+0000 7fd6373d4640 1 -- start start 2024-04-15T15:20:45.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.200+0000 7fd6373d4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6300ba480 0x7fd630157e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:45.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.200+0000 7fd6373d4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6300d1650 0x7fd6300cf330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:45.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.200+0000 7fd6373d4640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd63015e8d0 0x7fd6300cd150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:45.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.200+0000 7fd6373d4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd6300d44a0 con 0x7fd6300d1650 2024-04-15T15:20:45.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.200+0000 7fd6373d4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd6300d3f80 con 0x7fd6300ba480 2024-04-15T15:20:45.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.200+0000 7fd6373d4640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fd6300d2ff0 con 0x7fd63015e8d0 2024-04-15T15:20:45.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.201+0000 7fd63496f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6300d1650 0x7fd6300cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:45.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.201+0000 7fd63496f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6300d1650 0x7fd6300cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58798/0 (socket says 172.21.15.105:58798) 2024-04-15T15:20:45.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.201+0000 7fd63496f640 1 -- 172.21.15.105:0/2832859676 learned_addr learned my addr 172.21.15.105:0/2832859676 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:45.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.201+0000 7fd635971640 1 --2- 172.21.15.105:0/2832859676 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd63015e8d0 0x7fd6300cd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:45.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.201+0000 7fd635170640 1 --2- 172.21.15.105:0/2832859676 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6300ba480 0x7fd630157e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:45.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.201+0000 7fd63496f640 1 -- 172.21.15.105:0/2832859676 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd63015e8d0 msgr2=0x7fd6300cd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:45.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.201+0000 7fd63496f640 1 --2- 172.21.15.105:0/2832859676 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd63015e8d0 0x7fd6300cd150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:45.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.201+0000 7fd63496f640 1 -- 172.21.15.105:0/2832859676 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6300ba480 msgr2=0x7fd630157e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:45.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.201+0000 7fd63496f640 1 --2- 172.21.15.105:0/2832859676 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6300ba480 0x7fd630157e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:45.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.201+0000 7fd63496f640 1 -- 172.21.15.105:0/2832859676 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd63015b710 con 0x7fd6300d1650 2024-04-15T15:20:45.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.201+0000 7fd635170640 1 --2- 172.21.15.105:0/2832859676 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6300ba480 0x7fd630157e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:20:45.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.201+0000 7fd635971640 1 --2- 172.21.15.105:0/2832859676 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd63015e8d0 0x7fd6300cd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:20:45.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.201+0000 7fd63496f640 1 --2- 172.21.15.105:0/2832859676 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6300d1650 0x7fd6300cf330 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fd62c00e840 tx=0x7fd62c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:45.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.201+0000 7fd621ffb640 1 -- 172.21.15.105:0/2832859676 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62c017070 con 0x7fd6300d1650 2024-04-15T15:20:45.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.201+0000 7fd621ffb640 1 -- 172.21.15.105:0/2832859676 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd62c00c8d0 con 0x7fd6300d1650 2024-04-15T15:20:45.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.201+0000 7fd6373d4640 1 -- 172.21.15.105:0/2832859676 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6300cc3b0 con 0x7fd6300d1650 2024-04-15T15:20:45.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.201+0000 7fd621ffb640 1 -- 172.21.15.105:0/2832859676 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62c00f040 con 0x7fd6300d1650 2024-04-15T15:20:45.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.201+0000 7fd6373d4640 1 -- 172.21.15.105:0/2832859676 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6300cc970 con 0x7fd6300d1650 2024-04-15T15:20:45.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.202+0000 7fd6373d4640 1 -- 172.21.15.105:0/2832859676 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd630068ab0 con 0x7fd6300d1650 2024-04-15T15:20:45.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.203+0000 7fd621ffb640 1 -- 172.21.15.105:0/2832859676 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fd62c01a070 con 0x7fd6300d1650 2024-04-15T15:20:45.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.203+0000 7fd621ffb640 1 --2- 172.21.15.105:0/2832859676 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fd61406c980 0x7fd61406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:45.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.203+0000 7fd621ffb640 1 -- 172.21.15.105:0/2832859676 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fd62c08b790 con 0x7fd6300d1650 2024-04-15T15:20:45.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.203+0000 7fd635170640 1 --2- 172.21.15.105:0/2832859676 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fd61406c980 0x7fd61406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:45.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.204+0000 7fd635170640 1 --2- 172.21.15.105:0/2832859676 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fd61406c980 0x7fd61406ee40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd624006fd0 tx=0x7fd624008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:45.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.208+0000 7fd621ffb640 1 -- 172.21.15.105:0/2832859676 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd62c009e20 con 0x7fd6300d1650 2024-04-15T15:20:45.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.382+0000 7fd6373d4640 1 -- 172.21.15.105:0/2832859676 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd6300d2c30 con 0x7fd6300d1650 2024-04-15T15:20:45.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.383+0000 7fd621ffb640 1 -- 172.21.15.105:0/2832859676 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) ==== 93+0+10 (secure 0 0 0) 0x7fd62c089e20 con 0x7fd6300d1650 2024-04-15T15:20:45.385 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-04-15T15:20:45.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.385+0000 7fd6373d4640 1 -- 172.21.15.105:0/2832859676 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fd61406c980 msgr2=0x7fd61406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:45.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.385+0000 7fd6373d4640 1 --2- 172.21.15.105:0/2832859676 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fd61406c980 0x7fd61406ee40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd624006fd0 tx=0x7fd624008040 comp rx=0 tx=0).stop 2024-04-15T15:20:45.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.385+0000 7fd6373d4640 1 -- 172.21.15.105:0/2832859676 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6300d1650 msgr2=0x7fd6300cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:45.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.385+0000 7fd6373d4640 1 --2- 172.21.15.105:0/2832859676 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6300d1650 0x7fd6300cf330 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fd62c00e840 tx=0x7fd62c00ed00 comp rx=0 tx=0).stop 2024-04-15T15:20:45.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.385+0000 7fd6373d4640 1 -- 172.21.15.105:0/2832859676 shutdown_connections 2024-04-15T15:20:45.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.385+0000 7fd6373d4640 1 --2- 172.21.15.105:0/2832859676 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd63015e8d0 0x7fd6300cd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:45.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.385+0000 7fd6373d4640 1 --2- 172.21.15.105:0/2832859676 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fd61406c980 0x7fd61406ee40 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:45.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.385+0000 7fd6373d4640 1 --2- 172.21.15.105:0/2832859676 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6300d1650 0x7fd6300cf330 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:45.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.385+0000 7fd6373d4640 1 --2- 172.21.15.105:0/2832859676 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6300ba480 0x7fd630157e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:45.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.385+0000 7fd6373d4640 1 -- 172.21.15.105:0/2832859676 >> 172.21.15.105:0/2832859676 conn(0x7fd63006d460 msgr2=0x7fd63015dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:45.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.385+0000 7fd6373d4640 1 -- 172.21.15.105:0/2832859676 shutdown_connections 2024-04-15T15:20:45.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.385+0000 7fd6373d4640 1 -- 172.21.15.105:0/2832859676 wait complete. 2024-04-15T15:20:45.398 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-15T15:20:45.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.502+0000 7f6be5782640 1 Processor -- start 2024-04-15T15:20:45.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.502+0000 7f6be5782640 1 -- start start 2024-04-15T15:20:45.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.503+0000 7f6be5782640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6be015ea00 0x7f6be00d0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:45.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.503+0000 7f6be5782640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6be00ba0f0 0x7f6be00cf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:45.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.503+0000 7f6be5782640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6be00cf970 0x7f6be01624d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:45.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.503+0000 7f6be5782640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f6be00cfdc0 con 0x7f6be00cf970 2024-04-15T15:20:45.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.503+0000 7f6be5782640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f6be015ede0 con 0x7f6be015ea00 2024-04-15T15:20:45.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.503+0000 7f6be5782640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f6be00d1560 con 0x7f6be00ba0f0 2024-04-15T15:20:45.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.503+0000 7f6bdf7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6be00cf970 0x7f6be01624d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:45.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.503+0000 7f6bdf7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6be00cf970 0x7f6be01624d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58812/0 (socket says 172.21.15.105:58812) 2024-04-15T15:20:45.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.503+0000 7f6bdf7fe640 1 -- 172.21.15.105:0/839399432 learned_addr learned my addr 172.21.15.105:0/839399432 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:45.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.503+0000 7f6bdeffd640 1 --2- 172.21.15.105:0/839399432 >> v2:172.21.15.154:3300/0 conn(0x7f6be015ea00 0x7f6be00d0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:45.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.503+0000 7f6bdf7fe640 1 -- 172.21.15.105:0/839399432 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6be00ba0f0 msgr2=0x7f6be00cf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:45.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.503+0000 7f6bdf7fe640 1 --2- 172.21.15.105:0/839399432 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6be00ba0f0 0x7f6be00cf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:45.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.503+0000 7f6bdf7fe640 1 -- 172.21.15.105:0/839399432 >> v2:172.21.15.154:3300/0 conn(0x7f6be015ea00 msgr2=0x7f6be00d0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:45.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.503+0000 7f6bdf7fe640 1 --2- 172.21.15.105:0/839399432 >> v2:172.21.15.154:3300/0 conn(0x7f6be015ea00 0x7f6be00d0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:45.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.503+0000 7f6bdf7fe640 1 -- 172.21.15.105:0/839399432 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6be00d1840 con 0x7f6be00cf970 2024-04-15T15:20:45.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.504+0000 7f6bdf7fe640 1 --2- 172.21.15.105:0/839399432 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6be00cf970 0x7f6be01624d0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f6be00d5640 tx=0x7f6bd802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dce76314429df09 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:45.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.504+0000 7f6bddffb640 1 -- 172.21.15.105:0/839399432 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd802ca70 con 0x7f6be00cf970 2024-04-15T15:20:45.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.504+0000 7f6bddffb640 1 -- 172.21.15.105:0/839399432 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bd802cc10 con 0x7f6be00cf970 2024-04-15T15:20:45.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.504+0000 7f6bddffb640 1 -- 172.21.15.105:0/839399432 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd80343f0 con 0x7f6be00cf970 2024-04-15T15:20:45.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.505+0000 7f6be5782640 1 -- 172.21.15.105:0/839399432 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6be00cf970 msgr2=0x7f6be01624d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:45.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.505+0000 7f6be5782640 1 --2- 172.21.15.105:0/839399432 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6be00cf970 0x7f6be01624d0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f6be00d5640 tx=0x7f6bd802bed0 comp rx=0 tx=0).stop 2024-04-15T15:20:45.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.505+0000 7f6be5782640 1 -- 172.21.15.105:0/839399432 shutdown_connections 2024-04-15T15:20:45.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.505+0000 7f6be5782640 1 --2- 172.21.15.105:0/839399432 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6be00cf970 0x7f6be01624d0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:45.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.505+0000 7f6be5782640 1 --2- 172.21.15.105:0/839399432 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6be00ba0f0 0x7f6be00cf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:45.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.505+0000 7f6be5782640 1 --2- 172.21.15.105:0/839399432 >> v2:172.21.15.154:3300/0 conn(0x7f6be015ea00 0x7f6be00d0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:45.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.505+0000 7f6be5782640 1 -- 172.21.15.105:0/839399432 >> 172.21.15.105:0/839399432 conn(0x7f6be006ccb0 msgr2=0x7f6be006d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:45.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.505+0000 7f6be5782640 1 -- 172.21.15.105:0/839399432 shutdown_connections 2024-04-15T15:20:45.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.505+0000 7f6be5782640 1 -- 172.21.15.105:0/839399432 wait complete. 2024-04-15T15:20:45.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.506+0000 7f6be5782640 1 Processor -- start 2024-04-15T15:20:45.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.506+0000 7f6be5782640 1 -- start start 2024-04-15T15:20:45.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.507+0000 7f6be5782640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6be00ba0f0 0x7f6be015f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:45.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.507+0000 7f6be5782640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6be00cf970 0x7f6be00ce1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:45.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.507+0000 7f6be5782640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6be015ea00 0x7f6be015cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:45.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.507+0000 7f6be5782640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6be00d5290 con 0x7f6be00ba0f0 2024-04-15T15:20:45.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.507+0000 7f6be5782640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6be00d1e00 con 0x7f6be00cf970 2024-04-15T15:20:45.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.507+0000 7f6be5782640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f6be0162a10 con 0x7f6be015ea00 2024-04-15T15:20:45.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.507+0000 7f6bdeffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6be00ba0f0 0x7f6be015f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:45.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.507+0000 7f6bdeffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6be00ba0f0 0x7f6be015f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58816/0 (socket says 172.21.15.105:58816) 2024-04-15T15:20:45.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.507+0000 7f6bdeffd640 1 -- 172.21.15.105:0/1735484201 learned_addr learned my addr 172.21.15.105:0/1735484201 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:45.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.507+0000 7f6bdeffd640 1 -- 172.21.15.105:0/1735484201 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6be015ea00 msgr2=0x7f6be015cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:45.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.507+0000 7f6bdeffd640 1 --2- 172.21.15.105:0/1735484201 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6be015ea00 0x7f6be015cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:45.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.507+0000 7f6bdeffd640 1 -- 172.21.15.105:0/1735484201 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6be00cf970 msgr2=0x7f6be00ce1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:45.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.507+0000 7f6bdeffd640 1 --2- 172.21.15.105:0/1735484201 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6be00cf970 0x7f6be00ce1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:45.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.507+0000 7f6bdeffd640 1 -- 172.21.15.105:0/1735484201 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6be015d000 con 0x7f6be00ba0f0 2024-04-15T15:20:45.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.507+0000 7f6bdeffd640 1 --2- 172.21.15.105:0/1735484201 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6be00ba0f0 0x7f6be015f9a0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6bd000c820 tx=0x7f6bd000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:45.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.507+0000 7f6bc77fe640 1 -- 172.21.15.105:0/1735484201 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd0015070 con 0x7f6be00ba0f0 2024-04-15T15:20:45.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.507+0000 7f6bc77fe640 1 -- 172.21.15.105:0/1735484201 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bd0004020 con 0x7f6be00ba0f0 2024-04-15T15:20:45.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.507+0000 7f6bc77fe640 1 -- 172.21.15.105:0/1735484201 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd00043f0 con 0x7f6be00ba0f0 2024-04-15T15:20:45.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.507+0000 7f6be5782640 1 -- 172.21.15.105:0/1735484201 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6be0160040 con 0x7f6be00ba0f0 2024-04-15T15:20:45.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.507+0000 7f6be5782640 1 -- 172.21.15.105:0/1735484201 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6be00cb830 con 0x7f6be00ba0f0 2024-04-15T15:20:45.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.508+0000 7f6be5782640 1 -- 172.21.15.105:0/1735484201 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6be008c570 con 0x7f6be00ba0f0 2024-04-15T15:20:45.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.509+0000 7f6bc77fe640 1 -- 172.21.15.105:0/1735484201 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f6bd00077d0 con 0x7f6be00ba0f0 2024-04-15T15:20:45.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.509+0000 7f6bc77fe640 1 --2- 172.21.15.105:0/1735484201 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f6bcc06c980 0x7f6bcc06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:45.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.509+0000 7f6bc77fe640 1 -- 172.21.15.105:0/1735484201 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f6bd008ba00 con 0x7f6be00ba0f0 2024-04-15T15:20:45.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.509+0000 7f6bde7fc640 1 --2- 172.21.15.105:0/1735484201 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f6bcc06c980 0x7f6bcc06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:45.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.510+0000 7f6bde7fc640 1 --2- 172.21.15.105:0/1735484201 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f6bcc06c980 0x7f6bcc06ee40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6bd4009710 tx=0x7f6bd4006c90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:45.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.514+0000 7f6bc77fe640 1 -- 172.21.15.105:0/1735484201 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6be008c570 con 0x7f6be00ba0f0 2024-04-15T15:20:45.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.691+0000 7f6be5782640 1 -- 172.21.15.105:0/1735484201 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6be015d210 con 0x7f6be00ba0f0 2024-04-15T15:20:45.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.691+0000 7f6bc77fe640 1 -- 172.21.15.105:0/1735484201 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) ==== 104+0+11 (secure 0 0 0) 0x7f6bd0059d30 con 0x7f6be00ba0f0 2024-04-15T15:20:45.694 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-04-15T15:20:45.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.693+0000 7f6be5782640 1 -- 172.21.15.105:0/1735484201 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f6bcc06c980 msgr2=0x7f6bcc06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:45.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.693+0000 7f6be5782640 1 --2- 172.21.15.105:0/1735484201 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f6bcc06c980 0x7f6bcc06ee40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6bd4009710 tx=0x7f6bd4006c90 comp rx=0 tx=0).stop 2024-04-15T15:20:45.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.693+0000 7f6be5782640 1 -- 172.21.15.105:0/1735484201 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6be00ba0f0 msgr2=0x7f6be015f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:45.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.693+0000 7f6be5782640 1 --2- 172.21.15.105:0/1735484201 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6be00ba0f0 0x7f6be015f9a0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6bd000c820 tx=0x7f6bd000cce0 comp rx=0 tx=0).stop 2024-04-15T15:20:45.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.694+0000 7f6be5782640 1 -- 172.21.15.105:0/1735484201 shutdown_connections 2024-04-15T15:20:45.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.694+0000 7f6be5782640 1 --2- 172.21.15.105:0/1735484201 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6be015ea00 0x7f6be015cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:45.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.694+0000 7f6be5782640 1 --2- 172.21.15.105:0/1735484201 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f6bcc06c980 0x7f6bcc06ee40 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:45.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.694+0000 7f6be5782640 1 --2- 172.21.15.105:0/1735484201 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6be00cf970 0x7f6be00ce1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:45.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.694+0000 7f6be5782640 1 --2- 172.21.15.105:0/1735484201 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6be00ba0f0 0x7f6be015f9a0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:45.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.694+0000 7f6be5782640 1 -- 172.21.15.105:0/1735484201 >> 172.21.15.105:0/1735484201 conn(0x7f6be006ccb0 msgr2=0x7f6be00cef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:45.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.694+0000 7f6be5782640 1 -- 172.21.15.105:0/1735484201 shutdown_connections 2024-04-15T15:20:45.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.694+0000 7f6be5782640 1 -- 172.21.15.105:0/1735484201 wait complete. 2024-04-15T15:20:45.706 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-15T15:20:45.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.812+0000 7f443d6c7640 1 Processor -- start 2024-04-15T15:20:45.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.812+0000 7f443d6c7640 1 -- start start 2024-04-15T15:20:45.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.813+0000 7f443d6c7640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4438163490 0x7f44380cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:45.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.813+0000 7f443d6c7640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f44380b6580 0x7f44380b69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:45.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.813+0000 7f443d6c7640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44380b6f10 con 0x7f44380b3c20 2024-04-15T15:20:45.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.813+0000 7f443d6c7640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f4438163940 con 0x7f44380b6580 2024-04-15T15:20:45.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.813+0000 7f443d6c7640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f44380cff20 con 0x7f4438163490 2024-04-15T15:20:45.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.813+0000 7f44377fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f44380b6580 0x7f44380b69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:45.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.813+0000 7f44377fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f44380b6580 0x7f44380b69d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58820/0 (socket says 172.21.15.105:58820) 2024-04-15T15:20:45.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.813+0000 7f44377fe640 1 -- 172.21.15.105:0/1289902510 learned_addr learned my addr 172.21.15.105:0/1289902510 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:45.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.813+0000 7f44377fe640 1 -- 172.21.15.105:0/1289902510 >> v2:172.21.15.154:3300/0 conn(0x7f4438163490 msgr2=0x7f44380cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:45.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.813+0000 7f44377fe640 1 --2- 172.21.15.105:0/1289902510 >> v2:172.21.15.154:3300/0 conn(0x7f4438163490 0x7f44380cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:45.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.813+0000 7f44377fe640 1 -- 172.21.15.105:0/1289902510 >> v1:172.21.15.154:6789/0 conn(0x7f44380b3c20 legacy=0x7f44380b7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:45.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.813+0000 7f44377fe640 1 -- 172.21.15.105:0/1289902510 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44380d0200 con 0x7f44380b6580 2024-04-15T15:20:45.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.814+0000 7f44377fe640 1 --2- 172.21.15.105:0/1289902510 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44380b6580 0x7f44380b69d0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f44380687f0 tx=0x7f443002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6859c00c176b5c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:45.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.814+0000 7f44367fc640 1 -- 172.21.15.105:0/1289902510 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f443002ca80 con 0x7f44380b6580 2024-04-15T15:20:45.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.814+0000 7f44367fc640 1 -- 172.21.15.105:0/1289902510 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f443002cc20 con 0x7f44380b6580 2024-04-15T15:20:45.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.814+0000 7f44367fc640 1 -- 172.21.15.105:0/1289902510 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44300343f0 con 0x7f44380b6580 2024-04-15T15:20:45.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.815+0000 7f443d6c7640 1 -- 172.21.15.105:0/1289902510 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44380b6580 msgr2=0x7f44380b69d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:45.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.815+0000 7f443d6c7640 1 --2- 172.21.15.105:0/1289902510 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44380b6580 0x7f44380b69d0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f44380687f0 tx=0x7f443002bee0 comp rx=0 tx=0).stop 2024-04-15T15:20:45.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.815+0000 7f443d6c7640 1 -- 172.21.15.105:0/1289902510 shutdown_connections 2024-04-15T15:20:45.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.815+0000 7f443d6c7640 1 --2- 172.21.15.105:0/1289902510 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44380b6580 0x7f44380b69d0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:45.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.815+0000 7f443d6c7640 1 --2- 172.21.15.105:0/1289902510 >> v2:172.21.15.154:3300/0 conn(0x7f4438163490 0x7f44380cf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:45.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.815+0000 7f443d6c7640 1 -- 172.21.15.105:0/1289902510 >> 172.21.15.105:0/1289902510 conn(0x7f44380d1670 msgr2=0x7f44380d1a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:45.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.815+0000 7f443d6c7640 1 -- 172.21.15.105:0/1289902510 shutdown_connections 2024-04-15T15:20:45.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.815+0000 7f443d6c7640 1 -- 172.21.15.105:0/1289902510 wait complete. 2024-04-15T15:20:45.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.816+0000 7f443d6c7640 1 Processor -- start 2024-04-15T15:20:45.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.816+0000 7f443d6c7640 1 -- start start 2024-04-15T15:20:45.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.817+0000 7f443d6c7640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f44380b3c20 0x7f44380c98a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:45.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.817+0000 7f443d6c7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44380b6580 0x7f443814cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:45.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.817+0000 7f443d6c7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4438163490 0x7f44380cab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:45.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.817+0000 7f443d6c7640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f44380d3610 con 0x7f4438163490 2024-04-15T15:20:45.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.817+0000 7f443d6c7640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f44380d3330 con 0x7f44380b6580 2024-04-15T15:20:45.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.817+0000 7f443d6c7640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f44380b85d0 con 0x7f44380b3c20 2024-04-15T15:20:45.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.817+0000 7f44377fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4438163490 0x7f44380cab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:45.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.817+0000 7f44377fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4438163490 0x7f44380cab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58826/0 (socket says 172.21.15.105:58826) 2024-04-15T15:20:45.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.817+0000 7f44377fe640 1 -- 172.21.15.105:0/2341310284 learned_addr learned my addr 172.21.15.105:0/2341310284 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:45.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.817+0000 7f442e7fc640 1 --2- 172.21.15.105:0/2341310284 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44380b6580 0x7f443814cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:45.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.817+0000 7f4436ffd640 1 --2- 172.21.15.105:0/2341310284 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f44380b3c20 0x7f44380c98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:45.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.817+0000 7f44377fe640 1 -- 172.21.15.105:0/2341310284 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f44380b3c20 msgr2=0x7f44380c98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:45.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.817+0000 7f44377fe640 1 --2- 172.21.15.105:0/2341310284 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f44380b3c20 0x7f44380c98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:45.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.817+0000 7f44377fe640 1 -- 172.21.15.105:0/2341310284 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44380b6580 msgr2=0x7f443814cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:45.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.817+0000 7f44377fe640 1 --2- 172.21.15.105:0/2341310284 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44380b6580 0x7f443814cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:45.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.817+0000 7f44377fe640 1 -- 172.21.15.105:0/2341310284 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44380c8800 con 0x7f4438163490 2024-04-15T15:20:45.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.817+0000 7f442e7fc640 1 --2- 172.21.15.105:0/2341310284 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44380b6580 0x7f443814cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:45.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.817+0000 7f4436ffd640 1 --2- 172.21.15.105:0/2341310284 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f44380b3c20 0x7f44380c98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:45.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.818+0000 7f44377fe640 1 --2- 172.21.15.105:0/2341310284 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4438163490 0x7f44380cab30 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f443002c4f0 tx=0x7f4430004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:45.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.818+0000 7f442ffff640 1 -- 172.21.15.105:0/2341310284 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4430043020 con 0x7f4438163490 2024-04-15T15:20:45.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.818+0000 7f443d6c7640 1 -- 172.21.15.105:0/2341310284 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44380c8ae0 con 0x7f4438163490 2024-04-15T15:20:45.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.818+0000 7f442ffff640 1 -- 172.21.15.105:0/2341310284 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f443003e450 con 0x7f4438163490 2024-04-15T15:20:45.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.818+0000 7f442ffff640 1 -- 172.21.15.105:0/2341310284 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4430009510 con 0x7f4438163490 2024-04-15T15:20:45.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.818+0000 7f443d6c7640 1 -- 172.21.15.105:0/2341310284 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44380c90a0 con 0x7f4438163490 2024-04-15T15:20:45.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.819+0000 7f443d6c7640 1 -- 172.21.15.105:0/2341310284 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4438068ab0 con 0x7f4438163490 2024-04-15T15:20:45.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.819+0000 7f442ffff640 1 -- 172.21.15.105:0/2341310284 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f4430046020 con 0x7f4438163490 2024-04-15T15:20:45.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.820+0000 7f442ffff640 1 --2- 172.21.15.105:0/2341310284 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f441806c980 0x7f441806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:45.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.820+0000 7f4436ffd640 1 --2- 172.21.15.105:0/2341310284 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f441806c980 0x7f441806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:45.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.820+0000 7f442ffff640 1 -- 172.21.15.105:0/2341310284 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f44300ae8c0 con 0x7f4438163490 2024-04-15T15:20:45.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.820+0000 7f4436ffd640 1 --2- 172.21.15.105:0/2341310284 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f441806c980 0x7f441806ee40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f442800a810 tx=0x7f4428008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:45.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:45.824+0000 7f442ffff640 1 -- 172.21.15.105:0/2341310284 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f443004d020 con 0x7f4438163490 2024-04-15T15:20:46.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.000+0000 7f443d6c7640 1 -- 172.21.15.105:0/2341310284 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f44380d2a20 con 0x7f4438163490 2024-04-15T15:20:46.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.000+0000 7f442ffff640 1 -- 172.21.15.105:0/2341310284 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) ==== 100+0+11 (secure 0 0 0) 0x7f443007cbf0 con 0x7f4438163490 2024-04-15T15:20:46.002 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-04-15T15:20:46.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.002+0000 7f443d6c7640 1 -- 172.21.15.105:0/2341310284 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f441806c980 msgr2=0x7f441806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:46.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.002+0000 7f443d6c7640 1 --2- 172.21.15.105:0/2341310284 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f441806c980 0x7f441806ee40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f442800a810 tx=0x7f4428008040 comp rx=0 tx=0).stop 2024-04-15T15:20:46.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.002+0000 7f443d6c7640 1 -- 172.21.15.105:0/2341310284 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4438163490 msgr2=0x7f44380cab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:46.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.002+0000 7f443d6c7640 1 --2- 172.21.15.105:0/2341310284 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4438163490 0x7f44380cab30 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f443002c4f0 tx=0x7f4430004a40 comp rx=0 tx=0).stop 2024-04-15T15:20:46.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.002+0000 7f443d6c7640 1 -- 172.21.15.105:0/2341310284 shutdown_connections 2024-04-15T15:20:46.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.002+0000 7f443d6c7640 1 --2- 172.21.15.105:0/2341310284 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f441806c980 0x7f441806ee40 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:46.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.002+0000 7f443d6c7640 1 --2- 172.21.15.105:0/2341310284 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4438163490 0x7f44380cab30 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:46.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.002+0000 7f443d6c7640 1 --2- 172.21.15.105:0/2341310284 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44380b6580 0x7f443814cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:46.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.002+0000 7f443d6c7640 1 --2- 172.21.15.105:0/2341310284 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f44380b3c20 0x7f44380c98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:46.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.002+0000 7f443d6c7640 1 -- 172.21.15.105:0/2341310284 >> 172.21.15.105:0/2341310284 conn(0x7f44380d1670 msgr2=0x7f44380d1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:46.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.002+0000 7f443d6c7640 1 -- 172.21.15.105:0/2341310284 shutdown_connections 2024-04-15T15:20:46.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.002+0000 7f443d6c7640 1 -- 172.21.15.105:0/2341310284 wait complete. 2024-04-15T15:20:46.015 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-15T15:20:46.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.117+0000 7fa10c488640 1 Processor -- start 2024-04-15T15:20:46.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.118+0000 7fa10c488640 1 -- start start 2024-04-15T15:20:46.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.118+0000 7fa10c488640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1040d0c90 0x7fa1040d1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:46.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.118+0000 7fa10c488640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa1040b24e0 0x7fa1040b2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:46.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.118+0000 7fa10c488640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1040d15b0 con 0x7fa1040b1a40 2024-04-15T15:20:46.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.118+0000 7fa10c488640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fa1040ce570 con 0x7fa1040b24e0 2024-04-15T15:20:46.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.118+0000 7fa10c488640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fa1040cdcd0 con 0x7fa1040d0c90 2024-04-15T15:20:46.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.119+0000 7fa10aa25640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa1040b24e0 0x7fa1040b2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:46.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.119+0000 7fa10a224640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1040d0c90 0x7fa1040d1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:46.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.119+0000 7fa10aa25640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa1040b24e0 0x7fa1040b2970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.105:35954/0 (socket says 172.21.15.105:35954) 2024-04-15T15:20:46.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.119+0000 7fa109a23640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fa1040b1a40 0x7fa1040d0280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.105:56742/0 (socket says 172.21.15.105:56742) 2024-04-15T15:20:46.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.119+0000 7fa10aa25640 1 -- 172.21.15.105:0/3296145688 learned_addr learned my addr 172.21.15.105:0/3296145688 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:46.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.119+0000 7fa10a224640 1 -- 172.21.15.105:0/3296145688 >> v2:172.21.15.154:3300/0 conn(0x7fa1040b24e0 msgr2=0x7fa1040b2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:46.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.119+0000 7fa10a224640 1 --2- 172.21.15.105:0/3296145688 >> v2:172.21.15.154:3300/0 conn(0x7fa1040b24e0 0x7fa1040b2970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:46.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.119+0000 7fa10a224640 1 -- 172.21.15.105:0/3296145688 >> v1:172.21.15.154:6789/0 conn(0x7fa1040b1a40 legacy=0x7fa1040d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:46.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.119+0000 7fa10a224640 1 -- 172.21.15.105:0/3296145688 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1040cdfb0 con 0x7fa1040d0c90 2024-04-15T15:20:46.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.119+0000 7fa109222640 1 -- 172.21.15.105:0/3296145688 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 554987704 0 0) 0x7fa1040d15b0 con 0x7fa1040b1a40 2024-04-15T15:20:46.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.119+0000 7fa10aa25640 1 --2- 172.21.15.105:0/3296145688 >> v2:172.21.15.154:3300/0 conn(0x7fa1040b24e0 0x7fa1040b2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:20:46.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.120+0000 7fa10a224640 1 --2- 172.21.15.105:0/3296145688 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1040d0c90 0x7fa1040d1070 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa1000097a0 tx=0x7fa10002b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=723e19cd4d295b55 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:46.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.120+0000 7fa109222640 1 -- 172.21.15.105:0/3296145688 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa100035020 con 0x7fa1040d0c90 2024-04-15T15:20:46.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.120+0000 7fa109222640 1 -- 172.21.15.105:0/3296145688 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa100031830 con 0x7fa1040d0c90 2024-04-15T15:20:46.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.120+0000 7fa109222640 1 -- 172.21.15.105:0/3296145688 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa100031c00 con 0x7fa1040d0c90 2024-04-15T15:20:46.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.121+0000 7fa10c488640 1 -- 172.21.15.105:0/3296145688 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1040d0c90 msgr2=0x7fa1040d1070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:46.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.121+0000 7fa10c488640 1 --2- 172.21.15.105:0/3296145688 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1040d0c90 0x7fa1040d1070 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa1000097a0 tx=0x7fa10002b8d0 comp rx=0 tx=0).stop 2024-04-15T15:20:46.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.121+0000 7fa10c488640 1 -- 172.21.15.105:0/3296145688 shutdown_connections 2024-04-15T15:20:46.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.121+0000 7fa10c488640 1 --2- 172.21.15.105:0/3296145688 >> v2:172.21.15.154:3300/0 conn(0x7fa1040b24e0 0x7fa1040b2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:46.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.121+0000 7fa10c488640 1 --2- 172.21.15.105:0/3296145688 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1040d0c90 0x7fa1040d1070 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:46.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.121+0000 7fa10c488640 1 -- 172.21.15.105:0/3296145688 >> 172.21.15.105:0/3296145688 conn(0x7fa1040d26b0 msgr2=0x7fa1040d2ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:46.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.121+0000 7fa10c488640 1 -- 172.21.15.105:0/3296145688 shutdown_connections 2024-04-15T15:20:46.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.121+0000 7fa10c488640 1 -- 172.21.15.105:0/3296145688 wait complete. 2024-04-15T15:20:46.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.122+0000 7fa10c488640 1 Processor -- start 2024-04-15T15:20:46.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.122+0000 7fa10c488640 1 -- start start 2024-04-15T15:20:46.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.123+0000 7fa10c488640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1040b1a40 0x7fa104159c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:46.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.123+0000 7fa10c488640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1040b24e0 0x7fa1040cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:46.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.123+0000 7fa10c488640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1040d0c90 0x7fa10415a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:46.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.123+0000 7fa10c488640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa10415cc80 con 0x7fa1040b1a40 2024-04-15T15:20:46.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.123+0000 7fa10c488640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa1040b1fc0 con 0x7fa1040b24e0 2024-04-15T15:20:46.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.123+0000 7fa10c488640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fa1040b2eb0 con 0x7fa1040d0c90 2024-04-15T15:20:46.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.123+0000 7fa10a224640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1040b1a40 0x7fa104159c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:46.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.123+0000 7fa10a224640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1040b1a40 0x7fa104159c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58834/0 (socket says 172.21.15.105:58834) 2024-04-15T15:20:46.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.123+0000 7fa10a224640 1 -- 172.21.15.105:0/3206889546 learned_addr learned my addr 172.21.15.105:0/3206889546 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:46.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.123+0000 7fa109a23640 1 --2- 172.21.15.105:0/3206889546 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1040b24e0 0x7fa1040cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:46.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.123+0000 7fa10aa25640 1 --2- 172.21.15.105:0/3206889546 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1040d0c90 0x7fa10415a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:46.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.123+0000 7fa10a224640 1 -- 172.21.15.105:0/3206889546 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1040d0c90 msgr2=0x7fa10415a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:46.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.123+0000 7fa10a224640 1 --2- 172.21.15.105:0/3206889546 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1040d0c90 0x7fa10415a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:46.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.123+0000 7fa10a224640 1 -- 172.21.15.105:0/3206889546 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1040b24e0 msgr2=0x7fa1040cb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:46.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.123+0000 7fa10a224640 1 --2- 172.21.15.105:0/3206889546 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1040b24e0 0x7fa1040cb3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:46.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.123+0000 7fa10a224640 1 -- 172.21.15.105:0/3206889546 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa10415b340 con 0x7fa1040b1a40 2024-04-15T15:20:46.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.123+0000 7fa109a23640 1 --2- 172.21.15.105:0/3206889546 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1040b24e0 0x7fa1040cb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:20:46.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.124+0000 7fa10a224640 1 --2- 172.21.15.105:0/3206889546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1040b1a40 0x7fa104159c30 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fa1000097a0 tx=0x7fa1000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:46.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.124+0000 7fa0faffd640 1 -- 172.21.15.105:0/3206889546 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa100004230 con 0x7fa1040b1a40 2024-04-15T15:20:46.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.124+0000 7fa0faffd640 1 -- 172.21.15.105:0/3206889546 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1000043f0 con 0x7fa1040b1a40 2024-04-15T15:20:46.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.124+0000 7fa10c488640 1 -- 172.21.15.105:0/3206889546 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1040ca8c0 con 0x7fa1040b1a40 2024-04-15T15:20:46.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.124+0000 7fa0faffd640 1 -- 172.21.15.105:0/3206889546 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa10003a870 con 0x7fa1040b1a40 2024-04-15T15:20:46.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.124+0000 7fa10c488640 1 -- 172.21.15.105:0/3206889546 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1040cae80 con 0x7fa1040b1a40 2024-04-15T15:20:46.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.125+0000 7fa10c488640 1 -- 172.21.15.105:0/3206889546 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa104069150 con 0x7fa1040b1a40 2024-04-15T15:20:46.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.125+0000 7fa0faffd640 1 -- 172.21.15.105:0/3206889546 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fa100038020 con 0x7fa1040b1a40 2024-04-15T15:20:46.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.125+0000 7fa0faffd640 1 --2- 172.21.15.105:0/3206889546 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fa0e806c980 0x7fa0e806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:46.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.126+0000 7fa0faffd640 1 -- 172.21.15.105:0/3206889546 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fa1000ad9d0 con 0x7fa1040b1a40 2024-04-15T15:20:46.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.126+0000 7fa109a23640 1 --2- 172.21.15.105:0/3206889546 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fa0e806c980 0x7fa0e806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:46.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.126+0000 7fa109a23640 1 --2- 172.21.15.105:0/3206889546 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fa0e806c980 0x7fa0e806ee40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa104075310 tx=0x7fa0f4007c50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:46.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.130+0000 7fa0faffd640 1 -- 172.21.15.105:0/3206889546 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa100040070 con 0x7fa1040b1a40 2024-04-15T15:20:46.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.315+0000 7fa10c488640 1 -- 172.21.15.105:0/3206889546 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa1040d06c0 con 0x7fa1040b1a40 2024-04-15T15:20:46.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.316+0000 7fa0faffd640 1 -- 172.21.15.105:0/3206889546 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v34) ==== 80+0+0 (secure 0 0 0) 0x7fa10007bd00 con 0x7fa1040b1a40 2024-04-15T15:20:46.318 INFO:teuthology.orchestra.run.smithi105.stderr:listed 0 entries 2024-04-15T15:20:46.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.318+0000 7fa10c488640 1 -- 172.21.15.105:0/3206889546 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fa0e806c980 msgr2=0x7fa0e806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:46.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.318+0000 7fa10c488640 1 --2- 172.21.15.105:0/3206889546 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fa0e806c980 0x7fa0e806ee40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa104075310 tx=0x7fa0f4007c50 comp rx=0 tx=0).stop 2024-04-15T15:20:46.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.318+0000 7fa10c488640 1 -- 172.21.15.105:0/3206889546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1040b1a40 msgr2=0x7fa104159c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:46.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.318+0000 7fa10c488640 1 --2- 172.21.15.105:0/3206889546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1040b1a40 0x7fa104159c30 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fa1000097a0 tx=0x7fa1000046c0 comp rx=0 tx=0).stop 2024-04-15T15:20:46.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.318+0000 7fa10c488640 1 -- 172.21.15.105:0/3206889546 shutdown_connections 2024-04-15T15:20:46.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.318+0000 7fa10c488640 1 --2- 172.21.15.105:0/3206889546 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1040d0c90 0x7fa10415a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:46.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.318+0000 7fa10c488640 1 --2- 172.21.15.105:0/3206889546 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fa0e806c980 0x7fa0e806ee40 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:46.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.318+0000 7fa10c488640 1 --2- 172.21.15.105:0/3206889546 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1040b24e0 0x7fa1040cb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:46.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.318+0000 7fa10c488640 1 --2- 172.21.15.105:0/3206889546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1040b1a40 0x7fa104159c30 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:46.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.318+0000 7fa10c488640 1 -- 172.21.15.105:0/3206889546 >> 172.21.15.105:0/3206889546 conn(0x7fa1040d26b0 msgr2=0x7fa1040d1070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:46.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.318+0000 7fa10c488640 1 -- 172.21.15.105:0/3206889546 shutdown_connections 2024-04-15T15:20:46.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.318+0000 7fa10c488640 1 -- 172.21.15.105:0/3206889546 wait complete. 2024-04-15T15:20:46.331 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi105.front.sepia.ceph.com... 2024-04-15T15:20:46.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:46.331 DEBUG:teuthology.orchestra.run.smithi105:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-15T15:20:46.358 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi105.stderr:ceph-fuse[52166]: fuse finished with error 0 and tester_r 0 2024-04-15T15:20:46.360 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:20:46.428 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi105.front.sepia.ceph.com 2024-04-15T15:20:46.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:46.428 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-15T15:20:46.444 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-15T15:20:46.444 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:20:46.464 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:20:46.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.585+0000 7ff00cc2c640 1 Processor -- start 2024-04-15T15:20:46.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.585+0000 7ff00cc2c640 1 -- start start 2024-04-15T15:20:46.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.586+0000 7ff00cc2c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff00815eb60 0x7ff0080d11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:46.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.586+0000 7ff00cc2c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff0080d1710 0x7ff008162630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:46.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.586+0000 7ff00cc2c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff008162b70 0x7ff0080b6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:46.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.586+0000 7ff00cc2c640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7ff0080d1b60 con 0x7ff0080d1710 2024-04-15T15:20:46.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.586+0000 7ff00cc2c640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7ff008163540 con 0x7ff00815eb60 2024-04-15T15:20:46.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.586+0000 7ff00cc2c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7ff0080d2fc0 con 0x7ff008162b70 2024-04-15T15:20:46.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.586+0000 7ff005d9b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff0080d1710 0x7ff008162630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:46.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.586+0000 7ff005d9b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff0080d1710 0x7ff008162630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58836/0 (socket says 172.21.15.105:58836) 2024-04-15T15:20:46.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.586+0000 7ff005d9b640 1 -- 172.21.15.105:0/3367666958 learned_addr learned my addr 172.21.15.105:0/3367666958 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:46.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.587+0000 7ff00659c640 1 --2- 172.21.15.105:0/3367666958 >> v2:172.21.15.154:3300/0 conn(0x7ff00815eb60 0x7ff0080d11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:46.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.587+0000 7ff006d9d640 1 --2- 172.21.15.105:0/3367666958 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff008162b70 0x7ff0080b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:46.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.587+0000 7ff005d9b640 1 -- 172.21.15.105:0/3367666958 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff008162b70 msgr2=0x7ff0080b6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:46.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.587+0000 7ff005d9b640 1 --2- 172.21.15.105:0/3367666958 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff008162b70 0x7ff0080b6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:46.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.587+0000 7ff005d9b640 1 -- 172.21.15.105:0/3367666958 >> v2:172.21.15.154:3300/0 conn(0x7ff00815eb60 msgr2=0x7ff0080d11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:46.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.587+0000 7ff005d9b640 1 --2- 172.21.15.105:0/3367666958 >> v2:172.21.15.154:3300/0 conn(0x7ff00815eb60 0x7ff0080d11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:46.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.587+0000 7ff005d9b640 1 -- 172.21.15.105:0/3367666958 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff00815ef40 con 0x7ff0080d1710 2024-04-15T15:20:46.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.587+0000 7ff005d9b640 1 --2- 172.21.15.105:0/3367666958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0080d1710 0x7ff008162630 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ff0080d5880 tx=0x7feffc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb6411563fcfc872 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:46.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.588+0000 7ff00559a640 1 -- 172.21.15.105:0/3367666958 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feffc035020 con 0x7ff0080d1710 2024-04-15T15:20:46.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.588+0000 7ff00559a640 1 -- 172.21.15.105:0/3367666958 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feffc031870 con 0x7ff0080d1710 2024-04-15T15:20:46.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.588+0000 7ff00559a640 1 -- 172.21.15.105:0/3367666958 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feffc031c40 con 0x7ff0080d1710 2024-04-15T15:20:46.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.588+0000 7ff00cc2c640 1 -- 172.21.15.105:0/3367666958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0080d1710 msgr2=0x7ff008162630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:46.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.588+0000 7ff00cc2c640 1 --2- 172.21.15.105:0/3367666958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0080d1710 0x7ff008162630 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ff0080d5880 tx=0x7feffc02b910 comp rx=0 tx=0).stop 2024-04-15T15:20:46.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.588+0000 7ff00cc2c640 1 -- 172.21.15.105:0/3367666958 shutdown_connections 2024-04-15T15:20:46.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.588+0000 7ff00cc2c640 1 --2- 172.21.15.105:0/3367666958 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff008162b70 0x7ff0080b6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:46.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.588+0000 7ff00cc2c640 1 --2- 172.21.15.105:0/3367666958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0080d1710 0x7ff008162630 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:46.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.588+0000 7ff00cc2c640 1 --2- 172.21.15.105:0/3367666958 >> v2:172.21.15.154:3300/0 conn(0x7ff00815eb60 0x7ff0080d11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:46.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.588+0000 7ff00cc2c640 1 -- 172.21.15.105:0/3367666958 >> 172.21.15.105:0/3367666958 conn(0x7ff008067720 msgr2=0x7ff008067b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:46.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.588+0000 7ff00cc2c640 1 -- 172.21.15.105:0/3367666958 shutdown_connections 2024-04-15T15:20:46.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.588+0000 7ff00cc2c640 1 -- 172.21.15.105:0/3367666958 wait complete. 2024-04-15T15:20:46.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.590+0000 7ff00cc2c640 1 Processor -- start 2024-04-15T15:20:46.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.590+0000 7ff00cc2c640 1 -- start start 2024-04-15T15:20:46.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.590+0000 7ff00cc2c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0080d1710 0x7ff0080d3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:46.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.590+0000 7ff00cc2c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff00815eb60 0x7ff0081607d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:46.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.590+0000 7ff00cc2c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff008162b70 0x7ff0080cc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:46.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.590+0000 7ff00cc2c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff0080d2970 con 0x7ff008162b70 2024-04-15T15:20:46.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.590+0000 7ff00cc2c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff008163260 con 0x7ff0080d1710 2024-04-15T15:20:46.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.590+0000 7ff00cc2c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7ff0080d2c50 con 0x7ff00815eb60 2024-04-15T15:20:46.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.590+0000 7ff006d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff008162b70 0x7ff0080cc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:46.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.590+0000 7ff006d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff008162b70 0x7ff0080cc8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58848/0 (socket says 172.21.15.105:58848) 2024-04-15T15:20:46.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.590+0000 7ff006d9d640 1 -- 172.21.15.105:0/1162433520 learned_addr learned my addr 172.21.15.105:0/1162433520 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:46.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.591+0000 7ff00659c640 1 --2- 172.21.15.105:0/1162433520 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0080d1710 0x7ff0080d3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:46.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.591+0000 7ff006d9d640 1 -- 172.21.15.105:0/1162433520 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff00815eb60 msgr2=0x7ff0081607d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:46.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.591+0000 7ff005d9b640 1 --2- 172.21.15.105:0/1162433520 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff00815eb60 0x7ff0081607d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:46.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.591+0000 7ff006d9d640 1 --2- 172.21.15.105:0/1162433520 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff00815eb60 0x7ff0081607d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:46.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.591+0000 7ff006d9d640 1 -- 172.21.15.105:0/1162433520 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0080d1710 msgr2=0x7ff0080d3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:46.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.591+0000 7ff006d9d640 1 --2- 172.21.15.105:0/1162433520 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0080d1710 0x7ff0080d3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:46.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.591+0000 7ff006d9d640 1 -- 172.21.15.105:0/1162433520 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff008161290 con 0x7ff008162b70 2024-04-15T15:20:46.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.591+0000 7ff00659c640 1 --2- 172.21.15.105:0/1162433520 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0080d1710 0x7ff0080d3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:46.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.591+0000 7ff005d9b640 1 --2- 172.21.15.105:0/1162433520 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff00815eb60 0x7ff0081607d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:46.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.591+0000 7ff006d9d640 1 --2- 172.21.15.105:0/1162433520 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff008162b70 0x7ff0080cc8e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ff00000d600 tx=0x7ff00000dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:46.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.591+0000 7feff6ffd640 1 -- 172.21.15.105:0/1162433520 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0000125f0 con 0x7ff008162b70 2024-04-15T15:20:46.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.591+0000 7feff6ffd640 1 -- 172.21.15.105:0/1162433520 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff000019070 con 0x7ff008162b70 2024-04-15T15:20:46.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.591+0000 7ff00cc2c640 1 -- 172.21.15.105:0/1162433520 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff008161570 con 0x7ff008162b70 2024-04-15T15:20:46.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.591+0000 7feff6ffd640 1 -- 172.21.15.105:0/1162433520 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0000129f0 con 0x7ff008162b70 2024-04-15T15:20:46.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.591+0000 7ff00cc2c640 1 -- 172.21.15.105:0/1162433520 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff008161b30 con 0x7ff008162b70 2024-04-15T15:20:46.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.592+0000 7ff00cc2c640 1 -- 172.21.15.105:0/1162433520 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff008068a40 con 0x7ff008162b70 2024-04-15T15:20:46.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.593+0000 7feff6ffd640 1 -- 172.21.15.105:0/1162433520 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7ff000012b90 con 0x7ff008162b70 2024-04-15T15:20:46.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.593+0000 7feff6ffd640 1 --2- 172.21.15.105:0/1162433520 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fefe806c980 0x7fefe806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:46.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.593+0000 7feff6ffd640 1 -- 172.21.15.105:0/1162433520 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7ff00008bf80 con 0x7ff008162b70 2024-04-15T15:20:46.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.593+0000 7ff00659c640 1 --2- 172.21.15.105:0/1162433520 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fefe806c980 0x7fefe806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:46.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.593+0000 7ff00659c640 1 --2- 172.21.15.105:0/1162433520 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fefe806c980 0x7fefe806ee40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7feff8009f70 tx=0x7feff800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:46.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.598+0000 7feff6ffd640 1 -- 172.21.15.105:0/1162433520 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff00005a3d0 con 0x7ff008162b70 2024-04-15T15:20:46.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.774+0000 7ff00cc2c640 1 -- 172.21.15.105:0/1162433520 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff0080cfe30 con 0x7ff008162b70 2024-04-15T15:20:46.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.775+0000 7feff6ffd640 1 -- 172.21.15.105:0/1162433520 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16717 (secure 0 0 0) 0x7ff000059cd0 con 0x7ff008162b70 2024-04-15T15:20:46.777 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:20:46.778 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":34,"fsid":"09e4d2a4-a324-4793-af77-644b25e520e1","created":"2024-04-15T15:19:01.524543+0000","modified":"2024-04-15T15:20:30.645565+0000","last_up_change":"2024-04-15T15:19:12.825815+0000","last_in_change":"2024-04-15T15:19:03.788126+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:19:13.748585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:20:26.983353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-15T15:20:27.961802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6f6292b-2d7e-4aa0-9d1c-6ddd0f97287c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6801","nonce":2873040054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6803","nonce":2873040054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6807","nonce":2873040054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6805","nonce":2873040054}]},"public_addr":"172.21.15.105:6801/2873040054","cluster_addr":"172.21.15.105:6803/2873040054","heartbeat_back_addr":"172.21.15.105:6807/2873040054","heartbeat_front_addr":"172.21.15.105:6805/2873040054","state":["exists","up"]},{"osd":1,"uuid":"8900966a-b393-41e3-8f93-e5a43bd23622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6818","nonce":2494994479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6821","nonce":2494994479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6828","nonce":2494994479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6823","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6825","nonce":2494994479}]},"public_addr":"172.21.15.105:6818/2494994479","cluster_addr":"172.21.15.105:6821/2494994479","heartbeat_back_addr":"172.21.15.105:6828/2494994479","heartbeat_front_addr":"172.21.15.105:6825/2494994479","state":["exists","up"]},{"osd":2,"uuid":"81a5dc7a-9325-4ff5-941d-a2d5fba21f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6809","nonce":3798408699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6811","nonce":3798408699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6815","nonce":3798408699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6813","nonce":3798408699}]},"public_addr":"172.21.15.105:6809/3798408699","cluster_addr":"172.21.15.105:6811/3798408699","heartbeat_back_addr":"172.21.15.105:6815/3798408699","heartbeat_front_addr":"172.21.15.105:6813/3798408699","state":["exists","up"]},{"osd":3,"uuid":"df766ed2-5c71-4805-96f4-3d4970edc960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6819","nonce":1920720186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6824","nonce":1920720186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6831","nonce":1920720186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6827","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6829","nonce":1920720186}]},"public_addr":"172.21.15.105:6819/1920720186","cluster_addr":"172.21.15.105:6824/1920720186","heartbeat_back_addr":"172.21.15.105:6831/1920720186","heartbeat_front_addr":"172.21.15.105:6829/1920720186","state":["exists","up"]},{"osd":4,"uuid":"e1039e03-28b8-49df-b951-c91930b93c9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6810","nonce":3516494456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6813","nonce":3516494456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6820","nonce":3516494456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6817","nonce":3516494456}]},"public_addr":"172.21.15.154:6810/3516494456","cluster_addr":"172.21.15.154:6813/3516494456","heartbeat_back_addr":"172.21.15.154:6820/3516494456","heartbeat_front_addr":"172.21.15.154:6817/3516494456","state":["exists","up"]},{"osd":5,"uuid":"a9f5eaf1-66ff-405a-a9e4-ec3340812a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6801","nonce":540367751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6803","nonce":540367751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6807","nonce":540367751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6805","nonce":540367751}]},"public_addr":"172.21.15.154:6801/540367751","cluster_addr":"172.21.15.154:6803/540367751","heartbeat_back_addr":"172.21.15.154:6807/540367751","heartbeat_front_addr":"172.21.15.154:6805/540367751","state":["exists","up"]},{"osd":6,"uuid":"267d02ea-675f-428f-aae8-e0375e4d4feb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6825","nonce":1550829005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6827","nonce":1550829005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6831","nonce":1550829005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6829","nonce":1550829005}]},"public_addr":"172.21.15.154:6825/1550829005","cluster_addr":"172.21.15.154:6827/1550829005","heartbeat_back_addr":"172.21.15.154:6831/1550829005","heartbeat_front_addr":"172.21.15.154:6829/1550829005","state":["exists","up"]},{"osd":7,"uuid":"759ac6cc-896a-423c-885f-9fe90d8dcb14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6809","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6811","nonce":3920089903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6816","nonce":3920089903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6823","nonce":3920089903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6819","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6821","nonce":3920089903}]},"public_addr":"172.21.15.154:6811/3920089903","cluster_addr":"172.21.15.154:6816/3920089903","heartbeat_back_addr":"172.21.15.154:6823/3920089903","heartbeat_front_addr":"172.21.15.154:6821/3920089903","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.949509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.984072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.907278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.947391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.975447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.998896+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:09.028739+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.894119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:20:46.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.777+0000 7ff00cc2c640 1 -- 172.21.15.105:0/1162433520 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fefe806c980 msgr2=0x7fefe806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:46.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.777+0000 7ff00cc2c640 1 --2- 172.21.15.105:0/1162433520 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fefe806c980 0x7fefe806ee40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7feff8009f70 tx=0x7feff800b040 comp rx=0 tx=0).stop 2024-04-15T15:20:46.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.777+0000 7ff00cc2c640 1 -- 172.21.15.105:0/1162433520 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff008162b70 msgr2=0x7ff0080cc8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:46.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.777+0000 7ff00cc2c640 1 --2- 172.21.15.105:0/1162433520 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff008162b70 0x7ff0080cc8e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ff00000d600 tx=0x7ff00000dac0 comp rx=0 tx=0).stop 2024-04-15T15:20:46.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.777+0000 7ff00cc2c640 1 -- 172.21.15.105:0/1162433520 shutdown_connections 2024-04-15T15:20:46.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.777+0000 7ff00cc2c640 1 --2- 172.21.15.105:0/1162433520 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff00815eb60 0x7ff0081607d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:46.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.777+0000 7ff00cc2c640 1 --2- 172.21.15.105:0/1162433520 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fefe806c980 0x7fefe806ee40 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:46.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.777+0000 7ff00cc2c640 1 --2- 172.21.15.105:0/1162433520 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff008162b70 0x7ff0080cc8e0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:46.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.777+0000 7ff00cc2c640 1 --2- 172.21.15.105:0/1162433520 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0080d1710 0x7ff0080d3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:46.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.777+0000 7ff00cc2c640 1 -- 172.21.15.105:0/1162433520 >> 172.21.15.105:0/1162433520 conn(0x7ff008067720 msgr2=0x7ff0080d4d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:46.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.778+0000 7ff00cc2c640 1 -- 172.21.15.105:0/1162433520 shutdown_connections 2024-04-15T15:20:46.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.778+0000 7ff00cc2c640 1 -- 172.21.15.105:0/1162433520 wait complete. 2024-04-15T15:20:46.791 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:19:13.748585+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-15T15:20:26.983353+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-15T15:20:27.961802+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:20:46.791 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:20:46.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.896+0000 7f821448f640 1 Processor -- start 2024-04-15T15:20:46.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.896+0000 7f821448f640 1 -- start start 2024-04-15T15:20:46.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.897+0000 7f821448f640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f820c15e570 0x7f820c06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:46.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.897+0000 7f821448f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f820c0aebd0 0x7f820c0af010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:46.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.897+0000 7f821448f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f820c15e950 0x7f820c15eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:46.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.897+0000 7f821448f640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f820c06f860 con 0x7f820c15e950 2024-04-15T15:20:46.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.897+0000 7f821448f640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f820c15f3b0 con 0x7f820c15e570 2024-04-15T15:20:46.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.897+0000 7f821448f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f820c15f690 con 0x7f820c0aebd0 2024-04-15T15:20:46.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.897+0000 7f8212a2c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f820c15e950 0x7f820c15eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:46.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.897+0000 7f8212a2c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f820c15e950 0x7f820c15eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58860/0 (socket says 172.21.15.105:58860) 2024-04-15T15:20:46.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.897+0000 7f8212a2c640 1 -- 172.21.15.105:0/3432170046 learned_addr learned my addr 172.21.15.105:0/3432170046 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:46.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.897+0000 7f8211a2a640 1 --2- 172.21.15.105:0/3432170046 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f820c0aebd0 0x7f820c0af010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:46.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.898+0000 7f821222b640 1 --2- 172.21.15.105:0/3432170046 >> v2:172.21.15.154:3300/0 conn(0x7f820c15e570 0x7f820c06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:46.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.898+0000 7f8212a2c640 1 -- 172.21.15.105:0/3432170046 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f820c0aebd0 msgr2=0x7f820c0af010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:46.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.898+0000 7f8212a2c640 1 --2- 172.21.15.105:0/3432170046 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f820c0aebd0 0x7f820c0af010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:46.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.898+0000 7f8212a2c640 1 -- 172.21.15.105:0/3432170046 >> v2:172.21.15.154:3300/0 conn(0x7f820c15e570 msgr2=0x7f820c06fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:46.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.898+0000 7f8212a2c640 1 --2- 172.21.15.105:0/3432170046 >> v2:172.21.15.154:3300/0 conn(0x7f820c15e570 0x7f820c06fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:46.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.898+0000 7f8212a2c640 1 -- 172.21.15.105:0/3432170046 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f820c15f970 con 0x7f820c15e950 2024-04-15T15:20:46.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.898+0000 7f8212a2c640 1 --2- 172.21.15.105:0/3432170046 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f820c15e950 0x7f820c15eda0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f820c075450 tx=0x7f820402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85e79a7238fd4fd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:46.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.899+0000 7f8211229640 1 -- 172.21.15.105:0/3432170046 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8204035020 con 0x7f820c15e950 2024-04-15T15:20:46.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.899+0000 7f8211229640 1 -- 172.21.15.105:0/3432170046 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8204031870 con 0x7f820c15e950 2024-04-15T15:20:46.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.899+0000 7f8211229640 1 -- 172.21.15.105:0/3432170046 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8204031c40 con 0x7f820c15e950 2024-04-15T15:20:46.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.899+0000 7f821448f640 1 -- 172.21.15.105:0/3432170046 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f820c15e950 msgr2=0x7f820c15eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:46.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.899+0000 7f821448f640 1 --2- 172.21.15.105:0/3432170046 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f820c15e950 0x7f820c15eda0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f820c075450 tx=0x7f820402b910 comp rx=0 tx=0).stop 2024-04-15T15:20:46.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.899+0000 7f821448f640 1 -- 172.21.15.105:0/3432170046 shutdown_connections 2024-04-15T15:20:46.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.899+0000 7f821448f640 1 --2- 172.21.15.105:0/3432170046 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f820c15e950 0x7f820c15eda0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:46.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.899+0000 7f821448f640 1 --2- 172.21.15.105:0/3432170046 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f820c0aebd0 0x7f820c0af010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:46.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.899+0000 7f821448f640 1 --2- 172.21.15.105:0/3432170046 >> v2:172.21.15.154:3300/0 conn(0x7f820c15e570 0x7f820c06fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:46.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.899+0000 7f821448f640 1 -- 172.21.15.105:0/3432170046 >> 172.21.15.105:0/3432170046 conn(0x7f820c0b2640 msgr2=0x7f820c0b2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:46.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.899+0000 7f821448f640 1 -- 172.21.15.105:0/3432170046 shutdown_connections 2024-04-15T15:20:46.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.899+0000 7f821448f640 1 -- 172.21.15.105:0/3432170046 wait complete. 2024-04-15T15:20:46.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.900+0000 7f821448f640 1 Processor -- start 2024-04-15T15:20:46.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.900+0000 7f821448f640 1 -- start start 2024-04-15T15:20:46.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.901+0000 7f821448f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f820c0aebd0 0x7f820c0cdd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:46.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.901+0000 7f821448f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f820c15e570 0x7f820c0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:46.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.901+0000 7f821448f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f820c15e950 0x7f820c149e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:46.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.901+0000 7f821448f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f820c0723f0 con 0x7f820c15e570 2024-04-15T15:20:46.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.901+0000 7f821448f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f820c072270 con 0x7f820c15e950 2024-04-15T15:20:46.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.901+0000 7f821448f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f820c072570 con 0x7f820c0aebd0 2024-04-15T15:20:46.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.901+0000 7f8211a2a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f820c15e570 0x7f820c0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:46.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.901+0000 7f8211a2a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f820c15e570 0x7f820c0cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58866/0 (socket says 172.21.15.105:58866) 2024-04-15T15:20:46.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.901+0000 7f8211a2a640 1 -- 172.21.15.105:0/791794715 learned_addr learned my addr 172.21.15.105:0/791794715 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:46.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.901+0000 7f821222b640 1 --2- 172.21.15.105:0/791794715 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f820c0aebd0 0x7f820c0cdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:46.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.901+0000 7f8212a2c640 1 --2- 172.21.15.105:0/791794715 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f820c15e950 0x7f820c149e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:46.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.901+0000 7f8211a2a640 1 -- 172.21.15.105:0/791794715 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f820c0aebd0 msgr2=0x7f820c0cdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:46.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.901+0000 7f8211a2a640 1 --2- 172.21.15.105:0/791794715 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f820c0aebd0 0x7f820c0cdd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:46.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.901+0000 7f8211a2a640 1 -- 172.21.15.105:0/791794715 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f820c15e950 msgr2=0x7f820c149e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:46.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.901+0000 7f8211a2a640 1 --2- 172.21.15.105:0/791794715 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f820c15e950 0x7f820c149e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:46.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.901+0000 7f8211a2a640 1 -- 172.21.15.105:0/791794715 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f820c0ce4a0 con 0x7f820c15e570 2024-04-15T15:20:46.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.902+0000 7f8211a2a640 1 --2- 172.21.15.105:0/791794715 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f820c15e570 0x7f820c0cf290 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f81fc00d800 tx=0x7f81fc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:46.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.902+0000 7f8202ffd640 1 -- 172.21.15.105:0/791794715 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81fc018020 con 0x7f820c15e570 2024-04-15T15:20:46.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.902+0000 7f8202ffd640 1 -- 172.21.15.105:0/791794715 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81fc0096a0 con 0x7f820c15e570 2024-04-15T15:20:46.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.902+0000 7f8202ffd640 1 -- 172.21.15.105:0/791794715 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81fc009a70 con 0x7f820c15e570 2024-04-15T15:20:46.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.902+0000 7f821448f640 1 -- 172.21.15.105:0/791794715 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f820c0ce780 con 0x7f820c15e570 2024-04-15T15:20:46.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.902+0000 7f821448f640 1 -- 172.21.15.105:0/791794715 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f820c155200 con 0x7f820c15e570 2024-04-15T15:20:46.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.903+0000 7f821448f640 1 -- 172.21.15.105:0/791794715 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f820c068ab0 con 0x7f820c15e570 2024-04-15T15:20:46.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.903+0000 7f8202ffd640 1 -- 172.21.15.105:0/791794715 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f81fc01b020 con 0x7f820c15e570 2024-04-15T15:20:46.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.904+0000 7f8202ffd640 1 --2- 172.21.15.105:0/791794715 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f81f006c980 0x7f81f006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:46.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.904+0000 7f821222b640 1 --2- 172.21.15.105:0/791794715 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f81f006c980 0x7f81f006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:46.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.904+0000 7f8202ffd640 1 -- 172.21.15.105:0/791794715 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f81fc08bd60 con 0x7f820c15e570 2024-04-15T15:20:46.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.904+0000 7f821222b640 1 --2- 172.21.15.105:0/791794715 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f81f006c980 0x7f81f006ee40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8208009a60 tx=0x7f8208003990 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:46.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:46.908+0000 7f8202ffd640 1 -- 172.21.15.105:0/791794715 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f81fc022030 con 0x7f820c15e570 2024-04-15T15:20:47.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.082+0000 7f821448f640 1 -- 172.21.15.105:0/791794715 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f820c0cd100 con 0x7f820c15e570 2024-04-15T15:20:47.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.082+0000 7f8202ffd640 1 -- 172.21.15.105:0/791794715 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) ==== 93+0+10 (secure 0 0 0) 0x7f81fc05a230 con 0x7f820c15e570 2024-04-15T15:20:47.084 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-04-15T15:20:47.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.084+0000 7f821448f640 1 -- 172.21.15.105:0/791794715 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f81f006c980 msgr2=0x7f81f006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:47.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.084+0000 7f821448f640 1 --2- 172.21.15.105:0/791794715 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f81f006c980 0x7f81f006ee40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8208009a60 tx=0x7f8208003990 comp rx=0 tx=0).stop 2024-04-15T15:20:47.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.084+0000 7f821448f640 1 -- 172.21.15.105:0/791794715 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f820c15e570 msgr2=0x7f820c0cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:47.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.084+0000 7f821448f640 1 --2- 172.21.15.105:0/791794715 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f820c15e570 0x7f820c0cf290 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f81fc00d800 tx=0x7f81fc00dcc0 comp rx=0 tx=0).stop 2024-04-15T15:20:47.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.084+0000 7f821448f640 1 -- 172.21.15.105:0/791794715 shutdown_connections 2024-04-15T15:20:47.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.084+0000 7f821448f640 1 --2- 172.21.15.105:0/791794715 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f81f006c980 0x7f81f006ee40 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:47.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.084+0000 7f821448f640 1 --2- 172.21.15.105:0/791794715 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f820c15e950 0x7f820c149e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:47.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.084+0000 7f821448f640 1 --2- 172.21.15.105:0/791794715 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f820c15e570 0x7f820c0cf290 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:47.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.084+0000 7f821448f640 1 --2- 172.21.15.105:0/791794715 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f820c0aebd0 0x7f820c0cdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:47.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.084+0000 7f821448f640 1 -- 172.21.15.105:0/791794715 >> 172.21.15.105:0/791794715 conn(0x7f820c0b2640 msgr2=0x7f820c06e950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:47.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.084+0000 7f821448f640 1 -- 172.21.15.105:0/791794715 shutdown_connections 2024-04-15T15:20:47.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.084+0000 7f821448f640 1 -- 172.21.15.105:0/791794715 wait complete. 2024-04-15T15:20:47.096 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-15T15:20:47.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.201+0000 7f8bbe789640 1 Processor -- start 2024-04-15T15:20:47.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.201+0000 7f8bbe789640 1 -- start start 2024-04-15T15:20:47.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.202+0000 7f8bbe789640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8bb815a2e0 0x7f8bb80cd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:47.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.202+0000 7f8bbe789640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bb815a6c0 con 0x7f8bb80d0f10 2024-04-15T15:20:47.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.202+0000 7f8bbe789640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bb815cf30 con 0x7f8bb806c3f0 2024-04-15T15:20:47.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.202+0000 7f8bbe789640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f8bb80cd5a0 con 0x7f8bb815a2e0 2024-04-15T15:20:47.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.202+0000 7f8bbcd26640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f8bb80d0f10 0x7f8bb81604c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41194/0 (socket says 172.21.15.105:41194) 2024-04-15T15:20:47.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.202+0000 7f8bbcd26640 1 -- 172.21.15.105:0/772394394 learned_addr learned my addr 172.21.15.105:0/772394394 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:20:47.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.203+0000 7f8bb7fff640 1 --2- 172.21.15.105:0/772394394 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8bb815a2e0 0x7f8bb80cd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:47.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.203+0000 7f8bb77fe640 1 -- 172.21.15.105:0/772394394 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1254566398 0 0) 0x7f8bb815a6c0 con 0x7f8bb80d0f10 2024-04-15T15:20:47.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.203+0000 7f8bb77fe640 1 -- 172.21.15.105:0/772394394 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8ba4003520 con 0x7f8bb80d0f10 2024-04-15T15:20:47.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.203+0000 7f8bb77fe640 1 -- 172.21.15.105:0/772394394 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 374336664 0 0) 0x7f8bb815cf30 con 0x7f8bb806c3f0 2024-04-15T15:20:47.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.203+0000 7f8bb77fe640 1 -- 172.21.15.105:0/772394394 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8bb815a6c0 con 0x7f8bb806c3f0 2024-04-15T15:20:47.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.203+0000 7f8bb77fe640 1 -- 172.21.15.105:0/772394394 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3981430536 0 0) 0x7f8ba4003520 con 0x7f8bb80d0f10 2024-04-15T15:20:47.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.203+0000 7f8bb77fe640 1 -- 172.21.15.105:0/772394394 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8bb815a2e0 msgr2=0x7f8bb80cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:47.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.203+0000 7f8bb77fe640 1 --2- 172.21.15.105:0/772394394 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8bb815a2e0 0x7f8bb80cd060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:47.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.203+0000 7f8bb77fe640 1 -- 172.21.15.105:0/772394394 >> v1:172.21.15.154:6789/0 conn(0x7f8bb806c3f0 legacy=0x7f8bb80d0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:47.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.203+0000 7f8bb77fe640 1 -- 172.21.15.105:0/772394394 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bb815d210 con 0x7f8bb80d0f10 2024-04-15T15:20:47.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.203+0000 7f8bb77fe640 1 -- 172.21.15.105:0/772394394 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7f8bb00019f0 con 0x7f8bb80d0f10 2024-04-15T15:20:47.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.203+0000 7f8bb77fe640 1 -- 172.21.15.105:0/772394394 >> v1:172.21.15.105:6789/0 conn(0x7f8bb80d0f10 legacy=0x7f8bb81604c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:47.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.203+0000 7f8bb77fe640 1 --2- 172.21.15.105:0/772394394 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ba4003b70 0x7f8ba4003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:47.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.203+0000 7f8bb77fe640 1 --2- 172.21.15.105:0/772394394 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ba4004800 0x7f8ba4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:47.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.203+0000 7f8bb77fe640 1 --2- 172.21.15.105:0/772394394 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8bb815a2e0 0x7f8ba40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:47.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.203+0000 7f8bb77fe640 1 -- 172.21.15.105:0/772394394 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba400ba70 con 0x7f8ba4004800 2024-04-15T15:20:47.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.203+0000 7f8bb77fe640 1 -- 172.21.15.105:0/772394394 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba400bbf0 con 0x7f8ba4003b70 2024-04-15T15:20:47.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.203+0000 7f8bb77fe640 1 -- 172.21.15.105:0/772394394 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f8ba400bdf0 con 0x7f8bb815a2e0 2024-04-15T15:20:47.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.203+0000 7f8baffff640 1 --2- 172.21.15.105:0/772394394 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ba4004800 0x7f8ba4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:47.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.203+0000 7f8bb7fff640 1 --2- 172.21.15.105:0/772394394 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ba4003b70 0x7f8ba4003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:47.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.203+0000 7f8bbcd26640 1 --2- 172.21.15.105:0/772394394 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8bb815a2e0 0x7f8ba40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:47.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.203+0000 7f8baffff640 1 -- 172.21.15.105:0/772394394 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8bb815a2e0 msgr2=0x7f8ba40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:47.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.203+0000 7f8baffff640 1 --2- 172.21.15.105:0/772394394 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8bb815a2e0 0x7f8ba40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:47.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.203+0000 7f8baffff640 1 -- 172.21.15.105:0/772394394 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ba4003b70 msgr2=0x7f8ba4003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:47.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.203+0000 7f8baffff640 1 --2- 172.21.15.105:0/772394394 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ba4003b70 0x7f8ba4003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:47.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.203+0000 7f8baffff640 1 -- 172.21.15.105:0/772394394 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ba4003520 con 0x7f8ba4004800 2024-04-15T15:20:47.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.204+0000 7f8baffff640 1 --2- 172.21.15.105:0/772394394 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ba4004800 0x7f8ba4004c80 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f8ba00018a0 tx=0x7f8ba4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc1a35eb492de6b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:47.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.204+0000 7f8bb77fe640 1 -- 172.21.15.105:0/772394394 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba0037070 con 0x7f8ba4004800 2024-04-15T15:20:47.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.204+0000 7f8bb77fe640 1 -- 172.21.15.105:0/772394394 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ba00052d0 con 0x7f8ba4004800 2024-04-15T15:20:47.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.204+0000 7f8bb77fe640 1 -- 172.21.15.105:0/772394394 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba00014f0 con 0x7f8ba4004800 2024-04-15T15:20:47.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.205+0000 7f8bbe789640 1 -- 172.21.15.105:0/772394394 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ba4004800 msgr2=0x7f8ba4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:47.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.205+0000 7f8bbe789640 1 --2- 172.21.15.105:0/772394394 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ba4004800 0x7f8ba4004c80 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f8ba00018a0 tx=0x7f8ba4005830 comp rx=0 tx=0).stop 2024-04-15T15:20:47.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.205+0000 7f8bbcd26640 1 -- 172.21.15.105:0/772394394 reap_dead start 2024-04-15T15:20:47.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.205+0000 7f8bbe789640 1 -- 172.21.15.105:0/772394394 shutdown_connections 2024-04-15T15:20:47.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.205+0000 7f8bbe789640 1 -- 172.21.15.105:0/772394394 >> 172.21.15.105:0/772394394 conn(0x7f8bb80b2fc0 msgr2=0x7f8bb80b33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:47.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.205+0000 7f8bbe789640 1 -- 172.21.15.105:0/772394394 shutdown_connections 2024-04-15T15:20:47.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.205+0000 7f8bbe789640 1 -- 172.21.15.105:0/772394394 wait complete. 2024-04-15T15:20:47.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.206+0000 7f8bbe789640 1 Processor -- start 2024-04-15T15:20:47.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.206+0000 7f8bbe789640 1 -- start start 2024-04-15T15:20:47.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.207+0000 7f8bbe789640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ba4004800 0x7f8bb80c8620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:47.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.207+0000 7f8bbe789640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8bb80c8f10 0x7f8bb8152b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:47.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.207+0000 7f8bbe789640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bb80b4cd0 0x7f8bb80b5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:47.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.207+0000 7f8bbe789640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb815d7d0 con 0x7f8bb80b4cd0 2024-04-15T15:20:47.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.207+0000 7f8bbe789640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb815db40 con 0x7f8ba4004800 2024-04-15T15:20:47.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.207+0000 7f8bbe789640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f8bb80b3ae0 con 0x7f8bb80c8f10 2024-04-15T15:20:47.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.207+0000 7f8bbcd26640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bb80b4cd0 0x7f8bb80b5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:47.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.207+0000 7f8bbcd26640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bb80b4cd0 0x7f8bb80b5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58888/0 (socket says 172.21.15.105:58888) 2024-04-15T15:20:47.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.207+0000 7f8bbcd26640 1 -- 172.21.15.105:0/2950546580 learned_addr learned my addr 172.21.15.105:0/2950546580 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:47.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.207+0000 7f8bb7fff640 1 --2- 172.21.15.105:0/2950546580 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ba4004800 0x7f8bb80c8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:47.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.207+0000 7f8bbcd26640 1 -- 172.21.15.105:0/2950546580 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8bb80c8f10 msgr2=0x7f8bb8152b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:47.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.207+0000 7f8bbcd26640 1 --2- 172.21.15.105:0/2950546580 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8bb80c8f10 0x7f8bb8152b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:47.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.207+0000 7f8bbcd26640 1 -- 172.21.15.105:0/2950546580 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ba4004800 msgr2=0x7f8bb80c8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:47.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.207+0000 7f8bbcd26640 1 --2- 172.21.15.105:0/2950546580 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ba4004800 0x7f8bb80c8620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:47.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.207+0000 7f8bbcd26640 1 -- 172.21.15.105:0/2950546580 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bb80c9480 con 0x7f8bb80b4cd0 2024-04-15T15:20:47.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.207+0000 7f8bbcd26640 1 --2- 172.21.15.105:0/2950546580 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bb80b4cd0 0x7f8bb80b5680 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8bb80b4000 tx=0x7f8bb000a600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:47.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.208+0000 7f8bb57fa640 1 -- 172.21.15.105:0/2950546580 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb000ac00 con 0x7f8bb80b4cd0 2024-04-15T15:20:47.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.208+0000 7f8bb57fa640 1 -- 172.21.15.105:0/2950546580 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bb0019070 con 0x7f8bb80b4cd0 2024-04-15T15:20:47.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.208+0000 7f8bbe789640 1 -- 172.21.15.105:0/2950546580 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bb80b41f0 con 0x7f8bb80b4cd0 2024-04-15T15:20:47.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.208+0000 7f8bb57fa640 1 -- 172.21.15.105:0/2950546580 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb00133f0 con 0x7f8bb80b4cd0 2024-04-15T15:20:47.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.208+0000 7f8bbe789640 1 -- 172.21.15.105:0/2950546580 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bb80c8b60 con 0x7f8bb80b4cd0 2024-04-15T15:20:47.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.209+0000 7f8bbe789640 1 -- 172.21.15.105:0/2950546580 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bb8068250 con 0x7f8bb80b4cd0 2024-04-15T15:20:47.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.209+0000 7f8bb57fa640 1 -- 172.21.15.105:0/2950546580 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f8bb0013590 con 0x7f8bb80b4cd0 2024-04-15T15:20:47.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.209+0000 7f8bb57fa640 1 --2- 172.21.15.105:0/2950546580 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f8b9806c980 0x7f8b9806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:47.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.210+0000 7f8bb57fa640 1 -- 172.21.15.105:0/2950546580 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f8bb008c160 con 0x7f8bb80b4cd0 2024-04-15T15:20:47.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.210+0000 7f8bb7fff640 1 --2- 172.21.15.105:0/2950546580 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f8b9806c980 0x7f8b9806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:47.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.210+0000 7f8bb7fff640 1 --2- 172.21.15.105:0/2950546580 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f8b9806c980 0x7f8b9806ee40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8ba80098a0 tx=0x7f8ba8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:47.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.214+0000 7f8bb57fa640 1 -- 172.21.15.105:0/2950546580 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8bb8068250 con 0x7f8bb80b4cd0 2024-04-15T15:20:47.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.387+0000 7f8bbe789640 1 -- 172.21.15.105:0/2950546580 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8bb8068a40 con 0x7f8bb80b4cd0 2024-04-15T15:20:47.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.388+0000 7f8bb57fa640 1 -- 172.21.15.105:0/2950546580 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) ==== 104+0+11 (secure 0 0 0) 0x7f8bb005a660 con 0x7f8bb80b4cd0 2024-04-15T15:20:47.389 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-04-15T15:20:47.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.389+0000 7f8bbe789640 1 -- 172.21.15.105:0/2950546580 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f8b9806c980 msgr2=0x7f8b9806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:47.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.389+0000 7f8bbe789640 1 --2- 172.21.15.105:0/2950546580 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f8b9806c980 0x7f8b9806ee40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8ba80098a0 tx=0x7f8ba8008040 comp rx=0 tx=0).stop 2024-04-15T15:20:47.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.389+0000 7f8bbe789640 1 -- 172.21.15.105:0/2950546580 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bb80b4cd0 msgr2=0x7f8bb80b5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:47.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.389+0000 7f8bbe789640 1 --2- 172.21.15.105:0/2950546580 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bb80b4cd0 0x7f8bb80b5680 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8bb80b4000 tx=0x7f8bb000a600 comp rx=0 tx=0).stop 2024-04-15T15:20:47.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.389+0000 7f8bbe789640 1 -- 172.21.15.105:0/2950546580 shutdown_connections 2024-04-15T15:20:47.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.389+0000 7f8bbe789640 1 --2- 172.21.15.105:0/2950546580 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8bb80c8f10 0x7f8bb8152b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:47.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.389+0000 7f8bbe789640 1 --2- 172.21.15.105:0/2950546580 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f8b9806c980 0x7f8b9806ee40 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:47.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.389+0000 7f8bbe789640 1 --2- 172.21.15.105:0/2950546580 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bb80b4cd0 0x7f8bb80b5680 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:47.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.389+0000 7f8bbe789640 1 --2- 172.21.15.105:0/2950546580 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ba4004800 0x7f8bb80c8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:47.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.389+0000 7f8bbe789640 1 -- 172.21.15.105:0/2950546580 >> 172.21.15.105:0/2950546580 conn(0x7f8bb80b2fc0 msgr2=0x7f8bb80b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:47.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.390+0000 7f8bbe789640 1 -- 172.21.15.105:0/2950546580 shutdown_connections 2024-04-15T15:20:47.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.390+0000 7f8bbe789640 1 -- 172.21.15.105:0/2950546580 wait complete. 2024-04-15T15:20:47.402 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-15T15:20:47.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.506+0000 7f5feaf04640 1 Processor -- start 2024-04-15T15:20:47.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.506+0000 7f5feaf04640 1 -- start start 2024-04-15T15:20:47.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.507+0000 7f5feaf04640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5fe4072e00 0x7f5fe406eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:47.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.507+0000 7f5feaf04640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fe415bce0 con 0x7f5fe415b1c0 2024-04-15T15:20:47.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.507+0000 7f5feaf04640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fe415bfa0 con 0x7f5fe4151e30 2024-04-15T15:20:47.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.507+0000 7f5feaf04640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f5fe406e670 con 0x7f5fe4072e00 2024-04-15T15:20:47.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.507+0000 7f5fe3fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5fe4072e00 0x7f5fe406eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:47.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.507+0000 7f5fe3fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5fe4072e00 0x7f5fe406eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58898/0 (socket says 172.21.15.105:58898) 2024-04-15T15:20:47.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.507+0000 7f5fe94a1640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f5fe415b1c0 0x7f5fe415b5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41206/0 (socket says 172.21.15.105:41206) 2024-04-15T15:20:47.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.507+0000 7f5fe3fff640 1 -- 172.21.15.105:0/70562618 learned_addr learned my addr 172.21.15.105:0/70562618 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:47.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.508+0000 7f5fe3fff640 1 -- 172.21.15.105:0/70562618 >> v1:172.21.15.154:6789/0 conn(0x7f5fe4151e30 legacy=0x7f5fe40726f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:47.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.508+0000 7f5fe3fff640 1 -- 172.21.15.105:0/70562618 >> v1:172.21.15.105:6789/0 conn(0x7f5fe415b1c0 legacy=0x7f5fe415b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:47.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.508+0000 7f5fe37fe640 1 -- 172.21.15.105:0/70562618 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3373733 0 0) 0x7f5fe415bce0 con 0x7f5fe415b1c0 2024-04-15T15:20:47.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.508+0000 7f5fe3fff640 1 -- 172.21.15.105:0/70562618 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fe406e020 con 0x7f5fe4072e00 2024-04-15T15:20:47.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.508+0000 7f5fe3fff640 1 --2- 172.21.15.105:0/70562618 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fe4072e00 0x7f5fe406eb00 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc009e50 tx=0x7f5fcc02c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff2c169852526976 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:47.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.509+0000 7f5fe37fe640 1 -- 172.21.15.105:0/70562618 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc02cbb0 con 0x7f5fe4072e00 2024-04-15T15:20:47.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.509+0000 7f5fe37fe640 1 -- 172.21.15.105:0/70562618 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fcc02cd50 con 0x7f5fe4072e00 2024-04-15T15:20:47.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.509+0000 7f5fe37fe640 1 -- 172.21.15.105:0/70562618 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc034610 con 0x7f5fe4072e00 2024-04-15T15:20:47.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.509+0000 7f5feaf04640 1 -- 172.21.15.105:0/70562618 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fe4072e00 msgr2=0x7f5fe406eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:47.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.509+0000 7f5feaf04640 1 --2- 172.21.15.105:0/70562618 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fe4072e00 0x7f5fe406eb00 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc009e50 tx=0x7f5fcc02c010 comp rx=0 tx=0).stop 2024-04-15T15:20:47.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.509+0000 7f5feaf04640 1 -- 172.21.15.105:0/70562618 shutdown_connections 2024-04-15T15:20:47.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.509+0000 7f5feaf04640 1 --2- 172.21.15.105:0/70562618 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fe4072e00 0x7f5fe406eb00 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:47.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.509+0000 7f5feaf04640 1 -- 172.21.15.105:0/70562618 >> 172.21.15.105:0/70562618 conn(0x7f5fe4067d50 msgr2=0x7f5fe406b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:47.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.509+0000 7f5feaf04640 1 -- 172.21.15.105:0/70562618 shutdown_connections 2024-04-15T15:20:47.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.509+0000 7f5feaf04640 1 -- 172.21.15.105:0/70562618 wait complete. 2024-04-15T15:20:47.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.510+0000 7f5feaf04640 1 Processor -- start 2024-04-15T15:20:47.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.510+0000 7f5feaf04640 1 -- start start 2024-04-15T15:20:47.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.511+0000 7f5feaf04640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5fe4072e00 0x7f5fe40b0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:47.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.511+0000 7f5feaf04640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fe4151e30 0x7f5fe4152c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:47.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.511+0000 7f5feaf04640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5fe415b1c0 0x7f5fe4158160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:47.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.511+0000 7f5feaf04640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe40b3c30 con 0x7f5fe4151e30 2024-04-15T15:20:47.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.511+0000 7f5feaf04640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe40b3ab0 con 0x7f5fe415b1c0 2024-04-15T15:20:47.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.511+0000 7f5feaf04640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f5fe40b3db0 con 0x7f5fe4072e00 2024-04-15T15:20:47.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.511+0000 7f5fe3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fe4151e30 0x7f5fe4152c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:47.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.511+0000 7f5fe3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fe4151e30 0x7f5fe4152c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58900/0 (socket says 172.21.15.105:58900) 2024-04-15T15:20:47.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.511+0000 7f5fe3fff640 1 -- 172.21.15.105:0/4050037910 learned_addr learned my addr 172.21.15.105:0/4050037910 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:47.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.511+0000 7f5fe94a1640 1 --2- 172.21.15.105:0/4050037910 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5fe415b1c0 0x7f5fe4158160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:47.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.512+0000 7f5fe8ca0640 1 --2- 172.21.15.105:0/4050037910 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5fe4072e00 0x7f5fe40b0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:47.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.512+0000 7f5fe3fff640 1 -- 172.21.15.105:0/4050037910 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5fe4072e00 msgr2=0x7f5fe40b0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:47.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.512+0000 7f5fe3fff640 1 --2- 172.21.15.105:0/4050037910 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5fe4072e00 0x7f5fe40b0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:47.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.512+0000 7f5fe3fff640 1 -- 172.21.15.105:0/4050037910 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5fe415b1c0 msgr2=0x7f5fe4158160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:47.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.512+0000 7f5fe3fff640 1 --2- 172.21.15.105:0/4050037910 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5fe415b1c0 0x7f5fe4158160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:47.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.512+0000 7f5fe3fff640 1 -- 172.21.15.105:0/4050037910 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fe41522b0 con 0x7f5fe4151e30 2024-04-15T15:20:47.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.512+0000 7f5fe3fff640 1 --2- 172.21.15.105:0/4050037910 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fe4151e30 0x7f5fe4152c20 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc009e50 tx=0x7f5fcc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:47.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.512+0000 7f5fe17fa640 1 -- 172.21.15.105:0/4050037910 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc043020 con 0x7f5fe4151e30 2024-04-15T15:20:47.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.512+0000 7f5fe17fa640 1 -- 172.21.15.105:0/4050037910 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fcc03e450 con 0x7f5fe4151e30 2024-04-15T15:20:47.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.512+0000 7f5feaf04640 1 -- 172.21.15.105:0/4050037910 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fe4142340 con 0x7f5fe4151e30 2024-04-15T15:20:47.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.512+0000 7f5fe17fa640 1 -- 172.21.15.105:0/4050037910 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc009510 con 0x7f5fe4151e30 2024-04-15T15:20:47.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.512+0000 7f5feaf04640 1 -- 172.21.15.105:0/4050037910 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fe4152550 con 0x7f5fe4151e30 2024-04-15T15:20:47.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.513+0000 7f5feaf04640 1 -- 172.21.15.105:0/4050037910 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fe4068ab0 con 0x7f5fe4151e30 2024-04-15T15:20:47.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.514+0000 7f5fe17fa640 1 -- 172.21.15.105:0/4050037910 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f5fcc046020 con 0x7f5fe4151e30 2024-04-15T15:20:47.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.514+0000 7f5fe17fa640 1 --2- 172.21.15.105:0/4050037910 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f5fc406c980 0x7f5fc406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:47.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.514+0000 7f5fe17fa640 1 -- 172.21.15.105:0/4050037910 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f5fcc0ae4b0 con 0x7f5fe4151e30 2024-04-15T15:20:47.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.514+0000 7f5fe8ca0640 1 --2- 172.21.15.105:0/4050037910 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f5fc406c980 0x7f5fc406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:47.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.515+0000 7f5fe8ca0640 1 --2- 172.21.15.105:0/4050037910 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f5fc406c980 0x7f5fc406ee40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4000f60 tx=0x7f5fd4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:47.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.519+0000 7f5fe17fa640 1 -- 172.21.15.105:0/4050037910 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5fcc04d030 con 0x7f5fe4151e30 2024-04-15T15:20:47.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.695+0000 7f5feaf04640 1 -- 172.21.15.105:0/4050037910 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5fe41586a0 con 0x7f5fe4151e30 2024-04-15T15:20:47.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.695+0000 7f5fe17fa640 1 -- 172.21.15.105:0/4050037910 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) ==== 100+0+11 (secure 0 0 0) 0x7f5fcc07c980 con 0x7f5fe4151e30 2024-04-15T15:20:47.697 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-04-15T15:20:47.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.697+0000 7f5feaf04640 1 -- 172.21.15.105:0/4050037910 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f5fc406c980 msgr2=0x7f5fc406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:47.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.697+0000 7f5feaf04640 1 --2- 172.21.15.105:0/4050037910 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f5fc406c980 0x7f5fc406ee40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4000f60 tx=0x7f5fd4009040 comp rx=0 tx=0).stop 2024-04-15T15:20:47.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.697+0000 7f5feaf04640 1 -- 172.21.15.105:0/4050037910 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fe4151e30 msgr2=0x7f5fe4152c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:47.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.697+0000 7f5feaf04640 1 --2- 172.21.15.105:0/4050037910 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fe4151e30 0x7f5fe4152c20 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc009e50 tx=0x7f5fcc004a40 comp rx=0 tx=0).stop 2024-04-15T15:20:47.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.697+0000 7f5feaf04640 1 -- 172.21.15.105:0/4050037910 shutdown_connections 2024-04-15T15:20:47.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.697+0000 7f5feaf04640 1 --2- 172.21.15.105:0/4050037910 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f5fc406c980 0x7f5fc406ee40 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:47.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.697+0000 7f5feaf04640 1 --2- 172.21.15.105:0/4050037910 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5fe415b1c0 0x7f5fe4158160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:47.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.697+0000 7f5feaf04640 1 --2- 172.21.15.105:0/4050037910 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fe4151e30 0x7f5fe4152c20 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:47.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.697+0000 7f5feaf04640 1 --2- 172.21.15.105:0/4050037910 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5fe4072e00 0x7f5fe40b0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:47.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.697+0000 7f5feaf04640 1 -- 172.21.15.105:0/4050037910 >> 172.21.15.105:0/4050037910 conn(0x7f5fe4067d50 msgr2=0x7f5fe40b4cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:47.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.697+0000 7f5feaf04640 1 -- 172.21.15.105:0/4050037910 shutdown_connections 2024-04-15T15:20:47.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.697+0000 7f5feaf04640 1 -- 172.21.15.105:0/4050037910 wait complete. 2024-04-15T15:20:47.710 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-15T15:20:47.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.812+0000 7fc71d146640 1 Processor -- start 2024-04-15T15:20:47.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.812+0000 7fc71d146640 1 -- start start 2024-04-15T15:20:47.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.813+0000 7fc71d146640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc71813e680 0x7fc7180cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:47.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.813+0000 7fc71d146640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc71815bc90 con 0x7fc71813ec60 2024-04-15T15:20:47.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.813+0000 7fc71d146640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc71815bf70 con 0x7fc71806f6a0 2024-04-15T15:20:47.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.813+0000 7fc71d146640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fc71815c250 con 0x7fc71813e680 2024-04-15T15:20:47.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.813+0000 7fc71659c640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fc71813ec60 0x7fc71806ef90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41214/0 (socket says 172.21.15.105:41214) 2024-04-15T15:20:47.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.813+0000 7fc71659c640 1 -- 172.21.15.105:0/1051547763 learned_addr learned my addr 172.21.15.105:0/1051547763 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:20:47.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.813+0000 7fc716d9d640 1 --2- 172.21.15.105:0/1051547763 >> v2:172.21.15.154:3300/0 conn(0x7fc71813e680 0x7fc7180cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:47.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.813+0000 7fc715d9b640 1 -- 172.21.15.105:0/1051547763 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 373536379 0 0) 0x7fc71815bc90 con 0x7fc71813ec60 2024-04-15T15:20:47.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.813+0000 7fc715d9b640 1 -- 172.21.15.105:0/1051547763 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc700003520 con 0x7fc71813ec60 2024-04-15T15:20:47.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.813+0000 7fc715d9b640 1 -- 172.21.15.105:0/1051547763 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1164549434 0 0) 0x7fc700003520 con 0x7fc71813ec60 2024-04-15T15:20:47.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.813+0000 7fc715d9b640 1 -- 172.21.15.105:0/1051547763 >> v2:172.21.15.154:3300/0 conn(0x7fc71813e680 msgr2=0x7fc7180cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:47.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.813+0000 7fc715d9b640 1 --2- 172.21.15.105:0/1051547763 >> v2:172.21.15.154:3300/0 conn(0x7fc71813e680 0x7fc7180cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:47.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.813+0000 7fc715d9b640 1 -- 172.21.15.105:0/1051547763 >> v1:172.21.15.154:6789/0 conn(0x7fc71806f6a0 legacy=0x7fc71815f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:47.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.813+0000 7fc715d9b640 1 -- 172.21.15.105:0/1051547763 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc71815c530 con 0x7fc71813ec60 2024-04-15T15:20:47.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.813+0000 7fc715d9b640 1 -- 172.21.15.105:0/1051547763 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7fc70c001690 con 0x7fc71813ec60 2024-04-15T15:20:47.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.813+0000 7fc715d9b640 1 -- 172.21.15.105:0/1051547763 >> v1:172.21.15.105:6789/0 conn(0x7fc71813ec60 legacy=0x7fc71806ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:47.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.813+0000 7fc715d9b640 1 --2- 172.21.15.105:0/1051547763 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7000039b0 0x7fc700003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:47.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.813+0000 7fc715d9b640 1 --2- 172.21.15.105:0/1051547763 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc700004800 0x7fc700004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:47.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.813+0000 7fc715d9b640 1 --2- 172.21.15.105:0/1051547763 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc7000051c0 0x7fc70000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:47.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.813+0000 7fc715d9b640 1 -- 172.21.15.105:0/1051547763 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc71815c250 con 0x7fc7000039b0 2024-04-15T15:20:47.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.813+0000 7fc715d9b640 1 -- 172.21.15.105:0/1051547763 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc70000c060 con 0x7fc700004800 2024-04-15T15:20:47.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.813+0000 7fc715d9b640 1 -- 172.21.15.105:0/1051547763 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fc70000c260 con 0x7fc7000051c0 2024-04-15T15:20:47.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.813+0000 7fc716d9d640 1 --2- 172.21.15.105:0/1051547763 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7000039b0 0x7fc700003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:47.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.814+0000 7fc71659c640 1 --2- 172.21.15.105:0/1051547763 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc700004800 0x7fc700004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:47.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.814+0000 7fc71759e640 1 --2- 172.21.15.105:0/1051547763 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc7000051c0 0x7fc70000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:47.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.814+0000 7fc716d9d640 1 -- 172.21.15.105:0/1051547763 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc7000051c0 msgr2=0x7fc70000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:47.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.814+0000 7fc716d9d640 1 --2- 172.21.15.105:0/1051547763 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc7000051c0 0x7fc70000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:47.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.814+0000 7fc716d9d640 1 -- 172.21.15.105:0/1051547763 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc700004800 msgr2=0x7fc700004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:47.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.814+0000 7fc716d9d640 1 --2- 172.21.15.105:0/1051547763 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc700004800 0x7fc700004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:47.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.814+0000 7fc716d9d640 1 -- 172.21.15.105:0/1051547763 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc700003520 con 0x7fc7000039b0 2024-04-15T15:20:47.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.814+0000 7fc71759e640 1 -- 172.21.15.105:0/1051547763 reap_dead start 2024-04-15T15:20:47.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.814+0000 7fc71659c640 1 --2- 172.21.15.105:0/1051547763 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc700004800 0x7fc700004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:20:47.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.814+0000 7fc716d9d640 1 --2- 172.21.15.105:0/1051547763 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7000039b0 0x7fc700003dd0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fc708009ac0 tx=0x7fc700005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c0fb5e8a6a33bfa server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:47.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.815+0000 7fc715d9b640 1 -- 172.21.15.105:0/1051547763 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc708039070 con 0x7fc7000039b0 2024-04-15T15:20:47.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.815+0000 7fc715d9b640 1 -- 172.21.15.105:0/1051547763 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc70802c6b0 con 0x7fc7000039b0 2024-04-15T15:20:47.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.815+0000 7fc715d9b640 1 -- 172.21.15.105:0/1051547763 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc708009690 con 0x7fc7000039b0 2024-04-15T15:20:47.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.815+0000 7fc71d146640 1 -- 172.21.15.105:0/1051547763 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7000039b0 msgr2=0x7fc700003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:47.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.815+0000 7fc71d146640 1 --2- 172.21.15.105:0/1051547763 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7000039b0 0x7fc700003dd0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fc708009ac0 tx=0x7fc700005830 comp rx=0 tx=0).stop 2024-04-15T15:20:47.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.815+0000 7fc71d146640 1 -- 172.21.15.105:0/1051547763 shutdown_connections 2024-04-15T15:20:47.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.815+0000 7fc71d146640 1 --2- 172.21.15.105:0/1051547763 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7000039b0 0x7fc700003dd0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:47.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.815+0000 7fc71d146640 1 -- 172.21.15.105:0/1051547763 >> 172.21.15.105:0/1051547763 conn(0x7fc718067d50 msgr2=0x7fc71806b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:47.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.815+0000 7fc71d146640 1 -- 172.21.15.105:0/1051547763 shutdown_connections 2024-04-15T15:20:47.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.815+0000 7fc71d146640 1 -- 172.21.15.105:0/1051547763 wait complete. 2024-04-15T15:20:47.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.817+0000 7fc71d146640 1 Processor -- start 2024-04-15T15:20:47.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.817+0000 7fc71d146640 1 -- start start 2024-04-15T15:20:47.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.817+0000 7fc71d146640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc7000039b0 0x7fc718156420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:47.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.817+0000 7fc71d146640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc71815d8e0 0x7fc71815dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:47.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.817+0000 7fc71d146640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc71815a510 0x7fc71815a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:47.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.817+0000 7fc71d146640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc7180cf710 con 0x7fc71815a510 2024-04-15T15:20:47.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.817+0000 7fc71d146640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc7180cf590 con 0x7fc7000039b0 2024-04-15T15:20:47.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.817+0000 7fc71d146640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fc71806ef90 con 0x7fc71815d8e0 2024-04-15T15:20:47.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.817+0000 7fc71759e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc71815a510 0x7fc71815a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:47.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.817+0000 7fc71759e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc71815a510 0x7fc71815a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58928/0 (socket says 172.21.15.105:58928) 2024-04-15T15:20:47.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.817+0000 7fc71759e640 1 -- 172.21.15.105:0/1522626518 learned_addr learned my addr 172.21.15.105:0/1522626518 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:47.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.818+0000 7fc71659c640 1 --2- 172.21.15.105:0/1522626518 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc71815d8e0 0x7fc71815dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:47.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.818+0000 7fc716d9d640 1 --2- 172.21.15.105:0/1522626518 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc7000039b0 0x7fc718156420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:47.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.818+0000 7fc71759e640 1 -- 172.21.15.105:0/1522626518 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc71815d8e0 msgr2=0x7fc71815dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:47.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.818+0000 7fc71759e640 1 --2- 172.21.15.105:0/1522626518 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc71815d8e0 0x7fc71815dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:47.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.818+0000 7fc71759e640 1 -- 172.21.15.105:0/1522626518 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc7000039b0 msgr2=0x7fc718156420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:47.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.818+0000 7fc71759e640 1 --2- 172.21.15.105:0/1522626518 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc7000039b0 0x7fc718156420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:47.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.818+0000 7fc71759e640 1 -- 172.21.15.105:0/1522626518 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc71815e3a0 con 0x7fc71815a510 2024-04-15T15:20:47.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.818+0000 7fc71759e640 1 --2- 172.21.15.105:0/1522626518 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc71815a510 0x7fc71815a990 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fc710007fd0 tx=0x7fc71000ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:47.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.818+0000 7fc6ff7fe640 1 -- 172.21.15.105:0/1522626518 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc710003970 con 0x7fc71815a510 2024-04-15T15:20:47.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.818+0000 7fc6ff7fe640 1 -- 172.21.15.105:0/1522626518 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc710018070 con 0x7fc71815a510 2024-04-15T15:20:47.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.818+0000 7fc6ff7fe640 1 -- 172.21.15.105:0/1522626518 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc710003d70 con 0x7fc71815a510 2024-04-15T15:20:47.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.818+0000 7fc71d146640 1 -- 172.21.15.105:0/1522626518 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc71815e680 con 0x7fc71815a510 2024-04-15T15:20:47.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.818+0000 7fc71d146640 1 -- 172.21.15.105:0/1522626518 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc71813b960 con 0x7fc71815a510 2024-04-15T15:20:47.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.819+0000 7fc71d146640 1 -- 172.21.15.105:0/1522626518 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc71813b470 con 0x7fc71815a510 2024-04-15T15:20:47.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.820+0000 7fc6ff7fe640 1 -- 172.21.15.105:0/1522626518 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fc7100050d0 con 0x7fc71815a510 2024-04-15T15:20:47.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.820+0000 7fc6ff7fe640 1 --2- 172.21.15.105:0/1522626518 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fc70406c980 0x7fc70406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:47.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.820+0000 7fc6ff7fe640 1 -- 172.21.15.105:0/1522626518 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fc71008b850 con 0x7fc71815a510 2024-04-15T15:20:47.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.820+0000 7fc716d9d640 1 --2- 172.21.15.105:0/1522626518 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fc70406c980 0x7fc70406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:47.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.821+0000 7fc716d9d640 1 --2- 172.21.15.105:0/1522626518 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fc70406c980 0x7fc70406ee40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc708036000 tx=0x7fc708030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:47.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:47.825+0000 7fc6ff7fe640 1 -- 172.21.15.105:0/1522626518 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc710059c90 con 0x7fc71815a510 2024-04-15T15:20:48.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.013+0000 7fc71d146640 1 -- 172.21.15.105:0/1522626518 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc718068a40 con 0x7fc71815a510 2024-04-15T15:20:48.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.013+0000 7fc6ff7fe640 1 -- 172.21.15.105:0/1522626518 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v34) ==== 80+0+0 (secure 0 0 0) 0x7fc710059590 con 0x7fc71815a510 2024-04-15T15:20:48.015 INFO:teuthology.orchestra.run.smithi105.stderr:listed 0 entries 2024-04-15T15:20:48.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.015+0000 7fc71d146640 1 -- 172.21.15.105:0/1522626518 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fc70406c980 msgr2=0x7fc70406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:48.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.015+0000 7fc71d146640 1 --2- 172.21.15.105:0/1522626518 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fc70406c980 0x7fc70406ee40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc708036000 tx=0x7fc708030000 comp rx=0 tx=0).stop 2024-04-15T15:20:48.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.015+0000 7fc71d146640 1 -- 172.21.15.105:0/1522626518 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc71815a510 msgr2=0x7fc71815a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:48.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.015+0000 7fc71d146640 1 --2- 172.21.15.105:0/1522626518 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc71815a510 0x7fc71815a990 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fc710007fd0 tx=0x7fc71000ec10 comp rx=0 tx=0).stop 2024-04-15T15:20:48.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.015+0000 7fc71d146640 1 -- 172.21.15.105:0/1522626518 shutdown_connections 2024-04-15T15:20:48.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.015+0000 7fc71d146640 1 --2- 172.21.15.105:0/1522626518 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc71815d8e0 0x7fc71815dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:48.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.015+0000 7fc71d146640 1 --2- 172.21.15.105:0/1522626518 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fc70406c980 0x7fc70406ee40 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:48.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.015+0000 7fc71d146640 1 --2- 172.21.15.105:0/1522626518 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc71815a510 0x7fc71815a990 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:48.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.015+0000 7fc71d146640 1 --2- 172.21.15.105:0/1522626518 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc7000039b0 0x7fc718156420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:48.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.015+0000 7fc71d146640 1 -- 172.21.15.105:0/1522626518 >> 172.21.15.105:0/1522626518 conn(0x7fc718067d50 msgr2=0x7fc7180cf890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:48.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.015+0000 7fc71d146640 1 -- 172.21.15.105:0/1522626518 shutdown_connections 2024-04-15T15:20:48.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.015+0000 7fc71d146640 1 -- 172.21.15.105:0/1522626518 wait complete. 2024-04-15T15:20:48.028 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-15T15:20:48.028 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:20:48.048 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:20:48.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.149+0000 7f39d19ba640 1 Processor -- start 2024-04-15T15:20:48.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.149+0000 7f39d19ba640 1 -- start start 2024-04-15T15:20:48.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.150+0000 7f39d19ba640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f39cc15ff90 0x7f39cc0af410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:48.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.150+0000 7f39d19ba640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39cc0af950 con 0x7f39cc0cdfe0 2024-04-15T15:20:48.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.150+0000 7f39d19ba640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39cc0cd110 con 0x7f39cc161100 2024-04-15T15:20:48.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.150+0000 7f39d19ba640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f39cc1615c0 con 0x7f39cc15ff90 2024-04-15T15:20:48.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.150+0000 7f39cb7fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f39cc0cdfe0 0x7f39cc0d0500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41218/0 (socket says 172.21.15.105:41218) 2024-04-15T15:20:48.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.150+0000 7f39cb7fe640 1 -- 172.21.15.105:0/2878339796 learned_addr learned my addr 172.21.15.105:0/2878339796 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:20:48.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.150+0000 7f39c9ffb640 1 -- 172.21.15.105:0/2878339796 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2080102465 0 0) 0x7f39cc0af950 con 0x7f39cc0cdfe0 2024-04-15T15:20:48.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.150+0000 7f39c9ffb640 1 -- 172.21.15.105:0/2878339796 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f39b4003520 con 0x7f39cc0cdfe0 2024-04-15T15:20:48.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.150+0000 7f39caffd640 1 --2- 172.21.15.105:0/2878339796 >> v2:172.21.15.154:3300/0 conn(0x7f39cc15ff90 0x7f39cc0af410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:48.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.150+0000 7f39c9ffb640 1 -- 172.21.15.105:0/2878339796 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1981341696 0 0) 0x7f39b4003520 con 0x7f39cc0cdfe0 2024-04-15T15:20:48.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.151+0000 7f39c9ffb640 1 -- 172.21.15.105:0/2878339796 >> v2:172.21.15.154:3300/0 conn(0x7f39cc15ff90 msgr2=0x7f39cc0af410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:48.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.151+0000 7f39c9ffb640 1 --2- 172.21.15.105:0/2878339796 >> v2:172.21.15.154:3300/0 conn(0x7f39cc15ff90 0x7f39cc0af410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:48.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.151+0000 7f39c9ffb640 1 -- 172.21.15.105:0/2878339796 >> v1:172.21.15.154:6789/0 conn(0x7f39cc161100 legacy=0x7f39cc0cd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:48.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.151+0000 7f39c9ffb640 1 -- 172.21.15.105:0/2878339796 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39cc0ce990 con 0x7f39cc0cdfe0 2024-04-15T15:20:48.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.151+0000 7f39c9ffb640 1 -- 172.21.15.105:0/2878339796 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7f39c40019f0 con 0x7f39cc0cdfe0 2024-04-15T15:20:48.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.151+0000 7f39caffd640 1 --2- 172.21.15.105:0/2878339796 >> v2:172.21.15.154:3300/0 conn(0x7f39cc15ff90 0x7f39cc0af410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:48.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.151+0000 7f39c9ffb640 1 -- 172.21.15.105:0/2878339796 >> v1:172.21.15.105:6789/0 conn(0x7f39cc0cdfe0 legacy=0x7f39cc0d0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:48.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.151+0000 7f39c9ffb640 1 --2- 172.21.15.105:0/2878339796 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39b40039b0 0x7f39b4003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:48.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.151+0000 7f39c9ffb640 1 --2- 172.21.15.105:0/2878339796 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39b4004800 0x7f39b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:48.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.151+0000 7f39c9ffb640 1 --2- 172.21.15.105:0/2878339796 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f39b40052e0 0x7f39b400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:48.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.151+0000 7f39c9ffb640 1 -- 172.21.15.105:0/2878339796 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f39cc1615c0 con 0x7f39b4004800 2024-04-15T15:20:48.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.151+0000 7f39c9ffb640 1 -- 172.21.15.105:0/2878339796 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f39b400c1e0 con 0x7f39b40039b0 2024-04-15T15:20:48.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.151+0000 7f39c9ffb640 1 -- 172.21.15.105:0/2878339796 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f39b400c3e0 con 0x7f39b40052e0 2024-04-15T15:20:48.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.151+0000 7f39ca7fc640 1 --2- 172.21.15.105:0/2878339796 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39b4004800 0x7f39b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:48.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.151+0000 7f39caffd640 1 --2- 172.21.15.105:0/2878339796 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39b40039b0 0x7f39b4003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:48.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.151+0000 7f39cb7fe640 1 --2- 172.21.15.105:0/2878339796 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f39b40052e0 0x7f39b400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:48.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.151+0000 7f39ca7fc640 1 -- 172.21.15.105:0/2878339796 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f39b40052e0 msgr2=0x7f39b400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:48.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.151+0000 7f39ca7fc640 1 --2- 172.21.15.105:0/2878339796 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f39b40052e0 0x7f39b400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:48.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.151+0000 7f39ca7fc640 1 -- 172.21.15.105:0/2878339796 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39b40039b0 msgr2=0x7f39b4003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:48.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.151+0000 7f39ca7fc640 1 --2- 172.21.15.105:0/2878339796 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39b40039b0 0x7f39b4003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:48.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.151+0000 7f39ca7fc640 1 -- 172.21.15.105:0/2878339796 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39b4003520 con 0x7f39b4004800 2024-04-15T15:20:48.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.151+0000 7f39cb7fe640 1 -- 172.21.15.105:0/2878339796 reap_dead start 2024-04-15T15:20:48.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.151+0000 7f39caffd640 1 --2- 172.21.15.105:0/2878339796 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39b40039b0 0x7f39b4003db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:20:48.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.152+0000 7f39ca7fc640 1 --2- 172.21.15.105:0/2878339796 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39b4004800 0x7f39b4004c80 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f39cc073d90 tx=0x7f39b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b29406d95e4e0bbd server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:48.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.152+0000 7f39c9ffb640 1 -- 172.21.15.105:0/2878339796 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c0037070 con 0x7f39b4004800 2024-04-15T15:20:48.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.152+0000 7f39c9ffb640 1 -- 172.21.15.105:0/2878339796 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39c0032600 con 0x7f39b4004800 2024-04-15T15:20:48.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.152+0000 7f39c9ffb640 1 -- 172.21.15.105:0/2878339796 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c00039a0 con 0x7f39b4004800 2024-04-15T15:20:48.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.152+0000 7f39d19ba640 1 -- 172.21.15.105:0/2878339796 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39b4004800 msgr2=0x7f39b4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:48.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.152+0000 7f39d19ba640 1 --2- 172.21.15.105:0/2878339796 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39b4004800 0x7f39b4004c80 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f39cc073d90 tx=0x7f39b4005830 comp rx=0 tx=0).stop 2024-04-15T15:20:48.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.152+0000 7f39d19ba640 1 -- 172.21.15.105:0/2878339796 shutdown_connections 2024-04-15T15:20:48.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.152+0000 7f39d19ba640 1 --2- 172.21.15.105:0/2878339796 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39b4004800 0x7f39b4004c80 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:48.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.152+0000 7f39d19ba640 1 -- 172.21.15.105:0/2878339796 >> 172.21.15.105:0/2878339796 conn(0x7f39cc0d2600 msgr2=0x7f39cc0d2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:48.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.153+0000 7f39d19ba640 1 -- 172.21.15.105:0/2878339796 shutdown_connections 2024-04-15T15:20:48.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.153+0000 7f39d19ba640 1 -- 172.21.15.105:0/2878339796 wait complete. 2024-04-15T15:20:48.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.154+0000 7f39d19ba640 1 Processor -- start 2024-04-15T15:20:48.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.154+0000 7f39d19ba640 1 -- start start 2024-04-15T15:20:48.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.155+0000 7f39d19ba640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39b4004800 0x7f39cc14ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:48.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.155+0000 7f39d19ba640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f39cc15ec00 0x7f39cc15f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:48.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.155+0000 7f39d19ba640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39cc0b2eb0 0x7f39cc0b3330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:48.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.155+0000 7f39d19ba640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f39cc0ce450 con 0x7f39b4004800 2024-04-15T15:20:48.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.155+0000 7f39d19ba640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f39cc0cd3b0 con 0x7f39cc0b2eb0 2024-04-15T15:20:48.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.155+0000 7f39d19ba640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f39cc160370 con 0x7f39cc15ec00 2024-04-15T15:20:48.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.155+0000 7f39caffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39b4004800 0x7f39cc14ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:48.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.155+0000 7f39caffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39b4004800 0x7f39cc14ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58946/0 (socket says 172.21.15.105:58946) 2024-04-15T15:20:48.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.155+0000 7f39caffd640 1 -- 172.21.15.105:0/2125333220 learned_addr learned my addr 172.21.15.105:0/2125333220 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:48.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.155+0000 7f39cb7fe640 1 --2- 172.21.15.105:0/2125333220 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39cc0b2eb0 0x7f39cc0b3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:48.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.155+0000 7f39ca7fc640 1 --2- 172.21.15.105:0/2125333220 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f39cc15ec00 0x7f39cc15f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:48.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.155+0000 7f39caffd640 1 -- 172.21.15.105:0/2125333220 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f39cc15ec00 msgr2=0x7f39cc15f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:48.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.155+0000 7f39caffd640 1 --2- 172.21.15.105:0/2125333220 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f39cc15ec00 0x7f39cc15f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:48.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.155+0000 7f39caffd640 1 -- 172.21.15.105:0/2125333220 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39cc0b2eb0 msgr2=0x7f39cc0b3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:48.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.155+0000 7f39caffd640 1 --2- 172.21.15.105:0/2125333220 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39cc0b2eb0 0x7f39cc0b3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:48.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.155+0000 7f39caffd640 1 -- 172.21.15.105:0/2125333220 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39cc0cabc0 con 0x7f39b4004800 2024-04-15T15:20:48.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.155+0000 7f39cb7fe640 1 --2- 172.21.15.105:0/2125333220 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39cc0b2eb0 0x7f39cc0b3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:20:48.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.155+0000 7f39caffd640 1 --2- 172.21.15.105:0/2125333220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39b4004800 0x7f39cc14ff30 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f39bc00f4c0 tx=0x7f39bc00f980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:48.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.155+0000 7f39b3fff640 1 -- 172.21.15.105:0/2125333220 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39bc018070 con 0x7f39b4004800 2024-04-15T15:20:48.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.155+0000 7f39b3fff640 1 -- 172.21.15.105:0/2125333220 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39bc014890 con 0x7f39b4004800 2024-04-15T15:20:48.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.155+0000 7f39d19ba640 1 -- 172.21.15.105:0/2125333220 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39cc0caea0 con 0x7f39b4004800 2024-04-15T15:20:48.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.155+0000 7f39b3fff640 1 -- 172.21.15.105:0/2125333220 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39bc014c40 con 0x7f39b4004800 2024-04-15T15:20:48.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.155+0000 7f39d19ba640 1 -- 172.21.15.105:0/2125333220 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39cc0cb460 con 0x7f39b4004800 2024-04-15T15:20:48.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.156+0000 7f39d19ba640 1 -- 172.21.15.105:0/2125333220 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39cc0b3870 con 0x7f39b4004800 2024-04-15T15:20:48.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.157+0000 7f39b3fff640 1 -- 172.21.15.105:0/2125333220 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f39bc01b070 con 0x7f39b4004800 2024-04-15T15:20:48.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.157+0000 7f39b3fff640 1 --2- 172.21.15.105:0/2125333220 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f39b806c980 0x7f39b806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:48.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.157+0000 7f39b3fff640 1 -- 172.21.15.105:0/2125333220 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f39bc08be40 con 0x7f39b4004800 2024-04-15T15:20:48.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.157+0000 7f39ca7fc640 1 --2- 172.21.15.105:0/2125333220 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f39b806c980 0x7f39b806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:48.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.158+0000 7f39ca7fc640 1 --2- 172.21.15.105:0/2125333220 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f39b806c980 0x7f39b806ee40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f39c0034000 tx=0x7f39c0009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:48.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.162+0000 7f39b3fff640 1 -- 172.21.15.105:0/2125333220 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f39bc05a170 con 0x7f39b4004800 2024-04-15T15:20:48.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.336+0000 7f39d19ba640 1 -- 172.21.15.105:0/2125333220 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f39cc068a40 con 0x7f39b4004800 2024-04-15T15:20:48.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.337+0000 7f39b3fff640 1 -- 172.21.15.105:0/2125333220 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16717 (secure 0 0 0) 0x7f39bc059a70 con 0x7f39b4004800 2024-04-15T15:20:48.339 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:20:48.339 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":34,"fsid":"09e4d2a4-a324-4793-af77-644b25e520e1","created":"2024-04-15T15:19:01.524543+0000","modified":"2024-04-15T15:20:30.645565+0000","last_up_change":"2024-04-15T15:19:12.825815+0000","last_in_change":"2024-04-15T15:19:03.788126+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:19:13.748585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:20:26.983353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-15T15:20:27.961802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6f6292b-2d7e-4aa0-9d1c-6ddd0f97287c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6801","nonce":2873040054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6803","nonce":2873040054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6807","nonce":2873040054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6805","nonce":2873040054}]},"public_addr":"172.21.15.105:6801/2873040054","cluster_addr":"172.21.15.105:6803/2873040054","heartbeat_back_addr":"172.21.15.105:6807/2873040054","heartbeat_front_addr":"172.21.15.105:6805/2873040054","state":["exists","up"]},{"osd":1,"uuid":"8900966a-b393-41e3-8f93-e5a43bd23622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6818","nonce":2494994479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6821","nonce":2494994479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6828","nonce":2494994479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6823","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6825","nonce":2494994479}]},"public_addr":"172.21.15.105:6818/2494994479","cluster_addr":"172.21.15.105:6821/2494994479","heartbeat_back_addr":"172.21.15.105:6828/2494994479","heartbeat_front_addr":"172.21.15.105:6825/2494994479","state":["exists","up"]},{"osd":2,"uuid":"81a5dc7a-9325-4ff5-941d-a2d5fba21f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6809","nonce":3798408699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6811","nonce":3798408699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6815","nonce":3798408699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6813","nonce":3798408699}]},"public_addr":"172.21.15.105:6809/3798408699","cluster_addr":"172.21.15.105:6811/3798408699","heartbeat_back_addr":"172.21.15.105:6815/3798408699","heartbeat_front_addr":"172.21.15.105:6813/3798408699","state":["exists","up"]},{"osd":3,"uuid":"df766ed2-5c71-4805-96f4-3d4970edc960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6819","nonce":1920720186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6824","nonce":1920720186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6831","nonce":1920720186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6827","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6829","nonce":1920720186}]},"public_addr":"172.21.15.105:6819/1920720186","cluster_addr":"172.21.15.105:6824/1920720186","heartbeat_back_addr":"172.21.15.105:6831/1920720186","heartbeat_front_addr":"172.21.15.105:6829/1920720186","state":["exists","up"]},{"osd":4,"uuid":"e1039e03-28b8-49df-b951-c91930b93c9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6810","nonce":3516494456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6813","nonce":3516494456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6820","nonce":3516494456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6817","nonce":3516494456}]},"public_addr":"172.21.15.154:6810/3516494456","cluster_addr":"172.21.15.154:6813/3516494456","heartbeat_back_addr":"172.21.15.154:6820/3516494456","heartbeat_front_addr":"172.21.15.154:6817/3516494456","state":["exists","up"]},{"osd":5,"uuid":"a9f5eaf1-66ff-405a-a9e4-ec3340812a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6801","nonce":540367751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6803","nonce":540367751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6807","nonce":540367751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6805","nonce":540367751}]},"public_addr":"172.21.15.154:6801/540367751","cluster_addr":"172.21.15.154:6803/540367751","heartbeat_back_addr":"172.21.15.154:6807/540367751","heartbeat_front_addr":"172.21.15.154:6805/540367751","state":["exists","up"]},{"osd":6,"uuid":"267d02ea-675f-428f-aae8-e0375e4d4feb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6825","nonce":1550829005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6827","nonce":1550829005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6831","nonce":1550829005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6829","nonce":1550829005}]},"public_addr":"172.21.15.154:6825/1550829005","cluster_addr":"172.21.15.154:6827/1550829005","heartbeat_back_addr":"172.21.15.154:6831/1550829005","heartbeat_front_addr":"172.21.15.154:6829/1550829005","state":["exists","up"]},{"osd":7,"uuid":"759ac6cc-896a-423c-885f-9fe90d8dcb14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6809","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6811","nonce":3920089903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6816","nonce":3920089903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6823","nonce":3920089903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6819","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6821","nonce":3920089903}]},"public_addr":"172.21.15.154:6811/3920089903","cluster_addr":"172.21.15.154:6816/3920089903","heartbeat_back_addr":"172.21.15.154:6823/3920089903","heartbeat_front_addr":"172.21.15.154:6821/3920089903","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.949509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.984072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.907278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.947391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.975447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.998896+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:09.028739+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.894119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:20:48.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.339+0000 7f39d19ba640 1 -- 172.21.15.105:0/2125333220 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f39b806c980 msgr2=0x7f39b806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:48.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.339+0000 7f39d19ba640 1 --2- 172.21.15.105:0/2125333220 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f39b806c980 0x7f39b806ee40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f39c0034000 tx=0x7f39c0009030 comp rx=0 tx=0).stop 2024-04-15T15:20:48.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.339+0000 7f39d19ba640 1 -- 172.21.15.105:0/2125333220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39b4004800 msgr2=0x7f39cc14ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:48.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.339+0000 7f39d19ba640 1 --2- 172.21.15.105:0/2125333220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39b4004800 0x7f39cc14ff30 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f39bc00f4c0 tx=0x7f39bc00f980 comp rx=0 tx=0).stop 2024-04-15T15:20:48.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.339+0000 7f39d19ba640 1 -- 172.21.15.105:0/2125333220 shutdown_connections 2024-04-15T15:20:48.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.339+0000 7f39d19ba640 1 --2- 172.21.15.105:0/2125333220 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f39cc15ec00 0x7f39cc15f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:48.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.339+0000 7f39d19ba640 1 --2- 172.21.15.105:0/2125333220 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f39b806c980 0x7f39b806ee40 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:48.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.339+0000 7f39d19ba640 1 --2- 172.21.15.105:0/2125333220 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39cc0b2eb0 0x7f39cc0b3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:48.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.339+0000 7f39d19ba640 1 --2- 172.21.15.105:0/2125333220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39b4004800 0x7f39cc14ff30 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:48.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.339+0000 7f39d19ba640 1 -- 172.21.15.105:0/2125333220 >> 172.21.15.105:0/2125333220 conn(0x7f39cc0d2600 msgr2=0x7f39cc0ceeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:48.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.340+0000 7f39d19ba640 1 -- 172.21.15.105:0/2125333220 shutdown_connections 2024-04-15T15:20:48.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.340+0000 7f39d19ba640 1 -- 172.21.15.105:0/2125333220 wait complete. 2024-04-15T15:20:48.353 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:19:13.748585+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-15T15:20:26.983353+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-15T15:20:27.961802+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:20:48.353 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:20:48.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.458+0000 7f50e20a0640 1 Processor -- start 2024-04-15T15:20:48.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.458+0000 7f50e20a0640 1 -- start start 2024-04-15T15:20:48.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.459+0000 7f50e20a0640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f50dc0d12c0 0x7f50dc15d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:48.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.459+0000 7f50e20a0640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50dc15caf0 con 0x7f50dc09aa40 2024-04-15T15:20:48.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.459+0000 7f50e20a0640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50dc0d0220 con 0x7f50dc09b8f0 2024-04-15T15:20:48.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.459+0000 7f50e20a0640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f50dc0d0e30 con 0x7f50dc0d12c0 2024-04-15T15:20:48.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.459+0000 7f50db7fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f50dc09aa40 0x7f50dc09ae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41234/0 (socket says 172.21.15.105:41234) 2024-04-15T15:20:48.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.459+0000 7f50db7fe640 1 -- 172.21.15.105:0/1655461509 learned_addr learned my addr 172.21.15.105:0/1655461509 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:20:48.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.459+0000 7f50da7fc640 1 -- 172.21.15.105:0/1655461509 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1118234929 0 0) 0x7f50dc15caf0 con 0x7f50dc09aa40 2024-04-15T15:20:48.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.459+0000 7f50da7fc640 1 -- 172.21.15.105:0/1655461509 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f50c4003520 con 0x7f50dc09aa40 2024-04-15T15:20:48.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.459+0000 7f50daffd640 1 --2- 172.21.15.105:0/1655461509 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f50dc0d12c0 0x7f50dc15d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:48.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.459+0000 7f50da7fc640 1 -- 172.21.15.105:0/1655461509 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1218144284 0 0) 0x7f50c4003520 con 0x7f50dc09aa40 2024-04-15T15:20:48.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.459+0000 7f50da7fc640 1 -- 172.21.15.105:0/1655461509 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f50dc0d12c0 msgr2=0x7f50dc15d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:48.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.459+0000 7f50da7fc640 1 --2- 172.21.15.105:0/1655461509 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f50dc0d12c0 0x7f50dc15d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:48.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.459+0000 7f50da7fc640 1 -- 172.21.15.105:0/1655461509 >> v1:172.21.15.154:6789/0 conn(0x7f50dc09b8f0 legacy=0x7f50dc0cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:48.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.459+0000 7f50da7fc640 1 -- 172.21.15.105:0/1655461509 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50dc0d0500 con 0x7f50dc09aa40 2024-04-15T15:20:48.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.459+0000 7f50da7fc640 1 -- 172.21.15.105:0/1655461509 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7f50cc001670 con 0x7f50dc09aa40 2024-04-15T15:20:48.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.459+0000 7f50da7fc640 1 -- 172.21.15.105:0/1655461509 >> v1:172.21.15.105:6789/0 conn(0x7f50dc09aa40 legacy=0x7f50dc09ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:48.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.460+0000 7f50da7fc640 1 --2- 172.21.15.105:0/1655461509 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f50dc0d12c0 0x7f50c40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:48.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.460+0000 7f50da7fc640 1 --2- 172.21.15.105:0/1655461509 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50c4003f60 0x7f50c4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:48.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.460+0000 7f50da7fc640 1 --2- 172.21.15.105:0/1655461509 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50c4004d40 0x7f50c40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:48.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.460+0000 7f50da7fc640 1 -- 172.21.15.105:0/1655461509 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f50dc0d0e30 con 0x7f50c4003f60 2024-04-15T15:20:48.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.460+0000 7f50da7fc640 1 -- 172.21.15.105:0/1655461509 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f50c400baa0 con 0x7f50c4004d40 2024-04-15T15:20:48.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.460+0000 7f50da7fc640 1 -- 172.21.15.105:0/1655461509 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f50c400bca0 con 0x7f50dc0d12c0 2024-04-15T15:20:48.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.460+0000 7f50daffd640 1 --2- 172.21.15.105:0/1655461509 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50c4003f60 0x7f50c4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:48.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.460+0000 7f50db7fe640 1 --2- 172.21.15.105:0/1655461509 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f50dc0d12c0 0x7f50c40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:48.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.460+0000 7f50dbfff640 1 --2- 172.21.15.105:0/1655461509 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50c4004d40 0x7f50c40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:48.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.460+0000 7f50daffd640 1 -- 172.21.15.105:0/1655461509 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f50dc0d12c0 msgr2=0x7f50c40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:48.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.460+0000 7f50daffd640 1 --2- 172.21.15.105:0/1655461509 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f50dc0d12c0 0x7f50c40039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:48.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.460+0000 7f50daffd640 1 -- 172.21.15.105:0/1655461509 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50c4004d40 msgr2=0x7f50c40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:48.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.460+0000 7f50daffd640 1 --2- 172.21.15.105:0/1655461509 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50c4004d40 0x7f50c40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:48.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.460+0000 7f50daffd640 1 -- 172.21.15.105:0/1655461509 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50c4003520 con 0x7f50c4003f60 2024-04-15T15:20:48.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.460+0000 7f50dbfff640 1 --2- 172.21.15.105:0/1655461509 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50c4004d40 0x7f50c40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:20:48.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.461+0000 7f50daffd640 1 --2- 172.21.15.105:0/1655461509 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50c4003f60 0x7f50c4004800 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f50d0009ac0 tx=0x7f50c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42b66feda4042e2e server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:48.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.461+0000 7f50da7fc640 1 -- 172.21.15.105:0/1655461509 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d0039070 con 0x7f50c4003f60 2024-04-15T15:20:48.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.461+0000 7f50da7fc640 1 -- 172.21.15.105:0/1655461509 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50d002c6b0 con 0x7f50c4003f60 2024-04-15T15:20:48.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.461+0000 7f50da7fc640 1 -- 172.21.15.105:0/1655461509 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d0009690 con 0x7f50c4003f60 2024-04-15T15:20:48.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.461+0000 7f50e20a0640 1 -- 172.21.15.105:0/1655461509 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50c4003f60 msgr2=0x7f50c4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:48.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.461+0000 7f50e20a0640 1 --2- 172.21.15.105:0/1655461509 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50c4003f60 0x7f50c4004800 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f50d0009ac0 tx=0x7f50c4005830 comp rx=0 tx=0).stop 2024-04-15T15:20:48.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.461+0000 7f50dbfff640 1 -- 172.21.15.105:0/1655461509 reap_dead start 2024-04-15T15:20:48.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.461+0000 7f50e20a0640 1 -- 172.21.15.105:0/1655461509 shutdown_connections 2024-04-15T15:20:48.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.461+0000 7f50e20a0640 1 -- 172.21.15.105:0/1655461509 >> 172.21.15.105:0/1655461509 conn(0x7f50dc0b4030 msgr2=0x7f50dc0b4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:48.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.461+0000 7f50e20a0640 1 -- 172.21.15.105:0/1655461509 shutdown_connections 2024-04-15T15:20:48.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.461+0000 7f50e20a0640 1 -- 172.21.15.105:0/1655461509 wait complete. 2024-04-15T15:20:48.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.463+0000 7f50e20a0640 1 Processor -- start 2024-04-15T15:20:48.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.463+0000 7f50e20a0640 1 -- start start 2024-04-15T15:20:48.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.463+0000 7f50e20a0640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f50c4003f60 0x7f50dc15bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:48.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.463+0000 7f50e20a0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50dc0cbc20 0x7f50dc0ca020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:48.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.463+0000 7f50e20a0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50dc0b7010 0x7f50dc0b7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:48.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.463+0000 7f50e20a0640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f50dc15d6a0 con 0x7f50dc0b7010 2024-04-15T15:20:48.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.463+0000 7f50e20a0640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f50dc0d0ac0 con 0x7f50dc0cbc20 2024-04-15T15:20:48.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.463+0000 7f50e20a0640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f50dc15d820 con 0x7f50c4003f60 2024-04-15T15:20:48.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.464+0000 7f50dbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50dc0b7010 0x7f50dc0b7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:48.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.464+0000 7f50dbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50dc0b7010 0x7f50dc0b7490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58964/0 (socket says 172.21.15.105:58964) 2024-04-15T15:20:48.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.464+0000 7f50dbfff640 1 -- 172.21.15.105:0/764050988 learned_addr learned my addr 172.21.15.105:0/764050988 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:48.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.464+0000 7f50db7fe640 1 --2- 172.21.15.105:0/764050988 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f50c4003f60 0x7f50dc15bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:48.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.464+0000 7f50daffd640 1 --2- 172.21.15.105:0/764050988 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50dc0cbc20 0x7f50dc0ca020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:48.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.464+0000 7f50dbfff640 1 -- 172.21.15.105:0/764050988 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f50c4003f60 msgr2=0x7f50dc15bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:48.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.464+0000 7f50dbfff640 1 --2- 172.21.15.105:0/764050988 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f50c4003f60 0x7f50dc15bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:48.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.464+0000 7f50dbfff640 1 -- 172.21.15.105:0/764050988 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50dc0cbc20 msgr2=0x7f50dc0ca020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:48.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.464+0000 7f50dbfff640 1 --2- 172.21.15.105:0/764050988 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50dc0cbc20 0x7f50dc0ca020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:48.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.464+0000 7f50dbfff640 1 -- 172.21.15.105:0/764050988 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50dc0ca560 con 0x7f50dc0b7010 2024-04-15T15:20:48.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.464+0000 7f50dbfff640 1 --2- 172.21.15.105:0/764050988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50dc0b7010 0x7f50dc0b7490 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f50dc0b1f10 tx=0x7f50d400ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:48.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.464+0000 7f50c3fff640 1 -- 172.21.15.105:0/764050988 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d4003b80 con 0x7f50dc0b7010 2024-04-15T15:20:48.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.464+0000 7f50c3fff640 1 -- 172.21.15.105:0/764050988 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50d4018070 con 0x7f50dc0b7010 2024-04-15T15:20:48.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.464+0000 7f50e20a0640 1 -- 172.21.15.105:0/764050988 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50dc0ca800 con 0x7f50dc0b7010 2024-04-15T15:20:48.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.464+0000 7f50c3fff640 1 -- 172.21.15.105:0/764050988 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d40050d0 con 0x7f50dc0b7010 2024-04-15T15:20:48.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.464+0000 7f50e20a0640 1 -- 172.21.15.105:0/764050988 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50dc0cadc0 con 0x7f50dc0b7010 2024-04-15T15:20:48.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.465+0000 7f50e20a0640 1 -- 172.21.15.105:0/764050988 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50dc068ab0 con 0x7f50dc0b7010 2024-04-15T15:20:48.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.466+0000 7f50c3fff640 1 -- 172.21.15.105:0/764050988 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f50d4005270 con 0x7f50dc0b7010 2024-04-15T15:20:48.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.466+0000 7f50c3fff640 1 --2- 172.21.15.105:0/764050988 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f50c806c980 0x7f50c806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:48.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.466+0000 7f50c3fff640 1 -- 172.21.15.105:0/764050988 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f50d408b500 con 0x7f50dc0b7010 2024-04-15T15:20:48.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.466+0000 7f50db7fe640 1 --2- 172.21.15.105:0/764050988 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f50c806c980 0x7f50c806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:48.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.467+0000 7f50db7fe640 1 --2- 172.21.15.105:0/764050988 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f50c806c980 0x7f50c806ee40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f50cc001cd0 tx=0x7f50cc007040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:48.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.470+0000 7f50c3fff640 1 -- 172.21.15.105:0/764050988 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f50d4022020 con 0x7f50dc0b7010 2024-04-15T15:20:48.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.643+0000 7f50e20a0640 1 -- 172.21.15.105:0/764050988 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f50dc1f27b0 con 0x7f50dc0b7010 2024-04-15T15:20:48.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.644+0000 7f50c3fff640 1 -- 172.21.15.105:0/764050988 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) ==== 93+0+10 (secure 0 0 0) 0x7f50d40592f0 con 0x7f50dc0b7010 2024-04-15T15:20:48.646 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-04-15T15:20:48.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.646+0000 7f50e20a0640 1 -- 172.21.15.105:0/764050988 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f50c806c980 msgr2=0x7f50c806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:48.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.646+0000 7f50e20a0640 1 --2- 172.21.15.105:0/764050988 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f50c806c980 0x7f50c806ee40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f50cc001cd0 tx=0x7f50cc007040 comp rx=0 tx=0).stop 2024-04-15T15:20:48.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.646+0000 7f50e20a0640 1 -- 172.21.15.105:0/764050988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50dc0b7010 msgr2=0x7f50dc0b7490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:48.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.646+0000 7f50e20a0640 1 --2- 172.21.15.105:0/764050988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50dc0b7010 0x7f50dc0b7490 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f50dc0b1f10 tx=0x7f50d400ead0 comp rx=0 tx=0).stop 2024-04-15T15:20:48.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.646+0000 7f50e20a0640 1 -- 172.21.15.105:0/764050988 shutdown_connections 2024-04-15T15:20:48.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.646+0000 7f50e20a0640 1 --2- 172.21.15.105:0/764050988 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f50c806c980 0x7f50c806ee40 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:48.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.646+0000 7f50e20a0640 1 --2- 172.21.15.105:0/764050988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50dc0b7010 0x7f50dc0b7490 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:48.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.646+0000 7f50e20a0640 1 --2- 172.21.15.105:0/764050988 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50dc0cbc20 0x7f50dc0ca020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:48.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.646+0000 7f50e20a0640 1 --2- 172.21.15.105:0/764050988 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f50c4003f60 0x7f50dc15bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:48.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.646+0000 7f50e20a0640 1 -- 172.21.15.105:0/764050988 >> 172.21.15.105:0/764050988 conn(0x7f50dc0b4030 msgr2=0x7f50dc071050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:48.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.646+0000 7f50e20a0640 1 -- 172.21.15.105:0/764050988 shutdown_connections 2024-04-15T15:20:48.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.646+0000 7f50e20a0640 1 -- 172.21.15.105:0/764050988 wait complete. 2024-04-15T15:20:48.659 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-15T15:20:48.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.763+0000 7f01243cd640 1 Processor -- start 2024-04-15T15:20:48.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.763+0000 7f01243cd640 1 -- start start 2024-04-15T15:20:48.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.764+0000 7f01243cd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f011c0d0f10 0x7f011c0cd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:48.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.764+0000 7f01243cd640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f011c15a6c0 con 0x7f011c06c3f0 2024-04-15T15:20:48.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.764+0000 7f01243cd640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f011c15cf30 con 0x7f011c15a2e0 2024-04-15T15:20:48.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.764+0000 7f01243cd640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f011c0cd5a0 con 0x7f011c0d0f10 2024-04-15T15:20:48.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.764+0000 7f012296a640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f011c06c3f0 0x7f011c1604c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41248/0 (socket says 172.21.15.105:41248) 2024-04-15T15:20:48.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.764+0000 7f012296a640 1 -- 172.21.15.105:0/1012471673 learned_addr learned my addr 172.21.15.105:0/1012471673 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:20:48.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.764+0000 7f0121968640 1 --2- 172.21.15.105:0/1012471673 >> v2:172.21.15.154:3300/0 conn(0x7f011c0d0f10 0x7f011c0cd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:48.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.764+0000 7f0121167640 1 -- 172.21.15.105:0/1012471673 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 259279762 0 0) 0x7f011c15a6c0 con 0x7f011c06c3f0 2024-04-15T15:20:48.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.764+0000 7f0121167640 1 -- 172.21.15.105:0/1012471673 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0104003520 con 0x7f011c06c3f0 2024-04-15T15:20:48.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.765+0000 7f0121167640 1 -- 172.21.15.105:0/1012471673 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3064525361 0 0) 0x7f0104003520 con 0x7f011c06c3f0 2024-04-15T15:20:48.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.765+0000 7f0121167640 1 -- 172.21.15.105:0/1012471673 >> v2:172.21.15.154:3300/0 conn(0x7f011c0d0f10 msgr2=0x7f011c0cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:48.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.765+0000 7f0121167640 1 --2- 172.21.15.105:0/1012471673 >> v2:172.21.15.154:3300/0 conn(0x7f011c0d0f10 0x7f011c0cd060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:48.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.765+0000 7f0121167640 1 -- 172.21.15.105:0/1012471673 >> v1:172.21.15.154:6789/0 conn(0x7f011c15a2e0 legacy=0x7f011c0d0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:48.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.765+0000 7f0121167640 1 -- 172.21.15.105:0/1012471673 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f011c15d210 con 0x7f011c06c3f0 2024-04-15T15:20:48.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.765+0000 7f0121167640 1 -- 172.21.15.105:0/1012471673 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7f010c001670 con 0x7f011c06c3f0 2024-04-15T15:20:48.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.765+0000 7f0121167640 1 -- 172.21.15.105:0/1012471673 >> v1:172.21.15.105:6789/0 conn(0x7f011c06c3f0 legacy=0x7f011c1604c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:48.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.765+0000 7f0121167640 1 --2- 172.21.15.105:0/1012471673 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f01040039b0 0x7f0104003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:48.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.765+0000 7f0121167640 1 --2- 172.21.15.105:0/1012471673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0104004800 0x7f0104004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:48.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.765+0000 7f0121167640 1 --2- 172.21.15.105:0/1012471673 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f01040051c0 0x7f010400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:48.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.765+0000 7f0121167640 1 -- 172.21.15.105:0/1012471673 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f010400c030 con 0x7f0104004800 2024-04-15T15:20:48.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.765+0000 7f0121167640 1 -- 172.21.15.105:0/1012471673 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f010400c1e0 con 0x7f01040039b0 2024-04-15T15:20:48.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.765+0000 7f0121167640 1 -- 172.21.15.105:0/1012471673 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f010400c3e0 con 0x7f01040051c0 2024-04-15T15:20:48.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.765+0000 7f0121968640 1 --2- 172.21.15.105:0/1012471673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0104004800 0x7f0104004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:48.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.765+0000 7f0122169640 1 --2- 172.21.15.105:0/1012471673 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f01040039b0 0x7f0104003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:48.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.765+0000 7f012296a640 1 --2- 172.21.15.105:0/1012471673 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f01040051c0 0x7f010400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:48.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.765+0000 7f0121968640 1 -- 172.21.15.105:0/1012471673 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f01040051c0 msgr2=0x7f010400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:48.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.765+0000 7f0121968640 1 --2- 172.21.15.105:0/1012471673 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f01040051c0 0x7f010400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:48.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.765+0000 7f0121968640 1 -- 172.21.15.105:0/1012471673 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f01040039b0 msgr2=0x7f0104003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:48.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.765+0000 7f0121968640 1 --2- 172.21.15.105:0/1012471673 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f01040039b0 0x7f0104003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:48.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.765+0000 7f0121968640 1 -- 172.21.15.105:0/1012471673 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0104003520 con 0x7f0104004800 2024-04-15T15:20:48.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.765+0000 7f012296a640 1 --2- 172.21.15.105:0/1012471673 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f01040051c0 0x7f010400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:20:48.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.765+0000 7f012296a640 1 -- 172.21.15.105:0/1012471673 reap_dead start 2024-04-15T15:20:48.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.766+0000 7f0121968640 1 --2- 172.21.15.105:0/1012471673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0104004800 0x7f0104004c80 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f011c0b4020 tx=0x7f0104005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76ab273d628f923f server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:48.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.766+0000 7f0121167640 1 -- 172.21.15.105:0/1012471673 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0110039070 con 0x7f0104004800 2024-04-15T15:20:48.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.766+0000 7f0121167640 1 -- 172.21.15.105:0/1012471673 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f011002cb90 con 0x7f0104004800 2024-04-15T15:20:48.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.766+0000 7f0121167640 1 -- 172.21.15.105:0/1012471673 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0110009c10 con 0x7f0104004800 2024-04-15T15:20:48.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.766+0000 7f01243cd640 1 -- 172.21.15.105:0/1012471673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0104004800 msgr2=0x7f0104004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:48.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.766+0000 7f01243cd640 1 --2- 172.21.15.105:0/1012471673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0104004800 0x7f0104004c80 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f011c0b4020 tx=0x7f0104005830 comp rx=0 tx=0).stop 2024-04-15T15:20:48.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.767+0000 7f01243cd640 1 -- 172.21.15.105:0/1012471673 shutdown_connections 2024-04-15T15:20:48.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.767+0000 7f01243cd640 1 --2- 172.21.15.105:0/1012471673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0104004800 0x7f0104004c80 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:48.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.767+0000 7f01243cd640 1 -- 172.21.15.105:0/1012471673 >> 172.21.15.105:0/1012471673 conn(0x7f011c0b2fc0 msgr2=0x7f011c0b33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:48.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.767+0000 7f01243cd640 1 -- 172.21.15.105:0/1012471673 shutdown_connections 2024-04-15T15:20:48.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.767+0000 7f01243cd640 1 -- 172.21.15.105:0/1012471673 wait complete. 2024-04-15T15:20:48.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.768+0000 7f01243cd640 1 Processor -- start 2024-04-15T15:20:48.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.768+0000 7f01243cd640 1 -- start start 2024-04-15T15:20:48.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.768+0000 7f01243cd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0104004800 0x7f011c0c4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:48.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.769+0000 7f01243cd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f011c1f2660 0x7f011c157630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:48.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.769+0000 7f01243cd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f011c1f2ac0 0x7f011c157b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:48.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.769+0000 7f01243cd640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f011c15d7d0 con 0x7f0104004800 2024-04-15T15:20:48.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.769+0000 7f01243cd640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f011c15db40 con 0x7f011c1f2ac0 2024-04-15T15:20:48.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.769+0000 7f01243cd640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f011c0b3ae0 con 0x7f011c1f2660 2024-04-15T15:20:48.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.769+0000 7f0122169640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0104004800 0x7f011c0c4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:48.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.769+0000 7f0122169640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0104004800 0x7f011c0c4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58980/0 (socket says 172.21.15.105:58980) 2024-04-15T15:20:48.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.769+0000 7f0122169640 1 -- 172.21.15.105:0/1835271282 learned_addr learned my addr 172.21.15.105:0/1835271282 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:48.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.769+0000 7f0121968640 1 --2- 172.21.15.105:0/1835271282 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f011c1f2660 0x7f011c157630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:48.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.769+0000 7f012296a640 1 --2- 172.21.15.105:0/1835271282 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f011c1f2ac0 0x7f011c157b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:48.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.769+0000 7f0122169640 1 -- 172.21.15.105:0/1835271282 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f011c1f2660 msgr2=0x7f011c157630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:48.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.769+0000 7f0122169640 1 --2- 172.21.15.105:0/1835271282 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f011c1f2660 0x7f011c157630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:48.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.769+0000 7f0122169640 1 -- 172.21.15.105:0/1835271282 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f011c1f2ac0 msgr2=0x7f011c157b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:48.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.769+0000 7f0122169640 1 --2- 172.21.15.105:0/1835271282 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f011c1f2ac0 0x7f011c157b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:48.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.769+0000 7f0122169640 1 -- 172.21.15.105:0/1835271282 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f011c0c7750 con 0x7f0104004800 2024-04-15T15:20:48.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.769+0000 7f0121968640 1 --2- 172.21.15.105:0/1835271282 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f011c1f2660 0x7f011c157630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:20:48.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.769+0000 7f012296a640 1 --2- 172.21.15.105:0/1835271282 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f011c1f2ac0 0x7f011c157b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:20:48.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.769+0000 7f0122169640 1 --2- 172.21.15.105:0/1835271282 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0104004800 0x7f011c0c4f60 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f011400f820 tx=0x7f011400fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:48.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.769+0000 7f011affd640 1 -- 172.21.15.105:0/1835271282 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01140050d0 con 0x7f0104004800 2024-04-15T15:20:48.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.769+0000 7f011affd640 1 -- 172.21.15.105:0/1835271282 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0114018070 con 0x7f0104004800 2024-04-15T15:20:48.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.769+0000 7f01243cd640 1 -- 172.21.15.105:0/1835271282 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f011c0c7a30 con 0x7f0104004800 2024-04-15T15:20:48.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.769+0000 7f011affd640 1 -- 172.21.15.105:0/1835271282 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0114005490 con 0x7f0104004800 2024-04-15T15:20:48.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.769+0000 7f01243cd640 1 -- 172.21.15.105:0/1835271282 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f011c0c7ff0 con 0x7f0104004800 2024-04-15T15:20:48.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.770+0000 7f01243cd640 1 -- 172.21.15.105:0/1835271282 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f011c0b38d0 con 0x7f0104004800 2024-04-15T15:20:48.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.771+0000 7f011affd640 1 -- 172.21.15.105:0/1835271282 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f011400d730 con 0x7f0104004800 2024-04-15T15:20:48.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.771+0000 7f011affd640 1 --2- 172.21.15.105:0/1835271282 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f010806c980 0x7f010806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:48.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.771+0000 7f011affd640 1 -- 172.21.15.105:0/1835271282 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f011408ba20 con 0x7f0104004800 2024-04-15T15:20:48.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.771+0000 7f0121968640 1 --2- 172.21.15.105:0/1835271282 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f010806c980 0x7f010806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:48.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.772+0000 7f0121968640 1 --2- 172.21.15.105:0/1835271282 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f010806c980 0x7f010806ee40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0110036000 tx=0x7f011002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:48.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.776+0000 7f011affd640 1 -- 172.21.15.105:0/1835271282 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f011c0b38d0 con 0x7f0104004800 2024-04-15T15:20:48.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.949+0000 7f01243cd640 1 -- 172.21.15.105:0/1835271282 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f011c068a40 con 0x7f0104004800 2024-04-15T15:20:48.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.950+0000 7f011affd640 1 -- 172.21.15.105:0/1835271282 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) ==== 104+0+11 (secure 0 0 0) 0x7f0114059e60 con 0x7f0104004800 2024-04-15T15:20:48.952 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-04-15T15:20:48.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.951+0000 7f01243cd640 1 -- 172.21.15.105:0/1835271282 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f010806c980 msgr2=0x7f010806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:48.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.951+0000 7f01243cd640 1 --2- 172.21.15.105:0/1835271282 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f010806c980 0x7f010806ee40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0110036000 tx=0x7f011002e000 comp rx=0 tx=0).stop 2024-04-15T15:20:48.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.951+0000 7f01243cd640 1 -- 172.21.15.105:0/1835271282 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0104004800 msgr2=0x7f011c0c4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:48.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.951+0000 7f01243cd640 1 --2- 172.21.15.105:0/1835271282 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0104004800 0x7f011c0c4f60 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f011400f820 tx=0x7f011400fce0 comp rx=0 tx=0).stop 2024-04-15T15:20:48.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.951+0000 7f01243cd640 1 -- 172.21.15.105:0/1835271282 shutdown_connections 2024-04-15T15:20:48.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.951+0000 7f01243cd640 1 --2- 172.21.15.105:0/1835271282 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f011c1f2660 0x7f011c157630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:48.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.951+0000 7f01243cd640 1 --2- 172.21.15.105:0/1835271282 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f010806c980 0x7f010806ee40 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:48.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.951+0000 7f01243cd640 1 --2- 172.21.15.105:0/1835271282 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f011c1f2ac0 0x7f011c157b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:48.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.951+0000 7f01243cd640 1 --2- 172.21.15.105:0/1835271282 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0104004800 0x7f011c0c4f60 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:48.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.951+0000 7f01243cd640 1 -- 172.21.15.105:0/1835271282 >> 172.21.15.105:0/1835271282 conn(0x7f011c0b2fc0 msgr2=0x7f011c0b12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:48.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.952+0000 7f01243cd640 1 -- 172.21.15.105:0/1835271282 shutdown_connections 2024-04-15T15:20:48.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:48.952+0000 7f01243cd640 1 -- 172.21.15.105:0/1835271282 wait complete. 2024-04-15T15:20:48.964 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-15T15:20:49.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.068+0000 7f80743b5640 1 Processor -- start 2024-04-15T15:20:49.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.068+0000 7f80743b5640 1 -- start start 2024-04-15T15:20:49.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.069+0000 7f80743b5640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f806c15e8d0 0x7f806c0d2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:49.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.069+0000 7f80743b5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f806c0ba480 0x7f806c0d1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:49.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.069+0000 7f80743b5640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f806c0d3400 con 0x7f806c0d1650 2024-04-15T15:20:49.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.069+0000 7f80743b5640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f806c0d1ac0 con 0x7f806c15e8d0 2024-04-15T15:20:49.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.069+0000 7f80743b5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f806c0d36e0 con 0x7f806c0ba480 2024-04-15T15:20:49.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.069+0000 7f8072151640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f806c15e8d0 0x7f806c0d2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:49.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.070+0000 7f8072952640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f806c0d1650 0x7f806c0b61d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41258/0 (socket says 172.21.15.105:41258) 2024-04-15T15:20:49.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.070+0000 7f8072952640 1 -- 172.21.15.105:0/1070502687 learned_addr learned my addr 172.21.15.105:0/1070502687 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:20:49.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.070+0000 7f8071950640 1 --2- 172.21.15.105:0/1070502687 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f806c0ba480 0x7f806c0d1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:49.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.070+0000 7f8072151640 1 -- 172.21.15.105:0/1070502687 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f806c0ba480 msgr2=0x7f806c0d1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:49.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.070+0000 7f8072151640 1 --2- 172.21.15.105:0/1070502687 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f806c0ba480 0x7f806c0d1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:49.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.070+0000 7f8072151640 1 -- 172.21.15.105:0/1070502687 >> v1:172.21.15.105:6789/0 conn(0x7f806c0d1650 legacy=0x7f806c0b61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:49.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.070+0000 7f8072151640 1 -- 172.21.15.105:0/1070502687 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f806c0d39c0 con 0x7f806c15e8d0 2024-04-15T15:20:49.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.070+0000 7f807114f640 1 -- 172.21.15.105:0/1070502687 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3334988087 0 0) 0x7f806c0d3400 con 0x7f806c0d1650 2024-04-15T15:20:49.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.070+0000 7f8071950640 1 --2- 172.21.15.105:0/1070502687 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f806c0ba480 0x7f806c0d1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:20:49.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.071+0000 7f8072151640 1 --2- 172.21.15.105:0/1070502687 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f806c15e8d0 0x7f806c0d2ab0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f805c0097a0 tx=0x7f805c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1ed91c920495683 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:49.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.071+0000 7f807114f640 1 -- 172.21.15.105:0/1070502687 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805c034070 con 0x7f806c15e8d0 2024-04-15T15:20:49.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.071+0000 7f807114f640 1 -- 172.21.15.105:0/1070502687 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f805c030610 con 0x7f806c15e8d0 2024-04-15T15:20:49.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.071+0000 7f807114f640 1 -- 172.21.15.105:0/1070502687 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805c0309e0 con 0x7f806c15e8d0 2024-04-15T15:20:49.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.071+0000 7f80743b5640 1 -- 172.21.15.105:0/1070502687 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f806c15e8d0 msgr2=0x7f806c0d2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:49.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.071+0000 7f80743b5640 1 --2- 172.21.15.105:0/1070502687 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f806c15e8d0 0x7f806c0d2ab0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f805c0097a0 tx=0x7f805c02b960 comp rx=0 tx=0).stop 2024-04-15T15:20:49.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.071+0000 7f80743b5640 1 -- 172.21.15.105:0/1070502687 shutdown_connections 2024-04-15T15:20:49.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.071+0000 7f80743b5640 1 --2- 172.21.15.105:0/1070502687 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f806c0ba480 0x7f806c0d1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:49.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.071+0000 7f80743b5640 1 --2- 172.21.15.105:0/1070502687 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f806c15e8d0 0x7f806c0d2ab0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:49.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.071+0000 7f80743b5640 1 -- 172.21.15.105:0/1070502687 >> 172.21.15.105:0/1070502687 conn(0x7f806c06d460 msgr2=0x7f806c067720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:49.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.071+0000 7f80743b5640 1 -- 172.21.15.105:0/1070502687 shutdown_connections 2024-04-15T15:20:49.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.071+0000 7f80743b5640 1 -- 172.21.15.105:0/1070502687 wait complete. 2024-04-15T15:20:49.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.072+0000 7f80743b5640 1 Processor -- start 2024-04-15T15:20:49.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.073+0000 7f80743b5640 1 -- start start 2024-04-15T15:20:49.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.073+0000 7f80743b5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f806c0ba480 0x7f806c151fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:49.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.073+0000 7f80743b5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f806c0d1650 0x7f806c0c9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:49.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.073+0000 7f80743b5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f806c15e8d0 0x7f806c0c9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:49.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.073+0000 7f80743b5640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f806c0d44a0 con 0x7f806c0d1650 2024-04-15T15:20:49.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.073+0000 7f80743b5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f806c0d3f80 con 0x7f806c15e8d0 2024-04-15T15:20:49.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.073+0000 7f80743b5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f806c0b8890 con 0x7f806c0ba480 2024-04-15T15:20:49.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.073+0000 7f8071950640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f806c0d1650 0x7f806c0c9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:49.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.073+0000 7f8071950640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f806c0d1650 0x7f806c0c9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59002/0 (socket says 172.21.15.105:59002) 2024-04-15T15:20:49.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.073+0000 7f8071950640 1 -- 172.21.15.105:0/1863203425 learned_addr learned my addr 172.21.15.105:0/1863203425 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:49.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.073+0000 7f8072151640 1 --2- 172.21.15.105:0/1863203425 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f806c0ba480 0x7f806c151fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:49.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.074+0000 7f8072952640 1 --2- 172.21.15.105:0/1863203425 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f806c15e8d0 0x7f806c0c9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:49.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.074+0000 7f8071950640 1 -- 172.21.15.105:0/1863203425 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f806c0ba480 msgr2=0x7f806c151fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:49.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.074+0000 7f8071950640 1 --2- 172.21.15.105:0/1863203425 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f806c0ba480 0x7f806c151fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:49.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.074+0000 7f8071950640 1 -- 172.21.15.105:0/1863203425 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f806c15e8d0 msgr2=0x7f806c0c9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:49.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.074+0000 7f8071950640 1 --2- 172.21.15.105:0/1863203425 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f806c15e8d0 0x7f806c0c9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:49.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.074+0000 7f8071950640 1 -- 172.21.15.105:0/1863203425 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f806c15f990 con 0x7f806c0d1650 2024-04-15T15:20:49.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.074+0000 7f8072151640 1 --2- 172.21.15.105:0/1863203425 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f806c0ba480 0x7f806c151fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:20:49.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.074+0000 7f8072952640 1 --2- 172.21.15.105:0/1863203425 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f806c15e8d0 0x7f806c0c9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:20:49.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.074+0000 7f8071950640 1 --2- 172.21.15.105:0/1863203425 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f806c0d1650 0x7f806c0c9960 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f806000e810 tx=0x7f806000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:49.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.074+0000 7f806affd640 1 -- 172.21.15.105:0/1863203425 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8060017070 con 0x7f806c0d1650 2024-04-15T15:20:49.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.074+0000 7f806affd640 1 -- 172.21.15.105:0/1863203425 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f806000c6d0 con 0x7f806c0d1650 2024-04-15T15:20:49.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.074+0000 7f80743b5640 1 -- 172.21.15.105:0/1863203425 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f806c1528d0 con 0x7f806c0d1650 2024-04-15T15:20:49.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.074+0000 7f806affd640 1 -- 172.21.15.105:0/1863203425 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f806000f040 con 0x7f806c0d1650 2024-04-15T15:20:49.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.074+0000 7f80743b5640 1 -- 172.21.15.105:0/1863203425 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f806c15fc70 con 0x7f806c0d1650 2024-04-15T15:20:49.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.075+0000 7f80743b5640 1 -- 172.21.15.105:0/1863203425 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f806c068ab0 con 0x7f806c0d1650 2024-04-15T15:20:49.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.075+0000 7f806affd640 1 -- 172.21.15.105:0/1863203425 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f806001a070 con 0x7f806c0d1650 2024-04-15T15:20:49.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.076+0000 7f806affd640 1 --2- 172.21.15.105:0/1863203425 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f805806c980 0x7f805806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:49.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.076+0000 7f8072151640 1 --2- 172.21.15.105:0/1863203425 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f805806c980 0x7f805806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:49.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.076+0000 7f806affd640 1 -- 172.21.15.105:0/1863203425 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f806008b880 con 0x7f806c0d1650 2024-04-15T15:20:49.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.076+0000 7f8072151640 1 --2- 172.21.15.105:0/1863203425 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f805806c980 0x7f805806ee40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f805c036000 tx=0x7f805c03b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:49.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.080+0000 7f806affd640 1 -- 172.21.15.105:0/1863203425 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8060059bb0 con 0x7f806c0d1650 2024-04-15T15:20:49.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.256+0000 7f80743b5640 1 -- 172.21.15.105:0/1863203425 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f806c15e010 con 0x7f806c0d1650 2024-04-15T15:20:49.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.256+0000 7f806affd640 1 -- 172.21.15.105:0/1863203425 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) ==== 100+0+11 (secure 0 0 0) 0x7f80600594b0 con 0x7f806c0d1650 2024-04-15T15:20:49.258 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-04-15T15:20:49.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.258+0000 7f80743b5640 1 -- 172.21.15.105:0/1863203425 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f805806c980 msgr2=0x7f805806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:49.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.258+0000 7f80743b5640 1 --2- 172.21.15.105:0/1863203425 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f805806c980 0x7f805806ee40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f805c036000 tx=0x7f805c03b040 comp rx=0 tx=0).stop 2024-04-15T15:20:49.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.258+0000 7f80743b5640 1 -- 172.21.15.105:0/1863203425 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f806c0d1650 msgr2=0x7f806c0c9960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:49.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.258+0000 7f80743b5640 1 --2- 172.21.15.105:0/1863203425 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f806c0d1650 0x7f806c0c9960 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f806000e810 tx=0x7f806000ecd0 comp rx=0 tx=0).stop 2024-04-15T15:20:49.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.258+0000 7f80743b5640 1 -- 172.21.15.105:0/1863203425 shutdown_connections 2024-04-15T15:20:49.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.258+0000 7f80743b5640 1 --2- 172.21.15.105:0/1863203425 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f805806c980 0x7f805806ee40 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:49.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.258+0000 7f80743b5640 1 --2- 172.21.15.105:0/1863203425 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f806c15e8d0 0x7f806c0c9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:49.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.258+0000 7f80743b5640 1 --2- 172.21.15.105:0/1863203425 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f806c0d1650 0x7f806c0c9960 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:49.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.258+0000 7f80743b5640 1 --2- 172.21.15.105:0/1863203425 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f806c0ba480 0x7f806c151fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:49.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.258+0000 7f80743b5640 1 -- 172.21.15.105:0/1863203425 >> 172.21.15.105:0/1863203425 conn(0x7f806c06d460 msgr2=0x7f806c0c8d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:49.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.258+0000 7f80743b5640 1 -- 172.21.15.105:0/1863203425 shutdown_connections 2024-04-15T15:20:49.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.258+0000 7f80743b5640 1 -- 172.21.15.105:0/1863203425 wait complete. 2024-04-15T15:20:49.270 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-15T15:20:49.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.372+0000 7f3629014640 1 Processor -- start 2024-04-15T15:20:49.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.372+0000 7f3629014640 1 -- start start 2024-04-15T15:20:49.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.373+0000 7f3629014640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f362415c7b0 0x7f36240d09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:49.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.373+0000 7f3629014640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f36240b2cb0 0x7f36240cd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:49.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.373+0000 7f3629014640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36240ccf40 con 0x7f36240d0f30 2024-04-15T15:20:49.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.373+0000 7f3629014640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f36240b32d0 con 0x7f362415c7b0 2024-04-15T15:20:49.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.373+0000 7f3629014640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f36240b35b0 con 0x7f36240b2cb0 2024-04-15T15:20:49.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.373+0000 7f3622d9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f362415c7b0 0x7f36240d09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:49.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.373+0000 7f362259c640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f36240d0f30 0x7f36240b25a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41272/0 (socket says 172.21.15.105:41272) 2024-04-15T15:20:49.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.373+0000 7f362259c640 1 -- 172.21.15.105:0/873325032 learned_addr learned my addr 172.21.15.105:0/873325032 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:20:49.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.373+0000 7f3622d9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f362415c7b0 0x7f36240d09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59012/0 (socket says 172.21.15.105:59012) 2024-04-15T15:20:49.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.374+0000 7f3622d9d640 1 -- 172.21.15.105:0/873325032 >> v2:172.21.15.154:3300/0 conn(0x7f36240b2cb0 msgr2=0x7f36240cd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:49.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.374+0000 7f3622d9d640 1 --2- 172.21.15.105:0/873325032 >> v2:172.21.15.154:3300/0 conn(0x7f36240b2cb0 0x7f36240cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:49.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.374+0000 7f3622d9d640 1 -- 172.21.15.105:0/873325032 >> v1:172.21.15.105:6789/0 conn(0x7f36240d0f30 legacy=0x7f36240b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:49.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.374+0000 7f3621d9b640 1 -- 172.21.15.105:0/873325032 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1869500133 0 0) 0x7f36240ccf40 con 0x7f36240d0f30 2024-04-15T15:20:49.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.374+0000 7f3622d9d640 1 -- 172.21.15.105:0/873325032 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36240b3890 con 0x7f362415c7b0 2024-04-15T15:20:49.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.374+0000 7f3622d9d640 1 --2- 172.21.15.105:0/873325032 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f362415c7b0 0x7f36240d09f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f36240c9600 tx=0x7f361402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d72959885ad9d9f server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:49.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.375+0000 7f3621d9b640 1 -- 172.21.15.105:0/873325032 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3614039070 con 0x7f362415c7b0 2024-04-15T15:20:49.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.375+0000 7f3621d9b640 1 -- 172.21.15.105:0/873325032 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f361402cc80 con 0x7f362415c7b0 2024-04-15T15:20:49.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.375+0000 7f3621d9b640 1 -- 172.21.15.105:0/873325032 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3614034660 con 0x7f362415c7b0 2024-04-15T15:20:49.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.375+0000 7f3629014640 1 -- 172.21.15.105:0/873325032 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f362415c7b0 msgr2=0x7f36240d09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:49.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.375+0000 7f3629014640 1 --2- 172.21.15.105:0/873325032 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f362415c7b0 0x7f36240d09f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f36240c9600 tx=0x7f361402bed0 comp rx=0 tx=0).stop 2024-04-15T15:20:49.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.375+0000 7f3629014640 1 -- 172.21.15.105:0/873325032 shutdown_connections 2024-04-15T15:20:49.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.375+0000 7f3629014640 1 --2- 172.21.15.105:0/873325032 >> v2:172.21.15.154:3300/0 conn(0x7f36240b2cb0 0x7f36240cd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:49.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.375+0000 7f3629014640 1 --2- 172.21.15.105:0/873325032 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f362415c7b0 0x7f36240d09f0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:49.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.375+0000 7f3629014640 1 -- 172.21.15.105:0/873325032 >> 172.21.15.105:0/873325032 conn(0x7f36240d25a0 msgr2=0x7f36240d29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:49.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.375+0000 7f3629014640 1 -- 172.21.15.105:0/873325032 shutdown_connections 2024-04-15T15:20:49.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.375+0000 7f3629014640 1 -- 172.21.15.105:0/873325032 wait complete. 2024-04-15T15:20:49.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.377+0000 7f3629014640 1 Processor -- start 2024-04-15T15:20:49.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.377+0000 7f3629014640 1 -- start start 2024-04-15T15:20:49.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.377+0000 7f3629014640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f36240b2cb0 0x7f36241497f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:49.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.377+0000 7f3629014640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36240d0f30 0x7f36240cb350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:49.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.377+0000 7f3629014640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f362415c7b0 0x7f36241f52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:49.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.377+0000 7f3629014640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f362415cd10 con 0x7f362415c7b0 2024-04-15T15:20:49.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.377+0000 7f3629014640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f362415cb90 con 0x7f36240d0f30 2024-04-15T15:20:49.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.377+0000 7f3629014640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f36240cd960 con 0x7f36240b2cb0 2024-04-15T15:20:49.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.377+0000 7f362359e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f362415c7b0 0x7f36241f52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:49.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.377+0000 7f362359e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f362415c7b0 0x7f36241f52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59016/0 (socket says 172.21.15.105:59016) 2024-04-15T15:20:49.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.378+0000 7f362359e640 1 -- 172.21.15.105:0/3450733911 learned_addr learned my addr 172.21.15.105:0/3450733911 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:49.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.378+0000 7f362259c640 1 --2- 172.21.15.105:0/3450733911 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36240d0f30 0x7f36240cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:49.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.378+0000 7f3622d9d640 1 --2- 172.21.15.105:0/3450733911 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f36240b2cb0 0x7f36241497f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:49.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.378+0000 7f362359e640 1 -- 172.21.15.105:0/3450733911 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f36240b2cb0 msgr2=0x7f36241497f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:49.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.378+0000 7f362359e640 1 --2- 172.21.15.105:0/3450733911 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f36240b2cb0 0x7f36241497f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:49.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.378+0000 7f362359e640 1 -- 172.21.15.105:0/3450733911 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36240d0f30 msgr2=0x7f36240cb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:49.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.378+0000 7f362359e640 1 --2- 172.21.15.105:0/3450733911 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36240d0f30 0x7f36240cb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:49.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.378+0000 7f362359e640 1 -- 172.21.15.105:0/3450733911 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36241f5830 con 0x7f362415c7b0 2024-04-15T15:20:49.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.378+0000 7f362259c640 1 --2- 172.21.15.105:0/3450733911 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36240d0f30 0x7f36240cb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:20:49.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.378+0000 7f362359e640 1 --2- 172.21.15.105:0/3450733911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f362415c7b0 0x7f36241f52f0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f361c009fd0 tx=0x7f361c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:49.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.378+0000 7f360b7fe640 1 -- 172.21.15.105:0/3450733911 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f361c011830 con 0x7f362415c7b0 2024-04-15T15:20:49.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.378+0000 7f360b7fe640 1 -- 172.21.15.105:0/3450733911 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f361c016070 con 0x7f362415c7b0 2024-04-15T15:20:49.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.378+0000 7f3629014640 1 -- 172.21.15.105:0/3450733911 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36240cbaf0 con 0x7f362415c7b0 2024-04-15T15:20:49.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.378+0000 7f360b7fe640 1 -- 172.21.15.105:0/3450733911 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f361c011bf0 con 0x7f362415c7b0 2024-04-15T15:20:49.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.378+0000 7f3629014640 1 -- 172.21.15.105:0/3450733911 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36240cc0b0 con 0x7f362415c7b0 2024-04-15T15:20:49.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.379+0000 7f3629014640 1 -- 172.21.15.105:0/3450733911 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3624069150 con 0x7f362415c7b0 2024-04-15T15:20:49.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.380+0000 7f360b7fe640 1 -- 172.21.15.105:0/3450733911 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f361c01b460 con 0x7f362415c7b0 2024-04-15T15:20:49.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.380+0000 7f360b7fe640 1 --2- 172.21.15.105:0/3450733911 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f361006c980 0x7f361006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:49.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.380+0000 7f360b7fe640 1 -- 172.21.15.105:0/3450733911 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f361c08b8c0 con 0x7f362415c7b0 2024-04-15T15:20:49.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.380+0000 7f3622d9d640 1 --2- 172.21.15.105:0/3450733911 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f361006c980 0x7f361006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:49.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.381+0000 7f3622d9d640 1 --2- 172.21.15.105:0/3450733911 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f361006c980 0x7f361006ee40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3614036000 tx=0x7f3614030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:49.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.384+0000 7f360b7fe640 1 -- 172.21.15.105:0/3450733911 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f361c059d00 con 0x7f362415c7b0 2024-04-15T15:20:49.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.570+0000 7f3629014640 1 -- 172.21.15.105:0/3450733911 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f36240b8520 con 0x7f362415c7b0 2024-04-15T15:20:49.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.570+0000 7f360b7fe640 1 -- 172.21.15.105:0/3450733911 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v34) ==== 80+0+0 (secure 0 0 0) 0x7f361c059600 con 0x7f362415c7b0 2024-04-15T15:20:49.572 INFO:teuthology.orchestra.run.smithi105.stderr:listed 0 entries 2024-04-15T15:20:49.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.572+0000 7f3629014640 1 -- 172.21.15.105:0/3450733911 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f361006c980 msgr2=0x7f361006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:49.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.572+0000 7f3629014640 1 --2- 172.21.15.105:0/3450733911 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f361006c980 0x7f361006ee40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3614036000 tx=0x7f3614030000 comp rx=0 tx=0).stop 2024-04-15T15:20:49.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.572+0000 7f3629014640 1 -- 172.21.15.105:0/3450733911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f362415c7b0 msgr2=0x7f36241f52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:49.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.572+0000 7f3629014640 1 --2- 172.21.15.105:0/3450733911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f362415c7b0 0x7f36241f52f0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f361c009fd0 tx=0x7f361c00b8e0 comp rx=0 tx=0).stop 2024-04-15T15:20:49.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.572+0000 7f3629014640 1 -- 172.21.15.105:0/3450733911 shutdown_connections 2024-04-15T15:20:49.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.572+0000 7f3629014640 1 --2- 172.21.15.105:0/3450733911 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f361006c980 0x7f361006ee40 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:49.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.572+0000 7f3629014640 1 --2- 172.21.15.105:0/3450733911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f362415c7b0 0x7f36241f52f0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:49.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.572+0000 7f3629014640 1 --2- 172.21.15.105:0/3450733911 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36240d0f30 0x7f36240cb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:49.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.572+0000 7f3629014640 1 --2- 172.21.15.105:0/3450733911 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f36240b2cb0 0x7f36241497f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:49.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.572+0000 7f3629014640 1 -- 172.21.15.105:0/3450733911 >> 172.21.15.105:0/3450733911 conn(0x7f36240d25a0 msgr2=0x7f3624070490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:49.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.572+0000 7f3629014640 1 -- 172.21.15.105:0/3450733911 shutdown_connections 2024-04-15T15:20:49.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:49.572+0000 7f3629014640 1 -- 172.21.15.105:0/3450733911 wait complete. 2024-04-15T15:20:49.584 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi154.front.sepia.ceph.com... 2024-04-15T15:20:49.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:49.585 DEBUG:teuthology.orchestra.run.smithi154:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-15T15:20:49.617 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi154.stderr:ceph-fuse[45121]: fuse finished with error 0 and tester_r 0 2024-04-15T15:20:49.619 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:20:55.417 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-15T15:20:55.420 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-15T15:20:55.721 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi154.front.sepia.ceph.com 2024-04-15T15:20:55.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:55.722 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-15T15:20:55.742 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-04-15T15:20:55.742 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:20:55.761 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056369,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1468 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248840k,nr_inodes=812210,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-15T15:20:55.761 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.2 is not mounted at ubuntu@smithi105.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.2 2024-04-15T15:20:55.762 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi105.front.sepia.ceph.com 2024-04-15T15:20:55.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:55.762 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-15T15:20:55.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-15T15:20:55.820 INFO:teuthology.orchestra.run.smithi105.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-15T15:20:55.821 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-15T15:20:55.821 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:20:55.840 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056372,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=679 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-15T15:20:55.840 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.3 is not mounted at ubuntu@smithi154.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.3 2024-04-15T15:20:55.840 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi154.front.sepia.ceph.com 2024-04-15T15:20:55.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:55.840 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-15T15:20:55.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-15T15:20:55.899 INFO:teuthology.orchestra.run.smithi154.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-15T15:20:55.899 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-15T15:20:56.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.012+0000 7f024b047640 1 Processor -- start 2024-04-15T15:20:56.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.012+0000 7f024b047640 1 -- start start 2024-04-15T15:20:56.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.013+0000 7f024b047640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f02440cfdf0 0x7f024406c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:56.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.013+0000 7f024b047640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f02440725b0 0x7f02440729f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:56.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.013+0000 7f024b047640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02440d0280 con 0x7f02440b0240 2024-04-15T15:20:56.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.013+0000 7f024b047640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f02440d0ae0 con 0x7f02440725b0 2024-04-15T15:20:56.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.013+0000 7f024b047640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f024415d380 con 0x7f02440cfdf0 2024-04-15T15:20:56.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.013+0000 7f02495e4640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f02440725b0 0x7f02440729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:56.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.013+0000 7f0248de3640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f02440b0240 0x7f024415d810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:45890/0 (socket says 172.21.15.105:45890) 2024-04-15T15:20:56.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.013+0000 7f0248de3640 1 -- 172.21.15.105:0/489688823 learned_addr learned my addr 172.21.15.105:0/489688823 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:20:56.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.013+0000 7f0243fff640 1 --2- 172.21.15.105:0/489688823 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f02440cfdf0 0x7f024406c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:56.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.013+0000 7f02495e4640 1 -- 172.21.15.105:0/489688823 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f02440cfdf0 msgr2=0x7f024406c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:56.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.014+0000 7f02495e4640 1 --2- 172.21.15.105:0/489688823 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f02440cfdf0 0x7f024406c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:56.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.014+0000 7f02495e4640 1 -- 172.21.15.105:0/489688823 >> v1:172.21.15.105:6789/0 conn(0x7f02440b0240 legacy=0x7f024415d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:56.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.014+0000 7f02495e4640 1 -- 172.21.15.105:0/489688823 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02440d0520 con 0x7f02440725b0 2024-04-15T15:20:56.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.014+0000 7f0243fff640 1 --2- 172.21.15.105:0/489688823 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f02440cfdf0 0x7f024406c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:56.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.014+0000 7f02495e4640 1 --2- 172.21.15.105:0/489688823 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02440725b0 0x7f02440729f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f024415d500 tx=0x7f023c02be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53b733d7f27de0e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:56.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.014+0000 7f02437fe640 1 -- 172.21.15.105:0/489688823 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f023c039070 con 0x7f02440725b0 2024-04-15T15:20:56.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.015+0000 7f02437fe640 1 -- 172.21.15.105:0/489688823 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f023c02cc40 con 0x7f02440725b0 2024-04-15T15:20:56.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.015+0000 7f02437fe640 1 -- 172.21.15.105:0/489688823 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f023c034440 con 0x7f02440725b0 2024-04-15T15:20:56.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.015+0000 7f024b047640 1 -- 172.21.15.105:0/489688823 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02440725b0 msgr2=0x7f02440729f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:56.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.015+0000 7f024b047640 1 --2- 172.21.15.105:0/489688823 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02440725b0 0x7f02440729f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f024415d500 tx=0x7f023c02be90 comp rx=0 tx=0).stop 2024-04-15T15:20:56.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.015+0000 7f024b047640 1 -- 172.21.15.105:0/489688823 shutdown_connections 2024-04-15T15:20:56.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.015+0000 7f024b047640 1 --2- 172.21.15.105:0/489688823 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02440725b0 0x7f02440729f0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:56.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.015+0000 7f024b047640 1 --2- 172.21.15.105:0/489688823 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f02440cfdf0 0x7f024406c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:56.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.015+0000 7f024b047640 1 -- 172.21.15.105:0/489688823 >> 172.21.15.105:0/489688823 conn(0x7f02440d0cd0 msgr2=0x7f02440d10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:56.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.015+0000 7f024b047640 1 -- 172.21.15.105:0/489688823 shutdown_connections 2024-04-15T15:20:56.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.015+0000 7f024b047640 1 -- 172.21.15.105:0/489688823 wait complete. 2024-04-15T15:20:56.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.016+0000 7f024b047640 1 Processor -- start 2024-04-15T15:20:56.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.016+0000 7f024b047640 1 -- start start 2024-04-15T15:20:56.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.017+0000 7f024b047640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f02440725b0 0x7f02441f5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:56.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.017+0000 7f024b047640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02440b0240 0x7f0244153220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:56.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.017+0000 7f024b047640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f02440cfdf0 0x7f0244153760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:56.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.017+0000 7f024b047640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f024415d010 con 0x7f02440b0240 2024-04-15T15:20:56.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.017+0000 7f024b047640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0244072f30 con 0x7f02440725b0 2024-04-15T15:20:56.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.017+0000 7f024b047640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f024406c770 con 0x7f02440cfdf0 2024-04-15T15:20:56.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.017+0000 7f0243fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02440b0240 0x7f0244153220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:56.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.017+0000 7f0243fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02440b0240 0x7f0244153220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35068/0 (socket says 172.21.15.105:35068) 2024-04-15T15:20:56.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.017+0000 7f0243fff640 1 -- 172.21.15.105:0/1168535552 learned_addr learned my addr 172.21.15.105:0/1168535552 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:56.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.018+0000 7f02495e4640 1 --2- 172.21.15.105:0/1168535552 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f02440cfdf0 0x7f0244153760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:56.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.018+0000 7f0248de3640 1 --2- 172.21.15.105:0/1168535552 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f02440725b0 0x7f02441f5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:56.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.018+0000 7f0243fff640 1 -- 172.21.15.105:0/1168535552 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f02440cfdf0 msgr2=0x7f0244153760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:56.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.018+0000 7f0243fff640 1 --2- 172.21.15.105:0/1168535552 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f02440cfdf0 0x7f0244153760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:56.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.018+0000 7f0243fff640 1 -- 172.21.15.105:0/1168535552 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f02440725b0 msgr2=0x7f02441f5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:56.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.018+0000 7f0243fff640 1 --2- 172.21.15.105:0/1168535552 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f02440725b0 0x7f02441f5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:56.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.018+0000 7f0243fff640 1 -- 172.21.15.105:0/1168535552 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0244153d90 con 0x7f02440b0240 2024-04-15T15:20:56.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.018+0000 7f0243fff640 1 --2- 172.21.15.105:0/1168535552 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02440b0240 0x7f0244153220 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f023000c7f0 tx=0x7f023000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:56.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.018+0000 7f02417fa640 1 -- 172.21.15.105:0/1168535552 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0230015070 con 0x7f02440b0240 2024-04-15T15:20:56.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.018+0000 7f02417fa640 1 -- 172.21.15.105:0/1168535552 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0230007820 con 0x7f02440b0240 2024-04-15T15:20:56.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.018+0000 7f024b047640 1 -- 172.21.15.105:0/1168535552 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02441540d0 con 0x7f02440b0240 2024-04-15T15:20:56.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.018+0000 7f02417fa640 1 -- 172.21.15.105:0/1168535552 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0230007bd0 con 0x7f02440b0240 2024-04-15T15:20:56.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.018+0000 7f024b047640 1 -- 172.21.15.105:0/1168535552 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02441e67d0 con 0x7f02440b0240 2024-04-15T15:20:56.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.019+0000 7f024b047640 1 -- 172.21.15.105:0/1168535552 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0244068a40 con 0x7f02440b0240 2024-04-15T15:20:56.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.020+0000 7f02417fa640 1 -- 172.21.15.105:0/1168535552 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f0230018070 con 0x7f02440b0240 2024-04-15T15:20:56.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.020+0000 7f02417fa640 1 --2- 172.21.15.105:0/1168535552 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f022806c980 0x7f022806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:56.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.020+0000 7f02417fa640 1 -- 172.21.15.105:0/1168535552 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f023008bd50 con 0x7f02440b0240 2024-04-15T15:20:56.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.020+0000 7f0248de3640 1 --2- 172.21.15.105:0/1168535552 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f022806c980 0x7f022806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:56.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.021+0000 7f0248de3640 1 --2- 172.21.15.105:0/1168535552 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f022806c980 0x7f022806ee40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0238000f60 tx=0x7f0238009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:56.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.025+0000 7f02417fa640 1 -- 172.21.15.105:0/1168535552 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f02441540d0 con 0x7f02440b0240 2024-04-15T15:20:56.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.240+0000 7f024b047640 1 -- 172.21.15.105:0/1168535552 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f02440b2750 con 0x7f02440b0240 2024-04-15T15:20:56.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.240+0000 7f02417fa640 1 -- 172.21.15.105:0/1168535552 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v14) ==== 53+0+73 (secure 0 0 0) 0x7f0230059a30 con 0x7f02440b0240 2024-04-15T15:20:56.243 INFO:teuthology.orchestra.run.smithi105.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-15T15:20:56.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.242+0000 7f024b047640 1 -- 172.21.15.105:0/1168535552 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f022806c980 msgr2=0x7f022806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:56.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.242+0000 7f024b047640 1 --2- 172.21.15.105:0/1168535552 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f022806c980 0x7f022806ee40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0238000f60 tx=0x7f0238009040 comp rx=0 tx=0).stop 2024-04-15T15:20:56.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.242+0000 7f024b047640 1 -- 172.21.15.105:0/1168535552 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02440b0240 msgr2=0x7f0244153220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:56.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.242+0000 7f024b047640 1 --2- 172.21.15.105:0/1168535552 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02440b0240 0x7f0244153220 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f023000c7f0 tx=0x7f023000ccb0 comp rx=0 tx=0).stop 2024-04-15T15:20:56.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.242+0000 7f024b047640 1 -- 172.21.15.105:0/1168535552 shutdown_connections 2024-04-15T15:20:56.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.242+0000 7f024b047640 1 --2- 172.21.15.105:0/1168535552 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f02440cfdf0 0x7f0244153760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:56.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.242+0000 7f024b047640 1 --2- 172.21.15.105:0/1168535552 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f022806c980 0x7f022806ee40 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:56.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.242+0000 7f024b047640 1 --2- 172.21.15.105:0/1168535552 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02440b0240 0x7f0244153220 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:56.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.242+0000 7f024b047640 1 --2- 172.21.15.105:0/1168535552 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f02440725b0 0x7f02441f5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:56.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.242+0000 7f024b047640 1 -- 172.21.15.105:0/1168535552 >> 172.21.15.105:0/1168535552 conn(0x7f02440d0cd0 msgr2=0x7f02441567b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:56.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.242+0000 7f024b047640 1 -- 172.21.15.105:0/1168535552 shutdown_connections 2024-04-15T15:20:56.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56.242+0000 7f024b047640 1 -- 172.21.15.105:0/1168535552 wait complete. 2024-04-15T15:20:56.255 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-15T15:20:56.255 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-15T15:20:56.256 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi105.front.sepia.ceph.com 2024-04-15T15:20:56.256 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-15T15:20:56.256 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-15T15:20:56.256 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-15T15:20:56.256 INFO:tasks.cephfs.mount:self.cephfs_mntpt = /volumes/_nogroup/subvol_quiesce/3b88d86e-6ee6-481f-8fa4-d97e9f88b65d 2024-04-15T15:20:56.256 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-15T15:20:56.256 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-15T15:20:56.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:56.257 DEBUG:teuthology.orchestra.run.smithi105:> ip addr 2024-04-15T15:20:56.274 INFO:teuthology.orchestra.run.smithi105.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-15T15:20:56.274 INFO:teuthology.orchestra.run.smithi105.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-15T15:20:56.274 INFO:teuthology.orchestra.run.smithi105.stdout: inet 127.0.0.1/8 scope host lo 2024-04-15T15:20:56.275 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:56.275 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 ::1/128 scope host 2024-04-15T15:20:56.275 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:56.275 INFO:teuthology.orchestra.run.smithi105.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:20:56.275 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b8 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:20:56.275 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f0 2024-04-15T15:20:56.275 INFO:teuthology.orchestra.run.smithi105.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:20:56.276 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b9 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:20:56.276 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f1 2024-04-15T15:20:56.276 INFO:teuthology.orchestra.run.smithi105.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:20:56.276 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:28 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:20:56.276 INFO:teuthology.orchestra.run.smithi105.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-15T15:20:56.276 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:29 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:20:56.276 INFO:teuthology.orchestra.run.smithi105.stdout: inet 172.21.15.105/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-15T15:20:56.277 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft 42631sec preferred_lft 42631sec 2024-04-15T15:20:56.277 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::ec4:7aff:fe8f:d429/64 scope link 2024-04-15T15:20:56.277 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:56.277 INFO:teuthology.orchestra.run.smithi105.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-15T15:20:56.277 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 12:bc:19:27:c6:bf brd ff:ff:ff:ff:ff:ff 2024-04-15T15:20:56.277 INFO:teuthology.orchestra.run.smithi105.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-15T15:20:56.277 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:56.277 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::7c5f:a9ff:fe78:3a4f/64 scope link 2024-04-15T15:20:56.277 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:56.277 INFO:teuthology.orchestra.run.smithi105.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-15T15:20:56.277 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 9e:e8:fb:17:06:9d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-15T15:20:56.277 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::9ce8:fbff:fe17:69d/64 scope link 2024-04-15T15:20:56.278 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:56.278 INFO:teuthology.orchestra.run.smithi105.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-15T15:20:56.278 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 12:bc:19:27:c6:bf brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-15T15:20:56.278 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::10bc:19ff:fe27:c6bf/64 scope link 2024-04-15T15:20:56.278 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:56.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:56.278 DEBUG:teuthology.orchestra.run.smithi105:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-15T15:20:56.341 INFO:teuthology.orchestra.run.smithi105.stdout:1 2024-04-15T15:20:56.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:56.343 DEBUG:teuthology.orchestra.run.smithi105:> route 2024-04-15T15:20:56.399 INFO:teuthology.orchestra.run.smithi105.stdout:Kernel IP routing table 2024-04-15T15:20:56.399 INFO:teuthology.orchestra.run.smithi105.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-15T15:20:56.399 INFO:teuthology.orchestra.run.smithi105.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-15T15:20:56.399 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-15T15:20:56.399 INFO:teuthology.orchestra.run.smithi105.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-15T15:20:56.399 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:20:56.399 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-04-15T15:20:56.399 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-15T15:20:56.400 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-15T15:20:56.400 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-15T15:20:56.400 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-04-15T15:20:56.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:20:56.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:20:56.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:56.537 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2024-04-15T15:20:56.552 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-15T15:20:56.552 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-15T15:20:56.553 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-15T15:20:56.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:56.553 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set brx.0 up 2024-04-15T15:20:56.620 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-15T15:20:56.620 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-15T15:20:56.620 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-15T15:20:56.678 INFO:teuthology.orchestra.run.smithi105.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-15T15:20:56.679 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-15T15:20:56.679 DEBUG:teuthology.orchestra.run.smithi105:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-15T15:20:56.734 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2024-04-15T15:20:56.802 DEBUG:teuthology.orchestra.run.smithi105:> cat /proc/self/mounts | awk '{print $2}' 2024-04-15T15:20:56.859 INFO:teuthology.orchestra.run.smithi105.stdout:/proc 2024-04-15T15:20:56.859 INFO:teuthology.orchestra.run.smithi105.stdout:/sys 2024-04-15T15:20:56.859 INFO:teuthology.orchestra.run.smithi105.stdout:/dev 2024-04-15T15:20:56.859 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/security 2024-04-15T15:20:56.859 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/shm 2024-04-15T15:20:56.859 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/pts 2024-04-15T15:20:56.859 INFO:teuthology.orchestra.run.smithi105.stdout:/run 2024-04-15T15:20:56.859 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/cgroup 2024-04-15T15:20:56.860 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/pstore 2024-04-15T15:20:56.860 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/bpf 2024-04-15T15:20:56.860 INFO:teuthology.orchestra.run.smithi105.stdout:/ 2024-04-15T15:20:56.860 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/selinux 2024-04-15T15:20:56.860 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/sys/fs/binfmt_misc 2024-04-15T15:20:56.860 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/mqueue 2024-04-15T15:20:56.860 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/hugepages 2024-04-15T15:20:56.860 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/debug 2024-04-15T15:20:56.860 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/tracing 2024-04-15T15:20:56.860 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/fuse/connections 2024-04-15T15:20:56.860 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/config 2024-04-15T15:20:56.860 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-sysctl.service 2024-04-15T15:20:56.860 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-15T15:20:56.860 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-15T15:20:56.861 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/nfs/rpc_pipefs 2024-04-15T15:20:56.861 INFO:teuthology.orchestra.run.smithi105.stdout:/run/user/1000 2024-04-15T15:20:56.861 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph 2024-04-15T15:20:56.861 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-0 2024-04-15T15:20:56.861 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-1 2024-04-15T15:20:56.861 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-2 2024-04-15T15:20:56.861 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-3 2024-04-15T15:20:56.861 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns 2024-04-15T15:20:56.861 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-15T15:20:56.861 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-15T15:20:56.861 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-15T15:20:56.861 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-15T15:20:56.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:56.862 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2024-04-15T15:20:56.915 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-15T15:20:56.915 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/volumes/_nogroup/subvol_quiesce/3b88d86e-6ee6-481f-8fa4-d97e9f88b65d) 2024-04-15T15:20:56.958 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2024-04-15T15:20:56.985 DEBUG:teuthology.orchestra.run.smithi105:> cat /proc/self/mounts | awk '{print $2}' 2024-04-15T15:20:57.041 INFO:teuthology.orchestra.run.smithi105.stdout:/proc 2024-04-15T15:20:57.042 INFO:teuthology.orchestra.run.smithi105.stdout:/sys 2024-04-15T15:20:57.042 INFO:teuthology.orchestra.run.smithi105.stdout:/dev 2024-04-15T15:20:57.042 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/security 2024-04-15T15:20:57.042 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/shm 2024-04-15T15:20:57.042 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/pts 2024-04-15T15:20:57.042 INFO:teuthology.orchestra.run.smithi105.stdout:/run 2024-04-15T15:20:57.042 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/cgroup 2024-04-15T15:20:57.042 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/pstore 2024-04-15T15:20:57.042 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/bpf 2024-04-15T15:20:57.043 INFO:teuthology.orchestra.run.smithi105.stdout:/ 2024-04-15T15:20:57.043 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/selinux 2024-04-15T15:20:57.043 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/sys/fs/binfmt_misc 2024-04-15T15:20:57.043 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/mqueue 2024-04-15T15:20:57.043 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/hugepages 2024-04-15T15:20:57.043 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/debug 2024-04-15T15:20:57.043 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/tracing 2024-04-15T15:20:57.043 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/fuse/connections 2024-04-15T15:20:57.043 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/config 2024-04-15T15:20:57.043 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-sysctl.service 2024-04-15T15:20:57.043 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-15T15:20:57.044 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-15T15:20:57.044 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/nfs/rpc_pipefs 2024-04-15T15:20:57.044 INFO:teuthology.orchestra.run.smithi105.stdout:/run/user/1000 2024-04-15T15:20:57.044 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph 2024-04-15T15:20:57.044 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-0 2024-04-15T15:20:57.044 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-1 2024-04-15T15:20:57.044 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-2 2024-04-15T15:20:57.044 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-3 2024-04-15T15:20:57.044 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns 2024-04-15T15:20:57.044 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-15T15:20:57.044 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-15T15:20:57.044 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-15T15:20:57.045 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-15T15:20:57.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:57.045 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2024-04-15T15:20:57.047 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi105.stderr:2024-04-15T15:20:57.045+0000 7f7a7c00b480 -1 init, newargv = 0x55d18b3307c0 newargc=15 2024-04-15T15:20:57.047 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi105.stderr:ceph-fuse[53766]: starting ceph client 2024-04-15T15:20:57.055 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi105.stderr:ceph-fuse[53766]: starting fuse 2024-04-15T15:20:57.063 INFO:teuthology.orchestra.run.smithi105.stdout:42 2024-04-15T15:20:57.063 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-15T15:20:57.063 DEBUG:teuthology.orchestra.run.smithi105:> sudo stdin-killer -- python3 -c ' 2024-04-15T15:20:57.063 DEBUG:teuthology.orchestra.run.smithi105:> import glob 2024-04-15T15:20:57.063 DEBUG:teuthology.orchestra.run.smithi105:> import re 2024-04-15T15:20:57.063 DEBUG:teuthology.orchestra.run.smithi105:> import os 2024-04-15T15:20:57.063 DEBUG:teuthology.orchestra.run.smithi105:> import subprocess 2024-04-15T15:20:57.063 DEBUG:teuthology.orchestra.run.smithi105:> 2024-04-15T15:20:57.064 DEBUG:teuthology.orchestra.run.smithi105:> def _find_admin_socket(client_name): 2024-04-15T15:20:57.064 DEBUG:teuthology.orchestra.run.smithi105:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-15T15:20:57.064 DEBUG:teuthology.orchestra.run.smithi105:> files = glob.glob(asok_path) 2024-04-15T15:20:57.064 DEBUG:teuthology.orchestra.run.smithi105:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-15T15:20:57.064 DEBUG:teuthology.orchestra.run.smithi105:> 2024-04-15T15:20:57.064 DEBUG:teuthology.orchestra.run.smithi105:> # Given a non-glob path, it better be there 2024-04-15T15:20:57.064 DEBUG:teuthology.orchestra.run.smithi105:> if "*" not in asok_path: 2024-04-15T15:20:57.064 DEBUG:teuthology.orchestra.run.smithi105:> assert(len(files) == 1) 2024-04-15T15:20:57.064 DEBUG:teuthology.orchestra.run.smithi105:> return files[0] 2024-04-15T15:20:57.064 DEBUG:teuthology.orchestra.run.smithi105:> 2024-04-15T15:20:57.064 DEBUG:teuthology.orchestra.run.smithi105:> for f in files: 2024-04-15T15:20:57.064 DEBUG:teuthology.orchestra.run.smithi105:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-15T15:20:57.064 DEBUG:teuthology.orchestra.run.smithi105:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-15T15:20:57.064 DEBUG:teuthology.orchestra.run.smithi105:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-15T15:20:57.064 DEBUG:teuthology.orchestra.run.smithi105:> contents = proc_f.read() 2024-04-15T15:20:57.064 DEBUG:teuthology.orchestra.run.smithi105:> if mountpoint in contents: 2024-04-15T15:20:57.064 DEBUG:teuthology.orchestra.run.smithi105:> return f 2024-04-15T15:20:57.065 DEBUG:teuthology.orchestra.run.smithi105:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-15T15:20:57.065 DEBUG:teuthology.orchestra.run.smithi105:> 2024-04-15T15:20:57.065 DEBUG:teuthology.orchestra.run.smithi105:> print(_find_admin_socket("client.0")) 2024-04-15T15:20:57.065 DEBUG:teuthology.orchestra.run.smithi105:> ' 2024-04-15T15:20:57.185 INFO:teuthology.orchestra.run.smithi105.stdout:/var/run/ceph/ceph-client.0.53766.asok 2024-04-15T15:20:57.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:20:57.196 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.53766.asok 2024-04-15T15:20:57.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:57.196 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.53766.asok status 2024-04-15T15:20:57.333 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-04-15T15:20:57.333 INFO:teuthology.orchestra.run.smithi105.stdout: "metadata": { 2024-04-15T15:20:57.333 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_sha1": "9bf1c4ba967dd24077be6e5b87edda4ca87a6e14", 2024-04-15T15:20:57.333 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_version": "ceph version 19.0.0-2861-g9bf1c4ba (9bf1c4ba967dd24077be6e5b87edda4ca87a6e14) squid (dev)", 2024-04-15T15:20:57.333 INFO:teuthology.orchestra.run.smithi105.stdout: "entity_id": "0", 2024-04-15T15:20:57.333 INFO:teuthology.orchestra.run.smithi105.stdout: "hostname": "smithi105", 2024-04-15T15:20:57.333 INFO:teuthology.orchestra.run.smithi105.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-15T15:20:57.334 INFO:teuthology.orchestra.run.smithi105.stdout: "pid": "53766", 2024-04-15T15:20:57.334 INFO:teuthology.orchestra.run.smithi105.stdout: "root": "/volumes/_nogroup/subvol_quiesce/3b88d86e-6ee6-481f-8fa4-d97e9f88b65d" 2024-04-15T15:20:57.334 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-04-15T15:20:57.334 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_count": 0, 2024-04-15T15:20:57.334 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_pinned_count": 0, 2024-04-15T15:20:57.334 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 5619, 2024-04-15T15:20:57.334 INFO:teuthology.orchestra.run.smithi105.stdout: "inst": { 2024-04-15T15:20:57.334 INFO:teuthology.orchestra.run.smithi105.stdout: "name": { 2024-04-15T15:20:57.334 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "client", 2024-04-15T15:20:57.334 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 5619 2024-04-15T15:20:57.334 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-04-15T15:20:57.334 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": { 2024-04-15T15:20:57.335 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-04-15T15:20:57.335 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "192.168.0.1:0", 2024-04-15T15:20:57.335 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 382498032 2024-04-15T15:20:57.335 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-04-15T15:20:57.335 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-04-15T15:20:57.335 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": { 2024-04-15T15:20:57.335 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-04-15T15:20:57.335 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "192.168.0.1:0", 2024-04-15T15:20:57.335 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 382498032 2024-04-15T15:20:57.335 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-04-15T15:20:57.335 INFO:teuthology.orchestra.run.smithi105.stdout: "inst_str": "client.5619 192.168.0.1:0/382498032", 2024-04-15T15:20:57.335 INFO:teuthology.orchestra.run.smithi105.stdout: "addr_str": "192.168.0.1:0/382498032", 2024-04-15T15:20:57.335 INFO:teuthology.orchestra.run.smithi105.stdout: "inode_count": 5, 2024-04-15T15:20:57.335 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_epoch": 14, 2024-04-15T15:20:57.335 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_epoch": 34, 2024-04-15T15:20:57.336 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_epoch_barrier": 0, 2024-04-15T15:20:57.336 INFO:teuthology.orchestra.run.smithi105.stdout: "blocklisted": false, 2024-04-15T15:20:57.336 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs" 2024-04-15T15:20:57.336 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-04-15T15:20:57.341 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-15T15:20:57.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.444+0000 7fe3faac5640 1 Processor -- start 2024-04-15T15:20:57.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.444+0000 7fe3faac5640 1 -- start start 2024-04-15T15:20:57.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.445+0000 7fe3faac5640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe3f40b78f0 0x7fe3f40d11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:57.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.445+0000 7fe3faac5640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3f40d1ae0 con 0x7fe3f40d16e0 2024-04-15T15:20:57.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.445+0000 7fe3faac5640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3f4162ff0 con 0x7fe3f415eab0 2024-04-15T15:20:57.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.445+0000 7fe3faac5640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fe3f415ee90 con 0x7fe3f40b78f0 2024-04-15T15:20:57.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.445+0000 7fe3f9062640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fe3f40d16e0 0x7fe3f40d3530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:45906/0 (socket says 172.21.15.105:45906) 2024-04-15T15:20:57.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.445+0000 7fe3f9062640 1 -- 172.21.15.105:0/1633319927 learned_addr learned my addr 172.21.15.105:0/1633319927 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:20:57.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.445+0000 7fe3f3fff640 1 --2- 172.21.15.105:0/1633319927 >> v2:172.21.15.154:3300/0 conn(0x7fe3f40b78f0 0x7fe3f40d11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:57.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.445+0000 7fe3f37fe640 1 -- 172.21.15.105:0/1633319927 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2740444015 0 0) 0x7fe3f40d1ae0 con 0x7fe3f40d16e0 2024-04-15T15:20:57.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.445+0000 7fe3f37fe640 1 -- 172.21.15.105:0/1633319927 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe3e0003520 con 0x7fe3f40d16e0 2024-04-15T15:20:57.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.445+0000 7fe3f37fe640 1 -- 172.21.15.105:0/1633319927 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2555257054 0 0) 0x7fe3e0003520 con 0x7fe3f40d16e0 2024-04-15T15:20:57.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.445+0000 7fe3f37fe640 1 -- 172.21.15.105:0/1633319927 >> v2:172.21.15.154:3300/0 conn(0x7fe3f40b78f0 msgr2=0x7fe3f40d11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:57.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.445+0000 7fe3f37fe640 1 --2- 172.21.15.105:0/1633319927 >> v2:172.21.15.154:3300/0 conn(0x7fe3f40b78f0 0x7fe3f40d11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:57.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.445+0000 7fe3f37fe640 1 -- 172.21.15.105:0/1633319927 >> v1:172.21.15.154:6789/0 conn(0x7fe3f415eab0 legacy=0x7fe3f4162530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:57.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.445+0000 7fe3f37fe640 1 -- 172.21.15.105:0/1633319927 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3f41632d0 con 0x7fe3f40d16e0 2024-04-15T15:20:57.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.445+0000 7fe3f37fe640 1 -- 172.21.15.105:0/1633319927 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7fe3dc001670 con 0x7fe3f40d16e0 2024-04-15T15:20:57.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.445+0000 7fe3f3fff640 1 --2- 172.21.15.105:0/1633319927 >> v2:172.21.15.154:3300/0 conn(0x7fe3f40b78f0 0x7fe3f40d11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:20:57.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.445+0000 7fe3f37fe640 1 -- 172.21.15.105:0/1633319927 >> v1:172.21.15.105:6789/0 conn(0x7fe3f40d16e0 legacy=0x7fe3f40d3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:57.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.445+0000 7fe3f37fe640 1 --2- 172.21.15.105:0/1633319927 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3e00039b0 0x7fe3e0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:57.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.445+0000 7fe3f37fe640 1 --2- 172.21.15.105:0/1633319927 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3e0004800 0x7fe3e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:57.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.445+0000 7fe3f37fe640 1 --2- 172.21.15.105:0/1633319927 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3e00051c0 0x7fe3e000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:57.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.445+0000 7fe3f37fe640 1 -- 172.21.15.105:0/1633319927 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f415ee90 con 0x7fe3e00039b0 2024-04-15T15:20:57.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.445+0000 7fe3f37fe640 1 -- 172.21.15.105:0/1633319927 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e000c060 con 0x7fe3e0004800 2024-04-15T15:20:57.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.445+0000 7fe3f37fe640 1 -- 172.21.15.105:0/1633319927 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fe3e000c260 con 0x7fe3e00051c0 2024-04-15T15:20:57.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.445+0000 7fe3f8861640 1 --2- 172.21.15.105:0/1633319927 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3e00039b0 0x7fe3e0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:57.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.445+0000 7fe3f3fff640 1 --2- 172.21.15.105:0/1633319927 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3e0004800 0x7fe3e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:57.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.446+0000 7fe3f9062640 1 --2- 172.21.15.105:0/1633319927 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3e00051c0 0x7fe3e000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:57.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.446+0000 7fe3f8861640 1 -- 172.21.15.105:0/1633319927 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3e00051c0 msgr2=0x7fe3e000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:57.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.446+0000 7fe3f8861640 1 --2- 172.21.15.105:0/1633319927 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3e00051c0 0x7fe3e000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:57.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.446+0000 7fe3f8861640 1 -- 172.21.15.105:0/1633319927 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3e0004800 msgr2=0x7fe3e0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:57.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.446+0000 7fe3f8861640 1 --2- 172.21.15.105:0/1633319927 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3e0004800 0x7fe3e0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:57.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.446+0000 7fe3f8861640 1 -- 172.21.15.105:0/1633319927 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3e0003520 con 0x7fe3e00039b0 2024-04-15T15:20:57.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.446+0000 7fe3f9062640 1 -- 172.21.15.105:0/1633319927 reap_dead start 2024-04-15T15:20:57.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.446+0000 7fe3f8861640 1 --2- 172.21.15.105:0/1633319927 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3e00039b0 0x7fe3e0003dd0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec001750 tx=0x7fe3e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=876448ccf44048ae server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:57.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.447+0000 7fe3f37fe640 1 -- 172.21.15.105:0/1633319927 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec001370 con 0x7fe3e00039b0 2024-04-15T15:20:57.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.447+0000 7fe3f37fe640 1 -- 172.21.15.105:0/1633319927 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3ec0052d0 con 0x7fe3e00039b0 2024-04-15T15:20:57.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.447+0000 7fe3f37fe640 1 -- 172.21.15.105:0/1633319927 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec003830 con 0x7fe3e00039b0 2024-04-15T15:20:57.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.447+0000 7fe3faac5640 1 -- 172.21.15.105:0/1633319927 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3e00039b0 msgr2=0x7fe3e0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:57.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.447+0000 7fe3faac5640 1 --2- 172.21.15.105:0/1633319927 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3e00039b0 0x7fe3e0003dd0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec001750 tx=0x7fe3e0005830 comp rx=0 tx=0).stop 2024-04-15T15:20:57.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.447+0000 7fe3faac5640 1 -- 172.21.15.105:0/1633319927 shutdown_connections 2024-04-15T15:20:57.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.447+0000 7fe3faac5640 1 --2- 172.21.15.105:0/1633319927 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3e00039b0 0x7fe3e0003dd0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:57.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.447+0000 7fe3faac5640 1 -- 172.21.15.105:0/1633319927 >> 172.21.15.105:0/1633319927 conn(0x7fe3f4067720 msgr2=0x7fe3f4067b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:57.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.447+0000 7fe3faac5640 1 -- 172.21.15.105:0/1633319927 shutdown_connections 2024-04-15T15:20:57.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.447+0000 7fe3faac5640 1 -- 172.21.15.105:0/1633319927 wait complete. 2024-04-15T15:20:57.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.448+0000 7fe3faac5640 1 Processor -- start 2024-04-15T15:20:57.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.448+0000 7fe3faac5640 1 -- start start 2024-04-15T15:20:57.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.449+0000 7fe3faac5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3e00039b0 0x7fe3f40ca390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:57.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.449+0000 7fe3faac5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3f415ccc0 0x7fe3f4149c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:57.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.449+0000 7fe3faac5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3f40d4f00 0x7fe3f40cda60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:57.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.449+0000 7fe3faac5640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f40ba610 con 0x7fe3f415ccc0 2024-04-15T15:20:57.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.449+0000 7fe3faac5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f40d3010 con 0x7fe3e00039b0 2024-04-15T15:20:57.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.449+0000 7fe3faac5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fe3f40ba790 con 0x7fe3f40d4f00 2024-04-15T15:20:57.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.449+0000 7fe3f3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3f415ccc0 0x7fe3f4149c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:57.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.449+0000 7fe3f3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3f415ccc0 0x7fe3f4149c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35086/0 (socket says 172.21.15.105:35086) 2024-04-15T15:20:57.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.449+0000 7fe3f3fff640 1 -- 172.21.15.105:0/380518016 learned_addr learned my addr 172.21.15.105:0/380518016 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:57.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.450+0000 7fe3f3fff640 1 -- 172.21.15.105:0/380518016 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3f40d4f00 msgr2=0x7fe3f40cda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:57.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.450+0000 7fe3f8861640 1 --2- 172.21.15.105:0/380518016 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3e00039b0 0x7fe3f40ca390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:57.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.450+0000 7fe3f9062640 1 --2- 172.21.15.105:0/380518016 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3f40d4f00 0x7fe3f40cda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:57.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.450+0000 7fe3f3fff640 1 --2- 172.21.15.105:0/380518016 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3f40d4f00 0x7fe3f40cda60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:57.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.450+0000 7fe3f3fff640 1 -- 172.21.15.105:0/380518016 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3e00039b0 msgr2=0x7fe3f40ca390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:57.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.450+0000 7fe3f3fff640 1 --2- 172.21.15.105:0/380518016 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3e00039b0 0x7fe3f40ca390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:57.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.450+0000 7fe3f3fff640 1 -- 172.21.15.105:0/380518016 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3f40c7f70 con 0x7fe3f415ccc0 2024-04-15T15:20:57.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.450+0000 7fe3f3fff640 1 --2- 172.21.15.105:0/380518016 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3f415ccc0 0x7fe3f4149c20 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fe3e400f490 tx=0x7fe3e400f950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:57.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.450+0000 7fe3f17fa640 1 -- 172.21.15.105:0/380518016 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3e4003b90 con 0x7fe3f415ccc0 2024-04-15T15:20:57.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.450+0000 7fe3f17fa640 1 -- 172.21.15.105:0/380518016 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3e4003d50 con 0x7fe3f415ccc0 2024-04-15T15:20:57.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.450+0000 7fe3faac5640 1 -- 172.21.15.105:0/380518016 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3f40c7c90 con 0x7fe3f415ccc0 2024-04-15T15:20:57.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.450+0000 7fe3f17fa640 1 -- 172.21.15.105:0/380518016 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3e4015880 con 0x7fe3f415ccc0 2024-04-15T15:20:57.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.450+0000 7fe3faac5640 1 -- 172.21.15.105:0/380518016 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3f40c9e70 con 0x7fe3f415ccc0 2024-04-15T15:20:57.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.451+0000 7fe3faac5640 1 -- 172.21.15.105:0/380518016 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3f415d550 con 0x7fe3f415ccc0 2024-04-15T15:20:57.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.451+0000 7fe3f17fa640 1 -- 172.21.15.105:0/380518016 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fe3e4015a20 con 0x7fe3f415ccc0 2024-04-15T15:20:57.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.452+0000 7fe3f17fa640 1 --2- 172.21.15.105:0/380518016 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fe3d406c980 0x7fe3d406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:57.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.452+0000 7fe3f17fa640 1 -- 172.21.15.105:0/380518016 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fe3e408cd40 con 0x7fe3f415ccc0 2024-04-15T15:20:57.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.452+0000 7fe3f8861640 1 --2- 172.21.15.105:0/380518016 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fe3d406c980 0x7fe3d406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:57.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.452+0000 7fe3f8861640 1 --2- 172.21.15.105:0/380518016 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fe3d406c980 0x7fe3d406ee40 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec033000 tx=0x7fe3ec009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:57.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.456+0000 7fe3f17fa640 1 -- 172.21.15.105:0/380518016 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe3f415d550 con 0x7fe3f415ccc0 2024-04-15T15:20:57.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.670+0000 7fe3faac5640 1 -- 172.21.15.105:0/380518016 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe3f40c9c80 con 0x7fe3f415ccc0 2024-04-15T15:20:57.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.670+0000 7fe3f17fa640 1 -- 172.21.15.105:0/380518016 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v14) ==== 53+0+73 (secure 0 0 0) 0x7fe3e4019070 con 0x7fe3f415ccc0 2024-04-15T15:20:57.672 INFO:teuthology.orchestra.run.smithi105.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-15T15:20:57.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.672+0000 7fe3faac5640 1 -- 172.21.15.105:0/380518016 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fe3d406c980 msgr2=0x7fe3d406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:57.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.672+0000 7fe3faac5640 1 --2- 172.21.15.105:0/380518016 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fe3d406c980 0x7fe3d406ee40 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec033000 tx=0x7fe3ec009030 comp rx=0 tx=0).stop 2024-04-15T15:20:57.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.672+0000 7fe3faac5640 1 -- 172.21.15.105:0/380518016 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3f415ccc0 msgr2=0x7fe3f4149c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:57.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.672+0000 7fe3faac5640 1 --2- 172.21.15.105:0/380518016 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3f415ccc0 0x7fe3f4149c20 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fe3e400f490 tx=0x7fe3e400f950 comp rx=0 tx=0).stop 2024-04-15T15:20:57.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.672+0000 7fe3faac5640 1 -- 172.21.15.105:0/380518016 shutdown_connections 2024-04-15T15:20:57.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.672+0000 7fe3faac5640 1 --2- 172.21.15.105:0/380518016 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3f40d4f00 0x7fe3f40cda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:57.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.672+0000 7fe3faac5640 1 --2- 172.21.15.105:0/380518016 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fe3d406c980 0x7fe3d406ee40 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:57.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.672+0000 7fe3faac5640 1 --2- 172.21.15.105:0/380518016 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3f415ccc0 0x7fe3f4149c20 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:57.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.672+0000 7fe3faac5640 1 --2- 172.21.15.105:0/380518016 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3e00039b0 0x7fe3f40ca390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:57.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.672+0000 7fe3faac5640 1 -- 172.21.15.105:0/380518016 >> 172.21.15.105:0/380518016 conn(0x7fe3f4067720 msgr2=0x7fe3f414a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:57.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.673+0000 7fe3faac5640 1 -- 172.21.15.105:0/380518016 shutdown_connections 2024-04-15T15:20:57.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:57.673+0000 7fe3faac5640 1 -- 172.21.15.105:0/380518016 wait complete. 2024-04-15T15:20:57.685 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-15T15:20:57.685 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-15T15:20:57.685 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi154.front.sepia.ceph.com 2024-04-15T15:20:57.685 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-15T15:20:57.686 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-15T15:20:57.686 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-15T15:20:57.686 INFO:tasks.cephfs.mount:self.cephfs_mntpt = /volumes/_nogroup/subvol_quiesce/3b88d86e-6ee6-481f-8fa4-d97e9f88b65d 2024-04-15T15:20:57.686 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-15T15:20:57.686 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-15T15:20:57.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:57.686 DEBUG:teuthology.orchestra.run.smithi154:> ip addr 2024-04-15T15:20:57.706 INFO:teuthology.orchestra.run.smithi154.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-15T15:20:57.706 INFO:teuthology.orchestra.run.smithi154.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-15T15:20:57.706 INFO:teuthology.orchestra.run.smithi154.stdout: inet 127.0.0.1/8 scope host lo 2024-04-15T15:20:57.706 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:57.706 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 ::1/128 scope host 2024-04-15T15:20:57.706 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:57.707 INFO:teuthology.orchestra.run.smithi154.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:20:57.707 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:54 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:20:57.707 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f0 2024-04-15T15:20:57.707 INFO:teuthology.orchestra.run.smithi154.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:20:57.707 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:55 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:20:57.707 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f1 2024-04-15T15:20:57.707 INFO:teuthology.orchestra.run.smithi154.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:20:57.707 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:60 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:20:57.707 INFO:teuthology.orchestra.run.smithi154.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-15T15:20:57.707 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:61 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:20:57.707 INFO:teuthology.orchestra.run.smithi154.stdout: inet 172.21.15.154/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-15T15:20:57.707 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft 42618sec preferred_lft 42618sec 2024-04-15T15:20:57.707 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::ec4:7aff:fe88:7761/64 scope link 2024-04-15T15:20:57.707 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:57.708 INFO:teuthology.orchestra.run.smithi154.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-15T15:20:57.708 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether a2:e6:6b:6d:26:a4 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:20:57.708 INFO:teuthology.orchestra.run.smithi154.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-15T15:20:57.708 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:57.708 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::1cc6:e4ff:fe35:1850/64 scope link 2024-04-15T15:20:57.708 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:57.708 INFO:teuthology.orchestra.run.smithi154.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-15T15:20:57.708 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether ce:c3:4d:70:5f:15 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-15T15:20:57.708 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::ccc3:4dff:fe70:5f15/64 scope link 2024-04-15T15:20:57.708 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:57.708 INFO:teuthology.orchestra.run.smithi154.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-15T15:20:57.708 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether a2:e6:6b:6d:26:a4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-15T15:20:57.708 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::a0e6:6bff:fe6d:26a4/64 scope link 2024-04-15T15:20:57.708 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:57.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:57.709 DEBUG:teuthology.orchestra.run.smithi154:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-15T15:20:57.778 INFO:teuthology.orchestra.run.smithi154.stdout:1 2024-04-15T15:20:57.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:57.779 DEBUG:teuthology.orchestra.run.smithi154:> route 2024-04-15T15:20:57.837 INFO:teuthology.orchestra.run.smithi154.stdout:Kernel IP routing table 2024-04-15T15:20:57.837 INFO:teuthology.orchestra.run.smithi154.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-15T15:20:57.837 INFO:teuthology.orchestra.run.smithi154.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-15T15:20:57.837 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-15T15:20:57.837 INFO:teuthology.orchestra.run.smithi154.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-15T15:20:57.838 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:20:57.838 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2024-04-15T15:20:57.838 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-15T15:20:57.838 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-15T15:20:57.838 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-15T15:20:57.838 DEBUG:teuthology.orchestra.run.smithi154:> ') 2024-04-15T15:20:57.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:20:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:20:57.975 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:20:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:20:57.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:57.981 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list 2024-04-15T15:20:57.997 INFO:teuthology.orchestra.run.smithi154.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-15T15:20:57.997 INFO:teuthology.orchestra.run.smithi154.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-15T15:20:57.997 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-15T15:20:57.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:57.998 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link set brx.0 up 2024-04-15T15:20:58.066 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-15T15:20:58.066 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-15T15:20:58.066 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-15T15:20:58.122 INFO:teuthology.orchestra.run.smithi154.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-15T15:20:58.122 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-15T15:20:58.122 DEBUG:teuthology.orchestra.run.smithi154:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-15T15:20:58.178 DEBUG:teuthology.orchestra.run.smithi154:> sudo modprobe fuse 2024-04-15T15:20:58.247 DEBUG:teuthology.orchestra.run.smithi154:> cat /proc/self/mounts | awk '{print $2}' 2024-04-15T15:20:58.304 INFO:teuthology.orchestra.run.smithi154.stdout:/proc 2024-04-15T15:20:58.305 INFO:teuthology.orchestra.run.smithi154.stdout:/sys 2024-04-15T15:20:58.305 INFO:teuthology.orchestra.run.smithi154.stdout:/dev 2024-04-15T15:20:58.305 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/security 2024-04-15T15:20:58.305 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/shm 2024-04-15T15:20:58.305 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/pts 2024-04-15T15:20:58.306 INFO:teuthology.orchestra.run.smithi154.stdout:/run 2024-04-15T15:20:58.306 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/cgroup 2024-04-15T15:20:58.306 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/pstore 2024-04-15T15:20:58.306 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/bpf 2024-04-15T15:20:58.306 INFO:teuthology.orchestra.run.smithi154.stdout:/ 2024-04-15T15:20:58.306 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/selinux 2024-04-15T15:20:58.306 INFO:teuthology.orchestra.run.smithi154.stdout:/proc/sys/fs/binfmt_misc 2024-04-15T15:20:58.306 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/mqueue 2024-04-15T15:20:58.306 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/hugepages 2024-04-15T15:20:58.306 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/debug 2024-04-15T15:20:58.306 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/tracing 2024-04-15T15:20:58.306 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-15T15:20:58.307 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/fuse/connections 2024-04-15T15:20:58.307 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-sysctl.service 2024-04-15T15:20:58.307 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/config 2024-04-15T15:20:58.307 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-15T15:20:58.307 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/nfs/rpc_pipefs 2024-04-15T15:20:58.307 INFO:teuthology.orchestra.run.smithi154.stdout:/run/user/1000 2024-04-15T15:20:58.307 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph 2024-04-15T15:20:58.307 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-4 2024-04-15T15:20:58.307 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-5 2024-04-15T15:20:58.307 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-6 2024-04-15T15:20:58.307 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-7 2024-04-15T15:20:58.307 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns 2024-04-15T15:20:58.307 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-15T15:20:58.307 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-15T15:20:58.307 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-15T15:20:58.307 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-15T15:20:58.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:58.308 DEBUG:teuthology.orchestra.run.smithi154:> ls /sys/fs/fuse/connections 2024-04-15T15:20:58.360 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-15T15:20:58.360 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1 --client_mountpoint=/volumes/_nogroup/subvol_quiesce/3b88d86e-6ee6-481f-8fa4-d97e9f88b65d) 2024-04-15T15:20:58.403 DEBUG:teuthology.orchestra.run.smithi154:> sudo modprobe fuse 2024-04-15T15:20:58.431 DEBUG:teuthology.orchestra.run.smithi154:> cat /proc/self/mounts | awk '{print $2}' 2024-04-15T15:20:58.488 INFO:teuthology.orchestra.run.smithi154.stdout:/proc 2024-04-15T15:20:58.488 INFO:teuthology.orchestra.run.smithi154.stdout:/sys 2024-04-15T15:20:58.489 INFO:teuthology.orchestra.run.smithi154.stdout:/dev 2024-04-15T15:20:58.489 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/security 2024-04-15T15:20:58.489 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/shm 2024-04-15T15:20:58.489 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/pts 2024-04-15T15:20:58.489 INFO:teuthology.orchestra.run.smithi154.stdout:/run 2024-04-15T15:20:58.489 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/cgroup 2024-04-15T15:20:58.489 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/pstore 2024-04-15T15:20:58.489 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/bpf 2024-04-15T15:20:58.489 INFO:teuthology.orchestra.run.smithi154.stdout:/ 2024-04-15T15:20:58.489 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/selinux 2024-04-15T15:20:58.489 INFO:teuthology.orchestra.run.smithi154.stdout:/proc/sys/fs/binfmt_misc 2024-04-15T15:20:58.489 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/mqueue 2024-04-15T15:20:58.489 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/hugepages 2024-04-15T15:20:58.489 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/debug 2024-04-15T15:20:58.490 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/tracing 2024-04-15T15:20:58.490 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-15T15:20:58.490 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/fuse/connections 2024-04-15T15:20:58.490 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-sysctl.service 2024-04-15T15:20:58.490 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/config 2024-04-15T15:20:58.490 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-15T15:20:58.490 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/nfs/rpc_pipefs 2024-04-15T15:20:58.490 INFO:teuthology.orchestra.run.smithi154.stdout:/run/user/1000 2024-04-15T15:20:58.490 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph 2024-04-15T15:20:58.490 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-4 2024-04-15T15:20:58.490 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-5 2024-04-15T15:20:58.490 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-6 2024-04-15T15:20:58.490 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-7 2024-04-15T15:20:58.490 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns 2024-04-15T15:20:58.491 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-15T15:20:58.491 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-15T15:20:58.491 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-15T15:20:58.491 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-15T15:20:58.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:58.491 DEBUG:teuthology.orchestra.run.smithi154:> ls /sys/fs/fuse/connections 2024-04-15T15:20:58.493 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi154.stderr:2024-04-15T15:20:58.491+0000 7f1fb68bb480 -1 init, newargv = 0x5648003a4140 newargc=15 2024-04-15T15:20:58.493 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi154.stderr:ceph-fuse[45615]: starting ceph client 2024-04-15T15:20:58.499 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi154.stderr:ceph-fuse[45615]: starting fuse 2024-04-15T15:20:58.508 INFO:teuthology.orchestra.run.smithi154.stdout:43 2024-04-15T15:20:58.509 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-04-15T15:20:58.509 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2024-04-15T15:20:58.509 DEBUG:teuthology.orchestra.run.smithi154:> import glob 2024-04-15T15:20:58.509 DEBUG:teuthology.orchestra.run.smithi154:> import re 2024-04-15T15:20:58.509 DEBUG:teuthology.orchestra.run.smithi154:> import os 2024-04-15T15:20:58.509 DEBUG:teuthology.orchestra.run.smithi154:> import subprocess 2024-04-15T15:20:58.509 DEBUG:teuthology.orchestra.run.smithi154:> 2024-04-15T15:20:58.509 DEBUG:teuthology.orchestra.run.smithi154:> def _find_admin_socket(client_name): 2024-04-15T15:20:58.509 DEBUG:teuthology.orchestra.run.smithi154:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-15T15:20:58.509 DEBUG:teuthology.orchestra.run.smithi154:> files = glob.glob(asok_path) 2024-04-15T15:20:58.509 DEBUG:teuthology.orchestra.run.smithi154:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-15T15:20:58.509 DEBUG:teuthology.orchestra.run.smithi154:> 2024-04-15T15:20:58.509 DEBUG:teuthology.orchestra.run.smithi154:> # Given a non-glob path, it better be there 2024-04-15T15:20:58.509 DEBUG:teuthology.orchestra.run.smithi154:> if "*" not in asok_path: 2024-04-15T15:20:58.510 DEBUG:teuthology.orchestra.run.smithi154:> assert(len(files) == 1) 2024-04-15T15:20:58.510 DEBUG:teuthology.orchestra.run.smithi154:> return files[0] 2024-04-15T15:20:58.510 DEBUG:teuthology.orchestra.run.smithi154:> 2024-04-15T15:20:58.510 DEBUG:teuthology.orchestra.run.smithi154:> for f in files: 2024-04-15T15:20:58.510 DEBUG:teuthology.orchestra.run.smithi154:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-15T15:20:58.510 DEBUG:teuthology.orchestra.run.smithi154:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-15T15:20:58.510 DEBUG:teuthology.orchestra.run.smithi154:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-15T15:20:58.510 DEBUG:teuthology.orchestra.run.smithi154:> contents = proc_f.read() 2024-04-15T15:20:58.510 DEBUG:teuthology.orchestra.run.smithi154:> if mountpoint in contents: 2024-04-15T15:20:58.510 DEBUG:teuthology.orchestra.run.smithi154:> return f 2024-04-15T15:20:58.510 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-15T15:20:58.510 DEBUG:teuthology.orchestra.run.smithi154:> 2024-04-15T15:20:58.510 DEBUG:teuthology.orchestra.run.smithi154:> print(_find_admin_socket("client.1")) 2024-04-15T15:20:58.510 DEBUG:teuthology.orchestra.run.smithi154:> ' 2024-04-15T15:20:58.631 INFO:teuthology.orchestra.run.smithi154.stdout:/var/run/ceph/ceph-client.1.45615.asok 2024-04-15T15:20:58.634 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:20:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:20:58.641 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.45615.asok 2024-04-15T15:20:58.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:58.641 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.45615.asok status 2024-04-15T15:20:58.781 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-04-15T15:20:58.781 INFO:teuthology.orchestra.run.smithi154.stdout: "metadata": { 2024-04-15T15:20:58.781 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_sha1": "9bf1c4ba967dd24077be6e5b87edda4ca87a6e14", 2024-04-15T15:20:58.781 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_version": "ceph version 19.0.0-2861-g9bf1c4ba (9bf1c4ba967dd24077be6e5b87edda4ca87a6e14) squid (dev)", 2024-04-15T15:20:58.781 INFO:teuthology.orchestra.run.smithi154.stdout: "entity_id": "1", 2024-04-15T15:20:58.781 INFO:teuthology.orchestra.run.smithi154.stdout: "hostname": "smithi154", 2024-04-15T15:20:58.781 INFO:teuthology.orchestra.run.smithi154.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-15T15:20:58.782 INFO:teuthology.orchestra.run.smithi154.stdout: "pid": "45615", 2024-04-15T15:20:58.782 INFO:teuthology.orchestra.run.smithi154.stdout: "root": "/volumes/_nogroup/subvol_quiesce/3b88d86e-6ee6-481f-8fa4-d97e9f88b65d" 2024-04-15T15:20:58.782 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-15T15:20:58.782 INFO:teuthology.orchestra.run.smithi154.stdout: "dentry_count": 0, 2024-04-15T15:20:58.782 INFO:teuthology.orchestra.run.smithi154.stdout: "dentry_pinned_count": 0, 2024-04-15T15:20:58.782 INFO:teuthology.orchestra.run.smithi154.stdout: "id": 5003, 2024-04-15T15:20:58.782 INFO:teuthology.orchestra.run.smithi154.stdout: "inst": { 2024-04-15T15:20:58.783 INFO:teuthology.orchestra.run.smithi154.stdout: "name": { 2024-04-15T15:20:58.783 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "client", 2024-04-15T15:20:58.783 INFO:teuthology.orchestra.run.smithi154.stdout: "num": 5003 2024-04-15T15:20:58.783 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-15T15:20:58.783 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": { 2024-04-15T15:20:58.783 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v1", 2024-04-15T15:20:58.783 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "192.168.0.1:0", 2024-04-15T15:20:58.784 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3613535702 2024-04-15T15:20:58.784 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-15T15:20:58.784 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-15T15:20:58.784 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": { 2024-04-15T15:20:58.784 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v1", 2024-04-15T15:20:58.784 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "192.168.0.1:0", 2024-04-15T15:20:58.785 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3613535702 2024-04-15T15:20:58.785 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-15T15:20:58.785 INFO:teuthology.orchestra.run.smithi154.stdout: "inst_str": "client.5003 192.168.0.1:0/3613535702", 2024-04-15T15:20:58.785 INFO:teuthology.orchestra.run.smithi154.stdout: "addr_str": "192.168.0.1:0/3613535702", 2024-04-15T15:20:58.785 INFO:teuthology.orchestra.run.smithi154.stdout: "inode_count": 5, 2024-04-15T15:20:58.785 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_epoch": 14, 2024-04-15T15:20:58.785 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_epoch": 34, 2024-04-15T15:20:58.786 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_epoch_barrier": 0, 2024-04-15T15:20:58.786 INFO:teuthology.orchestra.run.smithi154.stdout: "blocklisted": false, 2024-04-15T15:20:58.786 INFO:teuthology.orchestra.run.smithi154.stdout: "fs_name": "cephfs" 2024-04-15T15:20:58.786 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-04-15T15:20:58.789 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-15T15:20:58.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:58.902+0000 7f72122ba640 1 Processor -- start 2024-04-15T15:20:58.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:58.902+0000 7f72122ba640 1 -- start start 2024-04-15T15:20:58.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:58.903+0000 7f72122ba640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f720c0d0f10 0x7f720c169c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:58.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:58.903+0000 7f72122ba640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f720c06d2a0 con 0x7f720c06e060 2024-04-15T15:20:58.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:58.903+0000 7f72122ba640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f720c06d580 con 0x7f720c15ecf0 2024-04-15T15:20:58.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:58.903+0000 7f72122ba640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f720c06d860 con 0x7f720c0d0f10 2024-04-15T15:20:58.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:58.903+0000 7f720b7fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f720c06e060 0x7f720c0cd360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:45920/0 (socket says 172.21.15.105:45920) 2024-04-15T15:20:58.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:58.903+0000 7f720b7fe640 1 -- 172.21.15.105:0/490780715 learned_addr learned my addr 172.21.15.105:0/490780715 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:20:58.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:58.903+0000 7f7210857640 1 --2- 172.21.15.105:0/490780715 >> v2:172.21.15.154:3300/0 conn(0x7f720c0d0f10 0x7f720c169c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:58.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:58.904+0000 7f720affd640 1 -- 172.21.15.105:0/490780715 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1228100180 0 0) 0x7f720c06d2a0 con 0x7f720c06e060 2024-04-15T15:20:58.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:58.904+0000 7f720affd640 1 -- 172.21.15.105:0/490780715 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f71f4003520 con 0x7f720c06e060 2024-04-15T15:20:58.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:58.904+0000 7f7210857640 1 -- 172.21.15.105:0/490780715 >> v1:172.21.15.154:6789/0 conn(0x7f720c15ecf0 legacy=0x7f720c0ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:58.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:58.904+0000 7f720affd640 1 -- 172.21.15.105:0/490780715 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2794037926 0 0) 0x7f720c06d580 con 0x7f720c15ecf0 2024-04-15T15:20:58.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:58.904+0000 7f7210857640 1 -- 172.21.15.105:0/490780715 >> v1:172.21.15.105:6789/0 conn(0x7f720c06e060 legacy=0x7f720c0cd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:58.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:58.904+0000 7f7210857640 1 -- 172.21.15.105:0/490780715 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f720c06db40 con 0x7f720c0d0f10 2024-04-15T15:20:58.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:58.905+0000 7f7210857640 1 --2- 172.21.15.105:0/490780715 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f720c0d0f10 0x7f720c169c90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7204009d90 tx=0x7f720402bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16001f3c6c055535 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:58.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:58.905+0000 7f720affd640 1 -- 172.21.15.105:0/490780715 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7204039070 con 0x7f720c0d0f10 2024-04-15T15:20:58.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:58.905+0000 7f720affd640 1 -- 172.21.15.105:0/490780715 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f720402cd00 con 0x7f720c0d0f10 2024-04-15T15:20:58.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:58.905+0000 7f720affd640 1 -- 172.21.15.105:0/490780715 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7204034660 con 0x7f720c0d0f10 2024-04-15T15:20:58.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:58.905+0000 7f72122ba640 1 -- 172.21.15.105:0/490780715 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f720c0d0f10 msgr2=0x7f720c169c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:58.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:58.905+0000 7f72122ba640 1 --2- 172.21.15.105:0/490780715 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f720c0d0f10 0x7f720c169c90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7204009d90 tx=0x7f720402bf50 comp rx=0 tx=0).stop 2024-04-15T15:20:58.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:58.905+0000 7f72122ba640 1 -- 172.21.15.105:0/490780715 shutdown_connections 2024-04-15T15:20:58.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:58.905+0000 7f72122ba640 1 --2- 172.21.15.105:0/490780715 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f720c0d0f10 0x7f720c169c90 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:58.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:58.905+0000 7f72122ba640 1 -- 172.21.15.105:0/490780715 >> 172.21.15.105:0/490780715 conn(0x7f720c0d2600 msgr2=0x7f720c0d2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:58.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:58.905+0000 7f72122ba640 1 -- 172.21.15.105:0/490780715 shutdown_connections 2024-04-15T15:20:58.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:58.905+0000 7f72122ba640 1 -- 172.21.15.105:0/490780715 wait complete. 2024-04-15T15:20:58.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:58.907+0000 7f72122ba640 1 Processor -- start 2024-04-15T15:20:58.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:58.907+0000 7f72122ba640 1 -- start start 2024-04-15T15:20:58.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:58.908+0000 7f72122ba640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f720c06e060 0x7f720c1593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:58.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:58.908+0000 7f72122ba640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f720c0d0f10 0x7f720c0b6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:58.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:58.908+0000 7f72122ba640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f720c15ecf0 0x7f720c09aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:58.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:58.908+0000 7f72122ba640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f720c15f0d0 con 0x7f720c0d0f10 2024-04-15T15:20:58.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:58.908+0000 7f72122ba640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f720c06cf90 con 0x7f720c06e060 2024-04-15T15:20:58.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:58.908+0000 7f72122ba640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f720c15f250 con 0x7f720c15ecf0 2024-04-15T15:20:58.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:58.908+0000 7f720b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f720c0d0f10 0x7f720c0b6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:58.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:58.908+0000 7f720b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f720c0d0f10 0x7f720c0b6de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35088/0 (socket says 172.21.15.105:35088) 2024-04-15T15:20:58.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:58.908+0000 7f720bfff640 1 --2- 172.21.15.105:0/3712205864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f720c06e060 0x7f720c1593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:58.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:58.908+0000 7f720b7fe640 1 -- 172.21.15.105:0/3712205864 learned_addr learned my addr 172.21.15.105:0/3712205864 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:20:58.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:58.908+0000 7f7210857640 1 --2- 172.21.15.105:0/3712205864 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f720c15ecf0 0x7f720c09aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:58.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:58.908+0000 7f720b7fe640 1 -- 172.21.15.105:0/3712205864 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f720c15ecf0 msgr2=0x7f720c09aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:58.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:58.908+0000 7f720b7fe640 1 --2- 172.21.15.105:0/3712205864 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f720c15ecf0 0x7f720c09aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:58.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:58.908+0000 7f720b7fe640 1 -- 172.21.15.105:0/3712205864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f720c06e060 msgr2=0x7f720c1593e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:58.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:58.908+0000 7f720b7fe640 1 --2- 172.21.15.105:0/3712205864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f720c06e060 0x7f720c1593e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:58.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:58.908+0000 7f720b7fe640 1 -- 172.21.15.105:0/3712205864 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f720c0c6e30 con 0x7f720c0d0f10 2024-04-15T15:20:58.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:58.908+0000 7f720bfff640 1 --2- 172.21.15.105:0/3712205864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f720c06e060 0x7f720c1593e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:20:58.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:58.908+0000 7f720b7fe640 1 --2- 172.21.15.105:0/3712205864 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f720c0d0f10 0x7f720c0b6de0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f720000c390 tx=0x7f720000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:58.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:58.908+0000 7f7208ff9640 1 -- 172.21.15.105:0/3712205864 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7200017070 con 0x7f720c0d0f10 2024-04-15T15:20:58.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:58.908+0000 7f7208ff9640 1 -- 172.21.15.105:0/3712205864 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7200005ce0 con 0x7f720c0d0f10 2024-04-15T15:20:58.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:58.908+0000 7f72122ba640 1 -- 172.21.15.105:0/3712205864 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f720c0c7110 con 0x7f720c0d0f10 2024-04-15T15:20:58.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:58.908+0000 7f7208ff9640 1 -- 172.21.15.105:0/3712205864 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72000072f0 con 0x7f720c0d0f10 2024-04-15T15:20:58.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:58.908+0000 7f72122ba640 1 -- 172.21.15.105:0/3712205864 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f720c0c76d0 con 0x7f720c0d0f10 2024-04-15T15:20:58.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:58.909+0000 7f72122ba640 1 -- 172.21.15.105:0/3712205864 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f720c068a40 con 0x7f720c0d0f10 2024-04-15T15:20:58.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:58.910+0000 7f7208ff9640 1 -- 172.21.15.105:0/3712205864 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f720001a070 con 0x7f720c0d0f10 2024-04-15T15:20:58.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:58.910+0000 7f7208ff9640 1 --2- 172.21.15.105:0/3712205864 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f71f806c980 0x7f71f806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:58.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:58.911+0000 7f7208ff9640 1 -- 172.21.15.105:0/3712205864 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f720008af90 con 0x7f720c0d0f10 2024-04-15T15:20:58.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:58.911+0000 7f720bfff640 1 --2- 172.21.15.105:0/3712205864 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f71f806c980 0x7f71f806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:58.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:58.911+0000 7f720bfff640 1 --2- 172.21.15.105:0/3712205864 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f71f806c980 0x7f71f806ee40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f71fc000f60 tx=0x7f71fc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:58.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:58.915+0000 7f7208ff9640 1 -- 172.21.15.105:0/3712205864 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7200058d00 con 0x7f720c0d0f10 2024-04-15T15:20:59.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:59.125+0000 7f72122ba640 1 -- 172.21.15.105:0/3712205864 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f720c0b8580 con 0x7f720c0d0f10 2024-04-15T15:20:59.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:59.126+0000 7f7208ff9640 1 -- 172.21.15.105:0/3712205864 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v14) ==== 53+0+73 (secure 0 0 0) 0x7f720005d000 con 0x7f720c0d0f10 2024-04-15T15:20:59.128 INFO:teuthology.orchestra.run.smithi105.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-15T15:20:59.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:59.127+0000 7f72122ba640 1 -- 172.21.15.105:0/3712205864 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f71f806c980 msgr2=0x7f71f806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:59.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:59.127+0000 7f72122ba640 1 --2- 172.21.15.105:0/3712205864 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f71f806c980 0x7f71f806ee40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f71fc000f60 tx=0x7f71fc009040 comp rx=0 tx=0).stop 2024-04-15T15:20:59.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:59.127+0000 7f72122ba640 1 -- 172.21.15.105:0/3712205864 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f720c0d0f10 msgr2=0x7f720c0b6de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:59.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:59.127+0000 7f72122ba640 1 --2- 172.21.15.105:0/3712205864 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f720c0d0f10 0x7f720c0b6de0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f720000c390 tx=0x7f720000c850 comp rx=0 tx=0).stop 2024-04-15T15:20:59.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:59.128+0000 7f72122ba640 1 -- 172.21.15.105:0/3712205864 shutdown_connections 2024-04-15T15:20:59.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:59.128+0000 7f72122ba640 1 --2- 172.21.15.105:0/3712205864 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f720c15ecf0 0x7f720c09aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:59.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:59.128+0000 7f72122ba640 1 --2- 172.21.15.105:0/3712205864 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f71f806c980 0x7f71f806ee40 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:59.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:59.128+0000 7f72122ba640 1 --2- 172.21.15.105:0/3712205864 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f720c0d0f10 0x7f720c0b6de0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:59.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:59.128+0000 7f72122ba640 1 --2- 172.21.15.105:0/3712205864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f720c06e060 0x7f720c1593e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:59.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:59.128+0000 7f72122ba640 1 -- 172.21.15.105:0/3712205864 >> 172.21.15.105:0/3712205864 conn(0x7f720c0d2600 msgr2=0x7f720c0ccc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:59.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:59.128+0000 7f72122ba640 1 -- 172.21.15.105:0/3712205864 shutdown_connections 2024-04-15T15:20:59.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:59.128+0000 7f72122ba640 1 -- 172.21.15.105:0/3712205864 wait complete. 2024-04-15T15:20:59.140 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-15T15:20:59.140 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-15T15:20:59.140 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi105.front.sepia.ceph.com 2024-04-15T15:20:59.140 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-15T15:20:59.140 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-15T15:20:59.141 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-15T15:20:59.141 INFO:tasks.cephfs.mount:self.cephfs_mntpt = /volumes/_nogroup/subvol_quiesce/3b88d86e-6ee6-481f-8fa4-d97e9f88b65d 2024-04-15T15:20:59.141 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-15T15:20:59.141 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.2' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-15T15:20:59.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:59.141 DEBUG:teuthology.orchestra.run.smithi105:> ip addr 2024-04-15T15:20:59.157 INFO:teuthology.orchestra.run.smithi105.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-15T15:20:59.157 INFO:teuthology.orchestra.run.smithi105.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-15T15:20:59.157 INFO:teuthology.orchestra.run.smithi105.stdout: inet 127.0.0.1/8 scope host lo 2024-04-15T15:20:59.157 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:59.157 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 ::1/128 scope host 2024-04-15T15:20:59.157 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:59.157 INFO:teuthology.orchestra.run.smithi105.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:20:59.157 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b8 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:20:59.157 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f0 2024-04-15T15:20:59.157 INFO:teuthology.orchestra.run.smithi105.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:20:59.157 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b9 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:20:59.157 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f1 2024-04-15T15:20:59.157 INFO:teuthology.orchestra.run.smithi105.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:20:59.157 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:28 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:20:59.158 INFO:teuthology.orchestra.run.smithi105.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-15T15:20:59.158 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:29 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:20:59.158 INFO:teuthology.orchestra.run.smithi105.stdout: inet 172.21.15.105/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-15T15:20:59.158 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft 42628sec preferred_lft 42628sec 2024-04-15T15:20:59.158 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::ec4:7aff:fe8f:d429/64 scope link 2024-04-15T15:20:59.158 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:59.158 INFO:teuthology.orchestra.run.smithi105.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-15T15:20:59.158 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 12:bc:19:27:c6:bf brd ff:ff:ff:ff:ff:ff 2024-04-15T15:20:59.158 INFO:teuthology.orchestra.run.smithi105.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-15T15:20:59.158 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:59.158 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::7c5f:a9ff:fe78:3a4f/64 scope link 2024-04-15T15:20:59.158 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:59.158 INFO:teuthology.orchestra.run.smithi105.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-15T15:20:59.158 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 9e:e8:fb:17:06:9d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-15T15:20:59.158 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::9ce8:fbff:fe17:69d/64 scope link 2024-04-15T15:20:59.159 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:59.159 INFO:teuthology.orchestra.run.smithi105.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-15T15:20:59.159 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 12:bc:19:27:c6:bf brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-15T15:20:59.159 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::10bc:19ff:fe27:c6bf/64 scope link 2024-04-15T15:20:59.159 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:59.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:59.159 DEBUG:teuthology.orchestra.run.smithi105:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-15T15:20:59.224 INFO:teuthology.orchestra.run.smithi105.stdout:1 2024-04-15T15:20:59.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:59.226 DEBUG:teuthology.orchestra.run.smithi105:> route 2024-04-15T15:20:59.281 INFO:teuthology.orchestra.run.smithi105.stdout:Kernel IP routing table 2024-04-15T15:20:59.281 INFO:teuthology.orchestra.run.smithi105.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-15T15:20:59.282 INFO:teuthology.orchestra.run.smithi105.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-15T15:20:59.282 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-15T15:20:59.282 INFO:teuthology.orchestra.run.smithi105.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-15T15:20:59.282 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:20:59.282 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-04-15T15:20:59.282 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-15T15:20:59.282 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-15T15:20:59.283 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-15T15:20:59.283 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-04-15T15:20:59.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:20:59.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:20:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:20:59.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:59.420 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2024-04-15T15:20:59.436 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-15T15:20:59.477 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-15T15:20:59.478 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.2' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-15T15:20:59.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:59.478 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set brx.1 up 2024-04-15T15:20:59.507 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-04-15T15:20:59.507 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-15T15:20:59.508 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-15T15:20:59.564 INFO:teuthology.orchestra.run.smithi105.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-15T15:20:59.564 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-15T15:20:59.564 DEBUG:teuthology.orchestra.run.smithi105:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-15T15:20:59.619 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2024-04-15T15:20:59.689 DEBUG:teuthology.orchestra.run.smithi105:> cat /proc/self/mounts | awk '{print $2}' 2024-04-15T15:20:59.746 INFO:teuthology.orchestra.run.smithi105.stdout:/proc 2024-04-15T15:20:59.746 INFO:teuthology.orchestra.run.smithi105.stdout:/sys 2024-04-15T15:20:59.746 INFO:teuthology.orchestra.run.smithi105.stdout:/dev 2024-04-15T15:20:59.746 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/security 2024-04-15T15:20:59.746 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/shm 2024-04-15T15:20:59.747 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/pts 2024-04-15T15:20:59.747 INFO:teuthology.orchestra.run.smithi105.stdout:/run 2024-04-15T15:20:59.747 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/cgroup 2024-04-15T15:20:59.747 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/pstore 2024-04-15T15:20:59.747 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/bpf 2024-04-15T15:20:59.747 INFO:teuthology.orchestra.run.smithi105.stdout:/ 2024-04-15T15:20:59.747 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/selinux 2024-04-15T15:20:59.747 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/sys/fs/binfmt_misc 2024-04-15T15:20:59.747 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/mqueue 2024-04-15T15:20:59.747 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/hugepages 2024-04-15T15:20:59.747 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/debug 2024-04-15T15:20:59.747 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/tracing 2024-04-15T15:20:59.747 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/fuse/connections 2024-04-15T15:20:59.748 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/config 2024-04-15T15:20:59.748 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-sysctl.service 2024-04-15T15:20:59.748 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-15T15:20:59.748 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-15T15:20:59.748 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/nfs/rpc_pipefs 2024-04-15T15:20:59.748 INFO:teuthology.orchestra.run.smithi105.stdout:/run/user/1000 2024-04-15T15:20:59.748 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph 2024-04-15T15:20:59.748 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-0 2024-04-15T15:20:59.748 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-1 2024-04-15T15:20:59.748 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-2 2024-04-15T15:20:59.748 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-3 2024-04-15T15:20:59.748 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns 2024-04-15T15:20:59.748 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-15T15:20:59.748 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-15T15:20:59.749 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-15T15:20:59.749 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-15T15:20:59.749 INFO:teuthology.orchestra.run.smithi105.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-15T15:20:59.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:59.749 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2024-04-15T15:20:59.802 INFO:teuthology.orchestra.run.smithi105.stdout:42 2024-04-15T15:20:59.802 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [42] 2024-04-15T15:20:59.803 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2 --client_mountpoint=/volumes/_nogroup/subvol_quiesce/3b88d86e-6ee6-481f-8fa4-d97e9f88b65d) 2024-04-15T15:20:59.845 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2024-04-15T15:20:59.873 DEBUG:teuthology.orchestra.run.smithi105:> cat /proc/self/mounts | awk '{print $2}' 2024-04-15T15:20:59.929 INFO:teuthology.orchestra.run.smithi105.stdout:/proc 2024-04-15T15:20:59.930 INFO:teuthology.orchestra.run.smithi105.stdout:/sys 2024-04-15T15:20:59.930 INFO:teuthology.orchestra.run.smithi105.stdout:/dev 2024-04-15T15:20:59.930 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/security 2024-04-15T15:20:59.930 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/shm 2024-04-15T15:20:59.930 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/pts 2024-04-15T15:20:59.930 INFO:teuthology.orchestra.run.smithi105.stdout:/run 2024-04-15T15:20:59.930 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/cgroup 2024-04-15T15:20:59.930 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/pstore 2024-04-15T15:20:59.930 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/bpf 2024-04-15T15:20:59.930 INFO:teuthology.orchestra.run.smithi105.stdout:/ 2024-04-15T15:20:59.930 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/selinux 2024-04-15T15:20:59.930 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/sys/fs/binfmt_misc 2024-04-15T15:20:59.930 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/mqueue 2024-04-15T15:20:59.931 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/hugepages 2024-04-15T15:20:59.931 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/debug 2024-04-15T15:20:59.931 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/tracing 2024-04-15T15:20:59.931 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/fuse/connections 2024-04-15T15:20:59.931 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/config 2024-04-15T15:20:59.931 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-sysctl.service 2024-04-15T15:20:59.931 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-15T15:20:59.931 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-15T15:20:59.931 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/nfs/rpc_pipefs 2024-04-15T15:20:59.931 INFO:teuthology.orchestra.run.smithi105.stdout:/run/user/1000 2024-04-15T15:20:59.931 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph 2024-04-15T15:20:59.931 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-0 2024-04-15T15:20:59.931 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-1 2024-04-15T15:20:59.931 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-2 2024-04-15T15:20:59.932 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-3 2024-04-15T15:20:59.932 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns 2024-04-15T15:20:59.932 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-15T15:20:59.932 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-15T15:20:59.932 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-15T15:20:59.932 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-15T15:20:59.932 INFO:teuthology.orchestra.run.smithi105.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-15T15:20:59.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:59.933 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2024-04-15T15:20:59.934 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi105.stderr:2024-04-15T15:20:59.932+0000 7f054c049480 -1 init, newargv = 0x55f8e2931ba0 newargc=15 2024-04-15T15:20:59.934 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi105.stderr:ceph-fuse[54191]: starting ceph client 2024-04-15T15:20:59.940 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi105.stderr:ceph-fuse[54191]: starting fuse 2024-04-15T15:20:59.950 INFO:teuthology.orchestra.run.smithi105.stdout:42 2024-04-15T15:20:59.950 INFO:teuthology.orchestra.run.smithi105.stdout:44 2024-04-15T15:20:59.950 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42, 44] 2024-04-15T15:20:59.950 DEBUG:teuthology.orchestra.run.smithi105:> sudo stdin-killer -- python3 -c ' 2024-04-15T15:20:59.950 DEBUG:teuthology.orchestra.run.smithi105:> import glob 2024-04-15T15:20:59.950 DEBUG:teuthology.orchestra.run.smithi105:> import re 2024-04-15T15:20:59.950 DEBUG:teuthology.orchestra.run.smithi105:> import os 2024-04-15T15:20:59.950 DEBUG:teuthology.orchestra.run.smithi105:> import subprocess 2024-04-15T15:20:59.951 DEBUG:teuthology.orchestra.run.smithi105:> 2024-04-15T15:20:59.951 DEBUG:teuthology.orchestra.run.smithi105:> def _find_admin_socket(client_name): 2024-04-15T15:20:59.951 DEBUG:teuthology.orchestra.run.smithi105:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-04-15T15:20:59.951 DEBUG:teuthology.orchestra.run.smithi105:> files = glob.glob(asok_path) 2024-04-15T15:20:59.951 DEBUG:teuthology.orchestra.run.smithi105:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-04-15T15:20:59.951 DEBUG:teuthology.orchestra.run.smithi105:> 2024-04-15T15:20:59.951 DEBUG:teuthology.orchestra.run.smithi105:> # Given a non-glob path, it better be there 2024-04-15T15:20:59.951 DEBUG:teuthology.orchestra.run.smithi105:> if "*" not in asok_path: 2024-04-15T15:20:59.951 DEBUG:teuthology.orchestra.run.smithi105:> assert(len(files) == 1) 2024-04-15T15:20:59.951 DEBUG:teuthology.orchestra.run.smithi105:> return files[0] 2024-04-15T15:20:59.951 DEBUG:teuthology.orchestra.run.smithi105:> 2024-04-15T15:20:59.951 DEBUG:teuthology.orchestra.run.smithi105:> for f in files: 2024-04-15T15:20:59.951 DEBUG:teuthology.orchestra.run.smithi105:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-15T15:20:59.951 DEBUG:teuthology.orchestra.run.smithi105:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-15T15:20:59.951 DEBUG:teuthology.orchestra.run.smithi105:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-15T15:20:59.952 DEBUG:teuthology.orchestra.run.smithi105:> contents = proc_f.read() 2024-04-15T15:20:59.952 DEBUG:teuthology.orchestra.run.smithi105:> if mountpoint in contents: 2024-04-15T15:20:59.952 DEBUG:teuthology.orchestra.run.smithi105:> return f 2024-04-15T15:20:59.952 DEBUG:teuthology.orchestra.run.smithi105:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-15T15:20:59.952 DEBUG:teuthology.orchestra.run.smithi105:> 2024-04-15T15:20:59.952 DEBUG:teuthology.orchestra.run.smithi105:> print(_find_admin_socket("client.2")) 2024-04-15T15:20:59.952 DEBUG:teuthology.orchestra.run.smithi105:> ' 2024-04-15T15:21:00.071 INFO:teuthology.orchestra.run.smithi105.stdout:/var/run/ceph/ceph-client.2.54191.asok 2024-04-15T15:21:00.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:21:00.082 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.54191.asok 2024-04-15T15:21:00.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:21:00.082 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.54191.asok status 2024-04-15T15:21:00.221 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-04-15T15:21:00.221 INFO:teuthology.orchestra.run.smithi105.stdout: "metadata": { 2024-04-15T15:21:00.221 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_sha1": "9bf1c4ba967dd24077be6e5b87edda4ca87a6e14", 2024-04-15T15:21:00.221 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_version": "ceph version 19.0.0-2861-g9bf1c4ba (9bf1c4ba967dd24077be6e5b87edda4ca87a6e14) squid (dev)", 2024-04-15T15:21:00.221 INFO:teuthology.orchestra.run.smithi105.stdout: "entity_id": "2", 2024-04-15T15:21:00.221 INFO:teuthology.orchestra.run.smithi105.stdout: "hostname": "smithi105", 2024-04-15T15:21:00.221 INFO:teuthology.orchestra.run.smithi105.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-04-15T15:21:00.221 INFO:teuthology.orchestra.run.smithi105.stdout: "pid": "54191", 2024-04-15T15:21:00.221 INFO:teuthology.orchestra.run.smithi105.stdout: "root": "/volumes/_nogroup/subvol_quiesce/3b88d86e-6ee6-481f-8fa4-d97e9f88b65d" 2024-04-15T15:21:00.221 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-04-15T15:21:00.222 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_count": 0, 2024-04-15T15:21:00.222 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_pinned_count": 0, 2024-04-15T15:21:00.222 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 5640, 2024-04-15T15:21:00.222 INFO:teuthology.orchestra.run.smithi105.stdout: "inst": { 2024-04-15T15:21:00.222 INFO:teuthology.orchestra.run.smithi105.stdout: "name": { 2024-04-15T15:21:00.222 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "client", 2024-04-15T15:21:00.222 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 5640 2024-04-15T15:21:00.222 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-04-15T15:21:00.222 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": { 2024-04-15T15:21:00.222 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-04-15T15:21:00.222 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "192.168.0.2:0", 2024-04-15T15:21:00.222 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 556020658 2024-04-15T15:21:00.222 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-04-15T15:21:00.223 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-04-15T15:21:00.223 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": { 2024-04-15T15:21:00.223 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-04-15T15:21:00.223 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "192.168.0.2:0", 2024-04-15T15:21:00.223 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 556020658 2024-04-15T15:21:00.223 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-04-15T15:21:00.223 INFO:teuthology.orchestra.run.smithi105.stdout: "inst_str": "client.5640 192.168.0.2:0/556020658", 2024-04-15T15:21:00.223 INFO:teuthology.orchestra.run.smithi105.stdout: "addr_str": "192.168.0.2:0/556020658", 2024-04-15T15:21:00.223 INFO:teuthology.orchestra.run.smithi105.stdout: "inode_count": 5, 2024-04-15T15:21:00.223 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_epoch": 14, 2024-04-15T15:21:00.223 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_epoch": 34, 2024-04-15T15:21:00.223 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_epoch_barrier": 0, 2024-04-15T15:21:00.223 INFO:teuthology.orchestra.run.smithi105.stdout: "blocklisted": false, 2024-04-15T15:21:00.224 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs" 2024-04-15T15:21:00.224 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-04-15T15:21:00.230 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-15T15:21:00.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.331+0000 7f2cbd596640 1 Processor -- start 2024-04-15T15:21:00.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.331+0000 7f2cbd596640 1 -- start start 2024-04-15T15:21:00.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.332+0000 7f2cbd596640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2cb815eab0 0x7f2cb8162530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:00.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.332+0000 7f2cbd596640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2cb80b78f0 0x7f2cb80d11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:00.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.332+0000 7f2cbd596640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2cb8162ff0 con 0x7f2cb8162a70 2024-04-15T15:21:00.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.332+0000 7f2cbd596640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f2cb80d4540 con 0x7f2cb80b78f0 2024-04-15T15:21:00.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.332+0000 7f2cbd596640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f2cb815ee90 con 0x7f2cb815eab0 2024-04-15T15:21:00.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.332+0000 7f2cae7fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2cb80b78f0 0x7f2cb80d11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:00.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.332+0000 7f2cae7fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2cb80b78f0 0x7f2cb80d11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35102/0 (socket says 172.21.15.105:35102) 2024-04-15T15:21:00.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.332+0000 7f2cae7fc640 1 -- 172.21.15.105:0/1669475456 learned_addr learned my addr 172.21.15.105:0/1669475456 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:00.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.332+0000 7f2cb6ffd640 1 --2- 172.21.15.105:0/1669475456 >> v2:172.21.15.154:3300/0 conn(0x7f2cb815eab0 0x7f2cb8162530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:00.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.332+0000 7f2cae7fc640 1 -- 172.21.15.105:0/1669475456 >> v2:172.21.15.154:3300/0 conn(0x7f2cb815eab0 msgr2=0x7f2cb8162530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:00.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.332+0000 7f2cae7fc640 1 --2- 172.21.15.105:0/1669475456 >> v2:172.21.15.154:3300/0 conn(0x7f2cb815eab0 0x7f2cb8162530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:00.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.332+0000 7f2cae7fc640 1 -- 172.21.15.105:0/1669475456 >> v1:172.21.15.154:6789/0 conn(0x7f2cb8162a70 legacy=0x7f2cb80b6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:00.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.332+0000 7f2cae7fc640 1 -- 172.21.15.105:0/1669475456 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cb81632d0 con 0x7f2cb80b78f0 2024-04-15T15:21:00.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.333+0000 7f2cae7fc640 1 --2- 172.21.15.105:0/1669475456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cb80b78f0 0x7f2cb80d11a0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f2cb808cdd0 tx=0x7f2ca002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59a448467a7a5a6d server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:00.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.333+0000 7f2cb67fc640 1 -- 172.21.15.105:0/1669475456 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca0034070 con 0x7f2cb80b78f0 2024-04-15T15:21:00.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.333+0000 7f2cb67fc640 1 -- 172.21.15.105:0/1669475456 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ca0030860 con 0x7f2cb80b78f0 2024-04-15T15:21:00.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.333+0000 7f2cb67fc640 1 -- 172.21.15.105:0/1669475456 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca0030c30 con 0x7f2cb80b78f0 2024-04-15T15:21:00.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.333+0000 7f2cbd596640 1 -- 172.21.15.105:0/1669475456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cb80b78f0 msgr2=0x7f2cb80d11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:00.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.333+0000 7f2cbd596640 1 --2- 172.21.15.105:0/1669475456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cb80b78f0 0x7f2cb80d11a0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f2cb808cdd0 tx=0x7f2ca002b9a0 comp rx=0 tx=0).stop 2024-04-15T15:21:00.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.334+0000 7f2cbd596640 1 -- 172.21.15.105:0/1669475456 shutdown_connections 2024-04-15T15:21:00.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.334+0000 7f2cbd596640 1 --2- 172.21.15.105:0/1669475456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cb80b78f0 0x7f2cb80d11a0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:00.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.334+0000 7f2cbd596640 1 --2- 172.21.15.105:0/1669475456 >> v2:172.21.15.154:3300/0 conn(0x7f2cb815eab0 0x7f2cb8162530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:00.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.334+0000 7f2cbd596640 1 -- 172.21.15.105:0/1669475456 >> 172.21.15.105:0/1669475456 conn(0x7f2cb8067720 msgr2=0x7f2cb8067b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:00.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.334+0000 7f2cbd596640 1 -- 172.21.15.105:0/1669475456 shutdown_connections 2024-04-15T15:21:00.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.334+0000 7f2cbd596640 1 -- 172.21.15.105:0/1669475456 wait complete. 2024-04-15T15:21:00.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.335+0000 7f2cbd596640 1 Processor -- start 2024-04-15T15:21:00.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.335+0000 7f2cbd596640 1 -- start start 2024-04-15T15:21:00.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.336+0000 7f2cbd596640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cb80b78f0 0x7f2cb81f55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:00.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.336+0000 7f2cbd596640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2cb815eab0 0x7f2cb81574b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:00.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.336+0000 7f2cbd596640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2cb8162a70 0x7f2cb814f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:00.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.336+0000 7f2cbd596640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb80ba610 con 0x7f2cb80b78f0 2024-04-15T15:21:00.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.336+0000 7f2cbd596640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb80d3010 con 0x7f2cb8162a70 2024-04-15T15:21:00.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.336+0000 7f2cbd596640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f2cb80ba790 con 0x7f2cb815eab0 2024-04-15T15:21:00.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.336+0000 7f2cb6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cb80b78f0 0x7f2cb81f55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:00.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.336+0000 7f2cb6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cb80b78f0 0x7f2cb81f55e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35110/0 (socket says 172.21.15.105:35110) 2024-04-15T15:21:00.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.336+0000 7f2cb6ffd640 1 -- 172.21.15.105:0/2175261330 learned_addr learned my addr 172.21.15.105:0/2175261330 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:00.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.336+0000 7f2cae7fc640 1 --2- 172.21.15.105:0/2175261330 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2cb815eab0 0x7f2cb81574b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:00.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.336+0000 7f2cb77fe640 1 --2- 172.21.15.105:0/2175261330 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2cb8162a70 0x7f2cb814f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:00.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.336+0000 7f2cb6ffd640 1 -- 172.21.15.105:0/2175261330 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2cb815eab0 msgr2=0x7f2cb81574b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:00.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.336+0000 7f2cb6ffd640 1 --2- 172.21.15.105:0/2175261330 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2cb815eab0 0x7f2cb81574b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:00.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.336+0000 7f2cb6ffd640 1 -- 172.21.15.105:0/2175261330 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2cb8162a70 msgr2=0x7f2cb814f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:00.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.336+0000 7f2cb6ffd640 1 --2- 172.21.15.105:0/2175261330 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2cb8162a70 0x7f2cb814f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:00.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.336+0000 7f2cb6ffd640 1 -- 172.21.15.105:0/2175261330 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cb81f86a0 con 0x7f2cb80b78f0 2024-04-15T15:21:00.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.336+0000 7f2cae7fc640 1 --2- 172.21.15.105:0/2175261330 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2cb815eab0 0x7f2cb81574b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:21:00.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.336+0000 7f2cb77fe640 1 --2- 172.21.15.105:0/2175261330 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2cb8162a70 0x7f2cb814f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:21:00.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.336+0000 7f2cb6ffd640 1 --2- 172.21.15.105:0/2175261330 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cb80b78f0 0x7f2cb81f55e0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f2ca800e9e0 tx=0x7f2ca800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:00.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.336+0000 7f2caffff640 1 -- 172.21.15.105:0/2175261330 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca8017070 con 0x7f2cb80b78f0 2024-04-15T15:21:00.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.336+0000 7f2caffff640 1 -- 172.21.15.105:0/2175261330 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ca800c9d0 con 0x7f2cb80b78f0 2024-04-15T15:21:00.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.336+0000 7f2caffff640 1 -- 172.21.15.105:0/2175261330 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca800cdc0 con 0x7f2cb80b78f0 2024-04-15T15:21:00.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.336+0000 7f2cbd596640 1 -- 172.21.15.105:0/2175261330 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cb81f8980 con 0x7f2cb80b78f0 2024-04-15T15:21:00.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.336+0000 7f2cbd596640 1 -- 172.21.15.105:0/2175261330 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cb80c8860 con 0x7f2cb80b78f0 2024-04-15T15:21:00.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.337+0000 7f2cbd596640 1 -- 172.21.15.105:0/2175261330 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cb8068a40 con 0x7f2cb80b78f0 2024-04-15T15:21:00.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.338+0000 7f2caffff640 1 -- 172.21.15.105:0/2175261330 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f2ca801a070 con 0x7f2cb80b78f0 2024-04-15T15:21:00.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.338+0000 7f2caffff640 1 --2- 172.21.15.105:0/2175261330 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f2c9806c980 0x7f2c9806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:00.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.338+0000 7f2caffff640 1 -- 172.21.15.105:0/2175261330 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f2ca808bf50 con 0x7f2cb80b78f0 2024-04-15T15:21:00.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.338+0000 7f2cae7fc640 1 --2- 172.21.15.105:0/2175261330 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f2c9806c980 0x7f2c9806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:00.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.339+0000 7f2cae7fc640 1 --2- 172.21.15.105:0/2175261330 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f2c9806c980 0x7f2c9806ee40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0036000 tx=0x7f2ca003b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:00.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.342+0000 7f2caffff640 1 -- 172.21.15.105:0/2175261330 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2ca805a390 con 0x7f2cb80b78f0 2024-04-15T15:21:00.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.554+0000 7f2cbd596640 1 -- 172.21.15.105:0/2175261330 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f2cb80d23f0 con 0x7f2cb80b78f0 2024-04-15T15:21:00.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.555+0000 7f2caffff640 1 -- 172.21.15.105:0/2175261330 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v14) ==== 53+0+73 (secure 0 0 0) 0x7f2ca8059c90 con 0x7f2cb80b78f0 2024-04-15T15:21:00.556 INFO:teuthology.orchestra.run.smithi105.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-15T15:21:00.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.556+0000 7f2cbd596640 1 -- 172.21.15.105:0/2175261330 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f2c9806c980 msgr2=0x7f2c9806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:00.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.556+0000 7f2cbd596640 1 --2- 172.21.15.105:0/2175261330 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f2c9806c980 0x7f2c9806ee40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0036000 tx=0x7f2ca003b040 comp rx=0 tx=0).stop 2024-04-15T15:21:00.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.556+0000 7f2cbd596640 1 -- 172.21.15.105:0/2175261330 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cb80b78f0 msgr2=0x7f2cb81f55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:00.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.556+0000 7f2cbd596640 1 --2- 172.21.15.105:0/2175261330 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cb80b78f0 0x7f2cb81f55e0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f2ca800e9e0 tx=0x7f2ca800eea0 comp rx=0 tx=0).stop 2024-04-15T15:21:00.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.556+0000 7f2cbd596640 1 -- 172.21.15.105:0/2175261330 shutdown_connections 2024-04-15T15:21:00.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.556+0000 7f2cbd596640 1 --2- 172.21.15.105:0/2175261330 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2cb815eab0 0x7f2cb81574b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:00.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.556+0000 7f2cbd596640 1 --2- 172.21.15.105:0/2175261330 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f2c9806c980 0x7f2c9806ee40 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:00.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.556+0000 7f2cbd596640 1 --2- 172.21.15.105:0/2175261330 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2cb8162a70 0x7f2cb814f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:00.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.556+0000 7f2cbd596640 1 --2- 172.21.15.105:0/2175261330 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cb80b78f0 0x7f2cb81f55e0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:00.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.556+0000 7f2cbd596640 1 -- 172.21.15.105:0/2175261330 >> 172.21.15.105:0/2175261330 conn(0x7f2cb8067720 msgr2=0x7f2cb815d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:00.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.557+0000 7f2cbd596640 1 -- 172.21.15.105:0/2175261330 shutdown_connections 2024-04-15T15:21:00.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:00.557+0000 7f2cbd596640 1 -- 172.21.15.105:0/2175261330 wait complete. 2024-04-15T15:21:00.569 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-15T15:21:00.569 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-15T15:21:00.569 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi154.front.sepia.ceph.com 2024-04-15T15:21:00.569 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-15T15:21:00.569 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-15T15:21:00.569 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-15T15:21:00.569 INFO:tasks.cephfs.mount:self.cephfs_mntpt = /volumes/_nogroup/subvol_quiesce/3b88d86e-6ee6-481f-8fa4-d97e9f88b65d 2024-04-15T15:21:00.570 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-15T15:21:00.570 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.3' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-15T15:21:00.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:21:00.570 DEBUG:teuthology.orchestra.run.smithi154:> ip addr 2024-04-15T15:21:00.590 INFO:teuthology.orchestra.run.smithi154.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-15T15:21:00.590 INFO:teuthology.orchestra.run.smithi154.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-15T15:21:00.590 INFO:teuthology.orchestra.run.smithi154.stdout: inet 127.0.0.1/8 scope host lo 2024-04-15T15:21:00.590 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:21:00.590 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 ::1/128 scope host 2024-04-15T15:21:00.590 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:21:00.591 INFO:teuthology.orchestra.run.smithi154.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:21:00.591 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:54 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:21:00.591 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f0 2024-04-15T15:21:00.591 INFO:teuthology.orchestra.run.smithi154.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:21:00.591 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:55 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:21:00.591 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f1 2024-04-15T15:21:00.591 INFO:teuthology.orchestra.run.smithi154.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:21:00.591 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:60 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:21:00.591 INFO:teuthology.orchestra.run.smithi154.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-15T15:21:00.591 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:61 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:21:00.591 INFO:teuthology.orchestra.run.smithi154.stdout: inet 172.21.15.154/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-15T15:21:00.591 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft 42615sec preferred_lft 42615sec 2024-04-15T15:21:00.591 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::ec4:7aff:fe88:7761/64 scope link 2024-04-15T15:21:00.591 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:21:00.592 INFO:teuthology.orchestra.run.smithi154.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-15T15:21:00.592 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether a2:e6:6b:6d:26:a4 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:21:00.592 INFO:teuthology.orchestra.run.smithi154.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-15T15:21:00.592 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:21:00.592 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::1cc6:e4ff:fe35:1850/64 scope link 2024-04-15T15:21:00.592 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:21:00.592 INFO:teuthology.orchestra.run.smithi154.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-15T15:21:00.592 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether ce:c3:4d:70:5f:15 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-15T15:21:00.592 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::ccc3:4dff:fe70:5f15/64 scope link 2024-04-15T15:21:00.592 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:21:00.592 INFO:teuthology.orchestra.run.smithi154.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-15T15:21:00.592 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether a2:e6:6b:6d:26:a4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-15T15:21:00.592 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::a0e6:6bff:fe6d:26a4/64 scope link 2024-04-15T15:21:00.592 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:21:00.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:21:00.593 DEBUG:teuthology.orchestra.run.smithi154:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-15T15:21:00.663 INFO:teuthology.orchestra.run.smithi154.stdout:1 2024-04-15T15:21:00.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:21:00.665 DEBUG:teuthology.orchestra.run.smithi154:> route 2024-04-15T15:21:00.725 INFO:teuthology.orchestra.run.smithi154.stdout:Kernel IP routing table 2024-04-15T15:21:00.725 INFO:teuthology.orchestra.run.smithi154.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-15T15:21:00.725 INFO:teuthology.orchestra.run.smithi154.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-15T15:21:00.725 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-15T15:21:00.725 INFO:teuthology.orchestra.run.smithi154.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-15T15:21:00.726 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:21:00.726 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2024-04-15T15:21:00.726 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-15T15:21:00.726 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-15T15:21:00.726 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-15T15:21:00.726 DEBUG:teuthology.orchestra.run.smithi154:> ') 2024-04-15T15:21:00.822 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:21:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:21:00.869 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:21:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:21:00.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:21:00.876 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list 2024-04-15T15:21:00.892 INFO:teuthology.orchestra.run.smithi154.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-15T15:21:00.892 INFO:teuthology.orchestra.run.smithi154.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-15T15:21:00.892 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.3' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-15T15:21:00.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:21:00.893 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link set brx.1 up 2024-04-15T15:21:00.962 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-04-15T15:21:00.962 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-15T15:21:00.963 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-15T15:21:01.018 INFO:teuthology.orchestra.run.smithi154.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-15T15:21:01.019 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-15T15:21:01.019 DEBUG:teuthology.orchestra.run.smithi154:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-15T15:21:01.074 DEBUG:teuthology.orchestra.run.smithi154:> sudo modprobe fuse 2024-04-15T15:21:01.144 DEBUG:teuthology.orchestra.run.smithi154:> cat /proc/self/mounts | awk '{print $2}' 2024-04-15T15:21:01.201 INFO:teuthology.orchestra.run.smithi154.stdout:/proc 2024-04-15T15:21:01.201 INFO:teuthology.orchestra.run.smithi154.stdout:/sys 2024-04-15T15:21:01.201 INFO:teuthology.orchestra.run.smithi154.stdout:/dev 2024-04-15T15:21:01.201 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/security 2024-04-15T15:21:01.201 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/shm 2024-04-15T15:21:01.201 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/pts 2024-04-15T15:21:01.201 INFO:teuthology.orchestra.run.smithi154.stdout:/run 2024-04-15T15:21:01.201 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/cgroup 2024-04-15T15:21:01.201 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/pstore 2024-04-15T15:21:01.202 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/bpf 2024-04-15T15:21:01.202 INFO:teuthology.orchestra.run.smithi154.stdout:/ 2024-04-15T15:21:01.202 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/selinux 2024-04-15T15:21:01.202 INFO:teuthology.orchestra.run.smithi154.stdout:/proc/sys/fs/binfmt_misc 2024-04-15T15:21:01.202 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/mqueue 2024-04-15T15:21:01.202 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/hugepages 2024-04-15T15:21:01.202 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/debug 2024-04-15T15:21:01.202 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/tracing 2024-04-15T15:21:01.202 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-15T15:21:01.202 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/fuse/connections 2024-04-15T15:21:01.202 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-sysctl.service 2024-04-15T15:21:01.202 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/config 2024-04-15T15:21:01.203 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-15T15:21:01.203 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/nfs/rpc_pipefs 2024-04-15T15:21:01.203 INFO:teuthology.orchestra.run.smithi154.stdout:/run/user/1000 2024-04-15T15:21:01.203 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph 2024-04-15T15:21:01.203 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-4 2024-04-15T15:21:01.203 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-5 2024-04-15T15:21:01.203 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-6 2024-04-15T15:21:01.203 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-7 2024-04-15T15:21:01.203 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns 2024-04-15T15:21:01.203 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-15T15:21:01.203 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-15T15:21:01.203 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-15T15:21:01.203 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-15T15:21:01.203 INFO:teuthology.orchestra.run.smithi154.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-15T15:21:01.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:21:01.204 DEBUG:teuthology.orchestra.run.smithi154:> ls /sys/fs/fuse/connections 2024-04-15T15:21:01.257 INFO:teuthology.orchestra.run.smithi154.stdout:43 2024-04-15T15:21:01.257 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [43] 2024-04-15T15:21:01.257 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3 --client_mountpoint=/volumes/_nogroup/subvol_quiesce/3b88d86e-6ee6-481f-8fa4-d97e9f88b65d) 2024-04-15T15:21:01.300 DEBUG:teuthology.orchestra.run.smithi154:> sudo modprobe fuse 2024-04-15T15:21:01.328 DEBUG:teuthology.orchestra.run.smithi154:> cat /proc/self/mounts | awk '{print $2}' 2024-04-15T15:21:01.386 INFO:teuthology.orchestra.run.smithi154.stdout:/proc 2024-04-15T15:21:01.386 INFO:teuthology.orchestra.run.smithi154.stdout:/sys 2024-04-15T15:21:01.386 INFO:teuthology.orchestra.run.smithi154.stdout:/dev 2024-04-15T15:21:01.386 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/security 2024-04-15T15:21:01.386 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/shm 2024-04-15T15:21:01.386 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/pts 2024-04-15T15:21:01.386 INFO:teuthology.orchestra.run.smithi154.stdout:/run 2024-04-15T15:21:01.386 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/cgroup 2024-04-15T15:21:01.386 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/pstore 2024-04-15T15:21:01.386 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/bpf 2024-04-15T15:21:01.387 INFO:teuthology.orchestra.run.smithi154.stdout:/ 2024-04-15T15:21:01.387 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/selinux 2024-04-15T15:21:01.387 INFO:teuthology.orchestra.run.smithi154.stdout:/proc/sys/fs/binfmt_misc 2024-04-15T15:21:01.387 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/mqueue 2024-04-15T15:21:01.387 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/hugepages 2024-04-15T15:21:01.387 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/debug 2024-04-15T15:21:01.387 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/tracing 2024-04-15T15:21:01.387 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-15T15:21:01.387 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/fuse/connections 2024-04-15T15:21:01.387 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-sysctl.service 2024-04-15T15:21:01.387 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/config 2024-04-15T15:21:01.387 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-15T15:21:01.387 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/nfs/rpc_pipefs 2024-04-15T15:21:01.387 INFO:teuthology.orchestra.run.smithi154.stdout:/run/user/1000 2024-04-15T15:21:01.388 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph 2024-04-15T15:21:01.388 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-4 2024-04-15T15:21:01.388 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-5 2024-04-15T15:21:01.388 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-6 2024-04-15T15:21:01.388 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-7 2024-04-15T15:21:01.388 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns 2024-04-15T15:21:01.388 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-15T15:21:01.389 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-15T15:21:01.389 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-15T15:21:01.389 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-15T15:21:01.389 INFO:teuthology.orchestra.run.smithi154.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-15T15:21:01.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:21:01.389 DEBUG:teuthology.orchestra.run.smithi154:> ls /sys/fs/fuse/connections 2024-04-15T15:21:01.390 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi154.stderr:2024-04-15T15:21:01.388+0000 7fdf4a043480 -1 init, newargv = 0x55cf0b8241d0 newargc=15 2024-04-15T15:21:01.390 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi154.stderr:ceph-fuse[45956]: starting ceph client 2024-04-15T15:21:01.396 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi154.stderr:ceph-fuse[45956]: starting fuse 2024-04-15T15:21:01.405 INFO:teuthology.orchestra.run.smithi154.stdout:43 2024-04-15T15:21:01.405 INFO:teuthology.orchestra.run.smithi154.stdout:44 2024-04-15T15:21:01.405 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43, 44] 2024-04-15T15:21:01.405 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2024-04-15T15:21:01.405 DEBUG:teuthology.orchestra.run.smithi154:> import glob 2024-04-15T15:21:01.405 DEBUG:teuthology.orchestra.run.smithi154:> import re 2024-04-15T15:21:01.406 DEBUG:teuthology.orchestra.run.smithi154:> import os 2024-04-15T15:21:01.406 DEBUG:teuthology.orchestra.run.smithi154:> import subprocess 2024-04-15T15:21:01.406 DEBUG:teuthology.orchestra.run.smithi154:> 2024-04-15T15:21:01.406 DEBUG:teuthology.orchestra.run.smithi154:> def _find_admin_socket(client_name): 2024-04-15T15:21:01.406 DEBUG:teuthology.orchestra.run.smithi154:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-04-15T15:21:01.406 DEBUG:teuthology.orchestra.run.smithi154:> files = glob.glob(asok_path) 2024-04-15T15:21:01.406 DEBUG:teuthology.orchestra.run.smithi154:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-04-15T15:21:01.406 DEBUG:teuthology.orchestra.run.smithi154:> 2024-04-15T15:21:01.406 DEBUG:teuthology.orchestra.run.smithi154:> # Given a non-glob path, it better be there 2024-04-15T15:21:01.406 DEBUG:teuthology.orchestra.run.smithi154:> if "*" not in asok_path: 2024-04-15T15:21:01.406 DEBUG:teuthology.orchestra.run.smithi154:> assert(len(files) == 1) 2024-04-15T15:21:01.406 DEBUG:teuthology.orchestra.run.smithi154:> return files[0] 2024-04-15T15:21:01.406 DEBUG:teuthology.orchestra.run.smithi154:> 2024-04-15T15:21:01.406 DEBUG:teuthology.orchestra.run.smithi154:> for f in files: 2024-04-15T15:21:01.407 DEBUG:teuthology.orchestra.run.smithi154:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-15T15:21:01.407 DEBUG:teuthology.orchestra.run.smithi154:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-15T15:21:01.407 DEBUG:teuthology.orchestra.run.smithi154:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-15T15:21:01.407 DEBUG:teuthology.orchestra.run.smithi154:> contents = proc_f.read() 2024-04-15T15:21:01.407 DEBUG:teuthology.orchestra.run.smithi154:> if mountpoint in contents: 2024-04-15T15:21:01.407 DEBUG:teuthology.orchestra.run.smithi154:> return f 2024-04-15T15:21:01.407 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-15T15:21:01.407 DEBUG:teuthology.orchestra.run.smithi154:> 2024-04-15T15:21:01.407 DEBUG:teuthology.orchestra.run.smithi154:> print(_find_admin_socket("client.3")) 2024-04-15T15:21:01.407 DEBUG:teuthology.orchestra.run.smithi154:> ' 2024-04-15T15:21:01.527 INFO:teuthology.orchestra.run.smithi154.stdout:/var/run/ceph/ceph-client.3.45956.asok 2024-04-15T15:21:01.530 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-15T15:21:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:21:01.538 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.45956.asok 2024-04-15T15:21:01.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:21:01.538 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.45956.asok status 2024-04-15T15:21:01.683 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-04-15T15:21:01.683 INFO:teuthology.orchestra.run.smithi154.stdout: "metadata": { 2024-04-15T15:21:01.683 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_sha1": "9bf1c4ba967dd24077be6e5b87edda4ca87a6e14", 2024-04-15T15:21:01.684 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_version": "ceph version 19.0.0-2861-g9bf1c4ba (9bf1c4ba967dd24077be6e5b87edda4ca87a6e14) squid (dev)", 2024-04-15T15:21:01.684 INFO:teuthology.orchestra.run.smithi154.stdout: "entity_id": "3", 2024-04-15T15:21:01.684 INFO:teuthology.orchestra.run.smithi154.stdout: "hostname": "smithi154", 2024-04-15T15:21:01.684 INFO:teuthology.orchestra.run.smithi154.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-04-15T15:21:01.684 INFO:teuthology.orchestra.run.smithi154.stdout: "pid": "45956", 2024-04-15T15:21:01.684 INFO:teuthology.orchestra.run.smithi154.stdout: "root": "/volumes/_nogroup/subvol_quiesce/3b88d86e-6ee6-481f-8fa4-d97e9f88b65d" 2024-04-15T15:21:01.684 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-15T15:21:01.684 INFO:teuthology.orchestra.run.smithi154.stdout: "dentry_count": 0, 2024-04-15T15:21:01.684 INFO:teuthology.orchestra.run.smithi154.stdout: "dentry_pinned_count": 0, 2024-04-15T15:21:01.684 INFO:teuthology.orchestra.run.smithi154.stdout: "id": 5021, 2024-04-15T15:21:01.684 INFO:teuthology.orchestra.run.smithi154.stdout: "inst": { 2024-04-15T15:21:01.684 INFO:teuthology.orchestra.run.smithi154.stdout: "name": { 2024-04-15T15:21:01.684 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "client", 2024-04-15T15:21:01.685 INFO:teuthology.orchestra.run.smithi154.stdout: "num": 5021 2024-04-15T15:21:01.685 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-15T15:21:01.685 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": { 2024-04-15T15:21:01.685 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v1", 2024-04-15T15:21:01.685 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "192.168.0.2:0", 2024-04-15T15:21:01.685 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 1363180920 2024-04-15T15:21:01.685 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-15T15:21:01.685 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-15T15:21:01.685 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": { 2024-04-15T15:21:01.685 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v1", 2024-04-15T15:21:01.685 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "192.168.0.2:0", 2024-04-15T15:21:01.685 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 1363180920 2024-04-15T15:21:01.685 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-15T15:21:01.686 INFO:teuthology.orchestra.run.smithi154.stdout: "inst_str": "client.5021 192.168.0.2:0/1363180920", 2024-04-15T15:21:01.686 INFO:teuthology.orchestra.run.smithi154.stdout: "addr_str": "192.168.0.2:0/1363180920", 2024-04-15T15:21:01.686 INFO:teuthology.orchestra.run.smithi154.stdout: "inode_count": 5, 2024-04-15T15:21:01.686 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_epoch": 14, 2024-04-15T15:21:01.686 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_epoch": 34, 2024-04-15T15:21:01.686 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_epoch_barrier": 0, 2024-04-15T15:21:01.686 INFO:teuthology.orchestra.run.smithi154.stdout: "blocklisted": false, 2024-04-15T15:21:01.686 INFO:teuthology.orchestra.run.smithi154.stdout: "fs_name": "cephfs" 2024-04-15T15:21:01.686 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-04-15T15:21:01.692 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'touch file') 2024-04-15T15:21:01.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-15T15:21:01.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:21:01.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:21:01.762 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.quiesce.block -v 1 file) 2024-04-15T15:21:01.790 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 quiesce path /volumes/_nogroup/subvol_quiesce/3b88d86e-6ee6-481f-8fa4-d97e9f88b65d --wait 2024-04-15T15:21:01.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.935+0000 7f82b1cd9640 1 Processor -- start 2024-04-15T15:21:01.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.936+0000 7f82b1cd9640 1 -- start start 2024-04-15T15:21:01.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.936+0000 7f82b1cd9640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f82ac0ba0f0 0x7f82ac0cf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:01.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.936+0000 7f82b1cd9640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82ac0cf970 0x7f82ac0d31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:01.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.936+0000 7f82b1cd9640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82ac0d17d0 con 0x7f82ac15e830 2024-04-15T15:21:01.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.936+0000 7f82b1cd9640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f82ac0d2070 con 0x7f82ac0ba0f0 2024-04-15T15:21:01.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.936+0000 7f82b1cd9640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f82ac0cfde0 con 0x7f82ac0cf970 2024-04-15T15:21:01.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.936+0000 7f82a2ffd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f82ac0ba0f0 0x7f82ac0cf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:01.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.936+0000 7f82a2ffd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f82ac0ba0f0 0x7f82ac0cf430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46522/0 (socket says 172.21.15.105:46522) 2024-04-15T15:21:01.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.936+0000 7f82a2ffd640 1 -- 172.21.15.105:0/2372965934 learned_addr learned my addr 172.21.15.105:0/2372965934 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:01.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.937+0000 7f82abfff640 1 --2- 172.21.15.105:0/2372965934 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82ac0cf970 0x7f82ac0d31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:01.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.937+0000 7f82a2ffd640 1 -- 172.21.15.105:0/2372965934 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82ac0cf970 msgr2=0x7f82ac0d31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:01.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.937+0000 7f82a2ffd640 1 --2- 172.21.15.105:0/2372965934 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82ac0cf970 0x7f82ac0d31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:01.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.937+0000 7f82a2ffd640 1 -- 172.21.15.105:0/2372965934 >> v1:172.21.15.154:6789/0 conn(0x7f82ac15e830 legacy=0x7f82ac0d0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:01.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.937+0000 7f82a2ffd640 1 -- 172.21.15.105:0/2372965934 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82ac0d1ab0 con 0x7f82ac0ba0f0 2024-04-15T15:21:01.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.937+0000 7f82abfff640 1 --2- 172.21.15.105:0/2372965934 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82ac0cf970 0x7f82ac0d31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:21:01.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.938+0000 7f82a2ffd640 1 --2- 172.21.15.105:0/2372965934 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82ac0ba0f0 0x7f82ac0cf430 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f8294009870 tx=0x7f829402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d1cdc632f20055f server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:01.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.938+0000 7f82aaffd640 1 -- 172.21.15.105:0/2372965934 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8294034070 con 0x7f82ac0ba0f0 2024-04-15T15:21:01.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.938+0000 7f82aaffd640 1 -- 172.21.15.105:0/2372965934 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8294030840 con 0x7f82ac0ba0f0 2024-04-15T15:21:01.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.938+0000 7f82aaffd640 1 -- 172.21.15.105:0/2372965934 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8294030c10 con 0x7f82ac0ba0f0 2024-04-15T15:21:01.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.938+0000 7f82b1cd9640 1 -- 172.21.15.105:0/2372965934 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82ac0ba0f0 msgr2=0x7f82ac0cf430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:01.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.938+0000 7f82b1cd9640 1 --2- 172.21.15.105:0/2372965934 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82ac0ba0f0 0x7f82ac0cf430 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f8294009870 tx=0x7f829402b980 comp rx=0 tx=0).stop 2024-04-15T15:21:01.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.938+0000 7f82b1cd9640 1 -- 172.21.15.105:0/2372965934 shutdown_connections 2024-04-15T15:21:01.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.938+0000 7f82b1cd9640 1 --2- 172.21.15.105:0/2372965934 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82ac0cf970 0x7f82ac0d31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:01.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.938+0000 7f82b1cd9640 1 --2- 172.21.15.105:0/2372965934 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82ac0ba0f0 0x7f82ac0cf430 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:01.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.938+0000 7f82b1cd9640 1 -- 172.21.15.105:0/2372965934 >> 172.21.15.105:0/2372965934 conn(0x7f82ac06ccb0 msgr2=0x7f82ac06d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:01.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.938+0000 7f82b1cd9640 1 -- 172.21.15.105:0/2372965934 shutdown_connections 2024-04-15T15:21:01.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.938+0000 7f82b1cd9640 1 -- 172.21.15.105:0/2372965934 wait complete. 2024-04-15T15:21:01.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.939+0000 7f82b1cd9640 1 Processor -- start 2024-04-15T15:21:01.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.940+0000 7f82b1cd9640 1 -- start start 2024-04-15T15:21:01.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.940+0000 7f82b1cd9640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82ac0ba0f0 0x7f82ac0c7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:01.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.940+0000 7f82b1cd9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82ac0cf970 0x7f82ac161250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:01.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.940+0000 7f82b1cd9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82ac15e830 0x7f82ac1572f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:01.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.940+0000 7f82b1cd9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f82ac15ed90 con 0x7f82ac0cf970 2024-04-15T15:21:01.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.940+0000 7f82b1cd9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f82ac15ec10 con 0x7f82ac15e830 2024-04-15T15:21:01.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.940+0000 7f82b1cd9640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f82ac0d36f0 con 0x7f82ac0ba0f0 2024-04-15T15:21:01.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.940+0000 7f82a2ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82ac0cf970 0x7f82ac161250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:01.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.940+0000 7f82a2ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82ac0cf970 0x7f82ac161250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46530/0 (socket says 172.21.15.105:46530) 2024-04-15T15:21:01.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.940+0000 7f82a2ffd640 1 -- 172.21.15.105:0/4152727552 learned_addr learned my addr 172.21.15.105:0/4152727552 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:01.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.941+0000 7f82ab7fe640 1 --2- 172.21.15.105:0/4152727552 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82ac0ba0f0 0x7f82ac0c7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:01.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.941+0000 7f82abfff640 1 --2- 172.21.15.105:0/4152727552 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82ac15e830 0x7f82ac1572f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:01.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.941+0000 7f82a2ffd640 1 -- 172.21.15.105:0/4152727552 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82ac0ba0f0 msgr2=0x7f82ac0c7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:01.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.941+0000 7f82a2ffd640 1 --2- 172.21.15.105:0/4152727552 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82ac0ba0f0 0x7f82ac0c7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:01.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.941+0000 7f82a2ffd640 1 -- 172.21.15.105:0/4152727552 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82ac15e830 msgr2=0x7f82ac1572f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:01.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.941+0000 7f82a2ffd640 1 --2- 172.21.15.105:0/4152727552 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82ac15e830 0x7f82ac1572f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:01.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.941+0000 7f82a2ffd640 1 -- 172.21.15.105:0/4152727552 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82ac1601b0 con 0x7f82ac0cf970 2024-04-15T15:21:01.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.941+0000 7f82ab7fe640 1 --2- 172.21.15.105:0/4152727552 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82ac0ba0f0 0x7f82ac0c7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:21:01.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.941+0000 7f82a2ffd640 1 --2- 172.21.15.105:0/4152727552 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82ac0cf970 0x7f82ac161250 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f8294000c00 tx=0x7f8294004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:01.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.941+0000 7f82a8ff9640 1 -- 172.21.15.105:0/4152727552 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8294048070 con 0x7f82ac0cf970 2024-04-15T15:21:01.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.941+0000 7f82a8ff9640 1 -- 172.21.15.105:0/4152727552 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8294030640 con 0x7f82ac0cf970 2024-04-15T15:21:01.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.941+0000 7f82b1cd9640 1 -- 172.21.15.105:0/4152727552 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82ac160490 con 0x7f82ac0cf970 2024-04-15T15:21:01.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.941+0000 7f82a8ff9640 1 -- 172.21.15.105:0/4152727552 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8294034040 con 0x7f82ac0cf970 2024-04-15T15:21:01.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.941+0000 7f82b1cd9640 1 -- 172.21.15.105:0/4152727552 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82ac160a50 con 0x7f82ac0cf970 2024-04-15T15:21:01.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.943+0000 7f82a8ff9640 1 -- 172.21.15.105:0/4152727552 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f829404b070 con 0x7f82ac0cf970 2024-04-15T15:21:01.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.943+0000 7f82a8ff9640 1 --2- 172.21.15.105:0/4152727552 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f828c06c980 0x7f828c06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:01.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.943+0000 7f82ab7fe640 1 --2- 172.21.15.105:0/4152727552 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f828c06c980 0x7f828c06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:01.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.943+0000 7f82a8ff9640 1 -- 172.21.15.105:0/4152727552 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f82940adc00 con 0x7f82ac0cf970 2024-04-15T15:21:01.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.944+0000 7f82ab7fe640 1 --2- 172.21.15.105:0/4152727552 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f828c06c980 0x7f828c06ee40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f829c001660 tx=0x7f829c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:01.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.944+0000 7f82b49a0740 1 Processor -- start 2024-04-15T15:21:01.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.944+0000 7f82b49a0740 1 -- start start 2024-04-15T15:21:01.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.945+0000 7f82b49a0740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c3376e6e80 0x55c3376b9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:01.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.945+0000 7f82b49a0740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55c3376e7260 0x55c3376e9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:01.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.945+0000 7f82b49a0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3376ba360 0x55c3376ea150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:01.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.945+0000 7f82b49a0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c337681ad0 con 0x55c3376ba360 2024-04-15T15:21:01.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.945+0000 7f82b49a0740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55c337625560 con 0x55c3376e6e80 2024-04-15T15:21:01.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.945+0000 7f82b49a0740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x55c3375cbc00 con 0x55c3376e7260 2024-04-15T15:21:01.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.945+0000 7f82abfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3376ba360 0x55c3376ea150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:01.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.945+0000 7f82abfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3376ba360 0x55c3376ea150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46534/0 (socket says 172.21.15.105:46534) 2024-04-15T15:21:01.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.945+0000 7f82abfff640 1 -- 172.21.15.105:0/3588634407 learned_addr learned my addr 172.21.15.105:0/3588634407 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:01.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.945+0000 7f82ab7fe640 1 --2- 172.21.15.105:0/3588634407 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c3376e6e80 0x55c3376b9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:01.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.945+0000 7f82a2ffd640 1 --2- 172.21.15.105:0/3588634407 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55c3376e7260 0x55c3376e9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:01.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.945+0000 7f82abfff640 1 -- 172.21.15.105:0/3588634407 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55c3376e7260 msgr2=0x55c3376e9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:01.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.945+0000 7f82abfff640 1 --2- 172.21.15.105:0/3588634407 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55c3376e7260 0x55c3376e9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:01.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.945+0000 7f82abfff640 1 -- 172.21.15.105:0/3588634407 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c3376e6e80 msgr2=0x55c3376b9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:01.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.945+0000 7f82abfff640 1 --2- 172.21.15.105:0/3588634407 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c3376e6e80 0x55c3376b9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:01.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.945+0000 7f82abfff640 1 -- 172.21.15.105:0/3588634407 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c337404030 con 0x55c3376ba360 2024-04-15T15:21:01.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.945+0000 7f82ab7fe640 1 --2- 172.21.15.105:0/3588634407 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c3376e6e80 0x55c3376b9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:21:01.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.946+0000 7f82abfff640 1 --2- 172.21.15.105:0/3588634407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3376ba360 0x55c3376ea150 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f82a4007510 tx=0x7f82a40079d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f042701572932f4c server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:01.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.946+0000 7f82a17fa640 1 -- 172.21.15.105:0/3588634407 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a401a020 con 0x55c3376ba360 2024-04-15T15:21:01.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.946+0000 7f82a17fa640 1 -- 172.21.15.105:0/3588634407 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82a4015850 con 0x55c3376ba360 2024-04-15T15:21:01.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.946+0000 7f82a17fa640 1 -- 172.21.15.105:0/3588634407 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a4015c00 con 0x55c3376ba360 2024-04-15T15:21:01.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.946+0000 7f82b49a0740 1 -- 172.21.15.105:0/3588634407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3376ba360 msgr2=0x55c3376ea150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:01.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.946+0000 7f82b49a0740 1 --2- 172.21.15.105:0/3588634407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3376ba360 0x55c3376ea150 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f82a4007510 tx=0x7f82a40079d0 comp rx=0 tx=0).stop 2024-04-15T15:21:01.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.947+0000 7f82b49a0740 1 -- 172.21.15.105:0/3588634407 shutdown_connections 2024-04-15T15:21:01.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.947+0000 7f82b49a0740 1 --2- 172.21.15.105:0/3588634407 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55c3376e7260 0x55c3376e9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:01.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.947+0000 7f82b49a0740 1 --2- 172.21.15.105:0/3588634407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3376ba360 0x55c3376ea150 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:01.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.947+0000 7f82b49a0740 1 --2- 172.21.15.105:0/3588634407 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c3376e6e80 0x55c3376b9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:01.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.947+0000 7f82b49a0740 1 -- 172.21.15.105:0/3588634407 >> 172.21.15.105:0/3588634407 conn(0x55c3376bc090 msgr2=0x55c337704c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:01.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.947+0000 7f82b49a0740 1 -- 172.21.15.105:0/3588634407 shutdown_connections 2024-04-15T15:21:01.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.947+0000 7f82b49a0740 1 -- 172.21.15.105:0/3588634407 wait complete. 2024-04-15T15:21:01.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.947+0000 7f82b49a0740 1 -- start start 2024-04-15T15:21:01.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.947+0000 7f82b49a0740 1 Processor -- start 2024-04-15T15:21:01.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.948+0000 7f82b49a0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3376e6e80 0x55c3376ed420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:01.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.948+0000 7f82b49a0740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55c3376e7260 0x55c3376f0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:01.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.948+0000 7f82b49a0740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c3376e4b70 0x55c3376f05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:01.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.948+0000 7f82b49a0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c3375df110 con 0x55c3376e6e80 2024-04-15T15:21:01.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.948+0000 7f82b49a0740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55c337621520 con 0x55c3376e4b70 2024-04-15T15:21:01.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.948+0000 7f82b49a0740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x55c3376be730 con 0x55c3376e7260 2024-04-15T15:21:01.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.948+0000 7f82b49a0740 10 client.0 _ll_register_callbacks cb 0x55c337337410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-15T15:21:01.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.948+0000 7f82ab7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3376e6e80 0x55c3376ed420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:01.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.948+0000 7f82ab7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3376e6e80 0x55c3376ed420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46536/0 (socket says 172.21.15.105:46536) 2024-04-15T15:21:01.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.948+0000 7f82ab7fe640 1 -- 172.21.15.105:0/160204517 learned_addr learned my addr 172.21.15.105:0/160204517 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:01.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.948+0000 7f82a2ffd640 1 --2- 172.21.15.105:0/160204517 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55c3376e7260 0x55c3376f0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:01.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.948+0000 7f82abfff640 1 --2- 172.21.15.105:0/160204517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c3376e4b70 0x55c3376f05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:01.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.948+0000 7f82ab7fe640 1 -- 172.21.15.105:0/160204517 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55c3376e7260 msgr2=0x55c3376f0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:01.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.948+0000 7f82ab7fe640 1 --2- 172.21.15.105:0/160204517 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55c3376e7260 0x55c3376f0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:01.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.948+0000 7f82ab7fe640 1 -- 172.21.15.105:0/160204517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c3376e4b70 msgr2=0x55c3376f05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:01.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.948+0000 7f82ab7fe640 1 --2- 172.21.15.105:0/160204517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c3376e4b70 0x55c3376f05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:01.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.948+0000 7f82ab7fe640 1 -- 172.21.15.105:0/160204517 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c337337c20 con 0x55c3376e6e80 2024-04-15T15:21:01.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.948+0000 7f82ab7fe640 1 --2- 172.21.15.105:0/160204517 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3376e6e80 0x55c3376ed420 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f829c001ea0 tx=0x7f829c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e76b33fbc09a78b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:01.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.949+0000 7f828a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-04-15T15:21:01.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.949+0000 7f828a7fc640 1 -- 172.21.15.105:0/160204517 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829c027060 con 0x55c3376e6e80 2024-04-15T15:21:01.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.949+0000 7f828a7fc640 1 -- 172.21.15.105:0/160204517 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f829c010040 con 0x55c3376e6e80 2024-04-15T15:21:01.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.949+0000 7f82b49a0740 1 -- 172.21.15.105:0/160204517 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82a4015850 con 0x55c3376e6e80 2024-04-15T15:21:01.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.949+0000 7f828a7fc640 1 -- 172.21.15.105:0/160204517 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) ==== 24+0+0 (secure 0 0 0) 0x7f829c003a70 con 0x55c3376e6e80 2024-04-15T15:21:01.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.949+0000 7f82b49a0740 10 client.5664 fetch_fsmap learned FSMap version 14 2024-04-15T15:21:01.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.949+0000 7f82b49a0740 1 -- 172.21.15.105:0/160204517 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) -- 0x55c3372f7920 con 0x55c3376e6e80 2024-04-15T15:21:01.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.949+0000 7f828a7fc640 1 -- 172.21.15.105:0/160204517 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82a4015850 con 0x55c3376e6e80 2024-04-15T15:21:01.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.949+0000 7f828a7fc640 1 -- 172.21.15.105:0/160204517 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 14) ==== 3494+0+0 (secure 0 0 0) 0x7f829c003c10 con 0x55c3376e6e80 2024-04-15T15:21:01.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.949+0000 7f82b49a0740 10 client.5664 fetch_fsmap finished waiting for FSMap version 14 2024-04-15T15:21:01.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.949+0000 7f82b49a0740 10 client.5664 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-15T15:21:01.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.949+0000 7f82b49a0740 20 client.5664 populate_metadata read hostname 'smithi105' 2024-04-15T15:21:01.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.949+0000 7f82b49a0740 1 --2- 172.21.15.105:0/160204517 >> [v2:172.21.15.105:6835/958066041,v1:172.21.15.105:6837/958066041] conn(0x55c3376eea10 0x55c3376eee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:01.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.949+0000 7f82b49a0740 4 client.5664 mds_command: new command op to 5196 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-15T15:21:01.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.949+0000 7f82b49a0740 1 -- 172.21.15.105:0/160204517 --> [v2:172.21.15.105:6835/958066041,v1:172.21.15.105:6837/958066041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c337632f80 con 0x55c3376eea10 2024-04-15T15:21:01.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.949+0000 7f82a2ffd640 1 --2- 172.21.15.105:0/160204517 >> [v2:172.21.15.105:6835/958066041,v1:172.21.15.105:6837/958066041] conn(0x55c3376eea10 0x55c3376eee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:01.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.949+0000 7f82a2ffd640 1 --2- 172.21.15.105:0/160204517 >> [v2:172.21.15.105:6835/958066041,v1:172.21.15.105:6837/958066041] conn(0x55c3376eea10 0x55c3376eee50 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b79d1c4b78e799db server_cookie=c08cb081a7009fee in_seq=0 out_seq=0 2024-04-15T15:21:01.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.949+0000 7f828a7fc640 10 client.5664 ms_handle_connect on v2:172.21.15.105:6835/958066041 2024-04-15T15:21:01.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.950+0000 7f828a7fc640 1 -- 172.21.15.105:0/160204517 <== mds.0 v2:172.21.15.105:6835/958066041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c337632f80 con 0x55c3376eea10 2024-04-15T15:21:01.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.950+0000 7f828a7fc640 10 client.5664 handle_command_reply: tid=0 2024-04-15T15:21:01.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.950+0000 7f82b49a0740 1 client.5664 shutdown 2024-04-15T15:21:01.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.950+0000 7f82b49a0740 1 -- 172.21.15.105:0/160204517 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3376e6e80 msgr2=0x55c3376ed420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:01.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.950+0000 7f82b49a0740 1 --2- 172.21.15.105:0/160204517 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3376e6e80 0x55c3376ed420 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f829c001ea0 tx=0x7f829c000bc0 comp rx=0 tx=0).stop 2024-04-15T15:21:01.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.950+0000 7f82b49a0740 1 -- 172.21.15.105:0/160204517 shutdown_connections 2024-04-15T15:21:01.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.950+0000 7f82b49a0740 1 --2- 172.21.15.105:0/160204517 >> [v2:172.21.15.105:6835/958066041,v1:172.21.15.105:6837/958066041] conn(0x55c3376eea10 0x55c3376eee50 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:01.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.950+0000 7f82b49a0740 1 --2- 172.21.15.105:0/160204517 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55c3376e7260 0x55c3376f0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:01.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.950+0000 7f82b49a0740 1 --2- 172.21.15.105:0/160204517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c3376e4b70 0x55c3376f05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:01.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.950+0000 7f82b49a0740 1 --2- 172.21.15.105:0/160204517 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3376e6e80 0x55c3376ed420 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:01.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.950+0000 7f828a7fc640 0 client.5664 ms_handle_reset on v2:172.21.15.105:6835/958066041 2024-04-15T15:21:01.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.950+0000 7f82b49a0740 1 -- 172.21.15.105:0/160204517 >> 172.21.15.105:0/160204517 conn(0x55c3376ba360 msgr2=0x55c337708120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:01.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.950+0000 7f82b49a0740 1 -- 172.21.15.105:0/160204517 shutdown_connections 2024-04-15T15:21:01.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.950+0000 7f82b49a0740 1 -- 172.21.15.105:0/160204517 wait complete. 2024-04-15T15:21:01.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.950+0000 7f82b49a0740 20 client.5664 trim_cache size 0 max 16384 2024-04-15T15:21:01.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.967+0000 7f82b49a0740 1 Processor -- start 2024-04-15T15:21:01.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.967+0000 7f82b49a0740 1 -- start start 2024-04-15T15:21:01.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.968+0000 7f82b49a0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3376e4b70 0x55c337704860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:01.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.968+0000 7f82b49a0740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c3376e6e80 0x55c337704da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:01.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.968+0000 7f82b49a0740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55c3376e7260 0x55c3376e4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:01.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.968+0000 7f82b49a0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f82ac201050 con 0x55c3376e4b70 2024-04-15T15:21:01.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.968+0000 7f82b49a0740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f82ac146cb0 con 0x55c3376e6e80 2024-04-15T15:21:01.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.968+0000 7f82b49a0740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f82ac1f3c60 con 0x55c3376e7260 2024-04-15T15:21:01.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.968+0000 7f82ab7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3376e4b70 0x55c337704860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:01.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.968+0000 7f82ab7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3376e4b70 0x55c337704860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46540/0 (socket says 172.21.15.105:46540) 2024-04-15T15:21:01.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.968+0000 7f82ab7fe640 1 -- 172.21.15.105:0/2424147102 learned_addr learned my addr 172.21.15.105:0/2424147102 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:01.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.968+0000 7f82ab7fe640 1 -- 172.21.15.105:0/2424147102 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55c3376e7260 msgr2=0x55c3376e4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:01.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.968+0000 7f82ab7fe640 1 --2- 172.21.15.105:0/2424147102 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55c3376e7260 0x55c3376e4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:01.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.968+0000 7f82ab7fe640 1 -- 172.21.15.105:0/2424147102 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c3376e6e80 msgr2=0x55c337704da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:01.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.968+0000 7f82ab7fe640 1 --2- 172.21.15.105:0/2424147102 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c3376e6e80 0x55c337704da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:01.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.968+0000 7f82ab7fe640 1 -- 172.21.15.105:0/2424147102 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3376ec3a0 con 0x55c3376e4b70 2024-04-15T15:21:01.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.968+0000 7f82ab7fe640 1 --2- 172.21.15.105:0/2424147102 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3376e4b70 0x55c337704860 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f829c001350 tx=0x7f829c01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cbb52b6297bb891 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:01.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.969+0000 7f82a0ff9640 1 -- 172.21.15.105:0/2424147102 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829c013510 con 0x55c3376e4b70 2024-04-15T15:21:01.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.969+0000 7f82a0ff9640 1 -- 172.21.15.105:0/2424147102 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f829c003c10 con 0x55c3376e4b70 2024-04-15T15:21:01.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.969+0000 7f82a0ff9640 1 -- 172.21.15.105:0/2424147102 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829c0137f0 con 0x55c3376e4b70 2024-04-15T15:21:01.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.969+0000 7f82b49a0740 1 -- 172.21.15.105:0/2424147102 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3376e4b70 msgr2=0x55c337704860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:01.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.969+0000 7f82b49a0740 1 --2- 172.21.15.105:0/2424147102 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3376e4b70 0x55c337704860 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f829c001350 tx=0x7f829c01f000 comp rx=0 tx=0).stop 2024-04-15T15:21:01.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.969+0000 7f82b49a0740 1 -- 172.21.15.105:0/2424147102 shutdown_connections 2024-04-15T15:21:01.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.969+0000 7f82b49a0740 1 --2- 172.21.15.105:0/2424147102 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55c3376e7260 0x55c3376e4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:01.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.969+0000 7f82b49a0740 1 --2- 172.21.15.105:0/2424147102 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c3376e6e80 0x55c337704da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:01.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.969+0000 7f82b49a0740 1 --2- 172.21.15.105:0/2424147102 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3376e4b70 0x55c337704860 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:01.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.969+0000 7f82b49a0740 1 -- 172.21.15.105:0/2424147102 >> 172.21.15.105:0/2424147102 conn(0x55c3376ba360 msgr2=0x55c337708120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:01.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.969+0000 7f82b49a0740 1 -- 172.21.15.105:0/2424147102 shutdown_connections 2024-04-15T15:21:01.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.969+0000 7f82b49a0740 1 -- 172.21.15.105:0/2424147102 wait complete. 2024-04-15T15:21:01.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.970+0000 7f82b49a0740 1 -- start start 2024-04-15T15:21:01.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.970+0000 7f82b49a0740 1 Processor -- start 2024-04-15T15:21:01.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.971+0000 7f82b49a0740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55c3376e4b70 0x55c3376efa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:01.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.971+0000 7f82b49a0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3376e6e80 0x55c3376eff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:01.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.971+0000 7f82b49a0740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c3376e7260 0x55c3376b8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:01.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.971+0000 7f82b49a0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f82ac146660 con 0x55c3376e6e80 2024-04-15T15:21:01.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.971+0000 7f82b49a0740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55c33777fa40 con 0x55c3376e7260 2024-04-15T15:21:01.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.971+0000 7f82b49a0740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x55c33771ea70 con 0x55c3376e4b70 2024-04-15T15:21:01.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.971+0000 7f82b49a0740 10 client.0 _ll_register_callbacks cb 0x7f829c001380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-15T15:21:01.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.971+0000 7f82a2ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3376e6e80 0x55c3376eff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:01.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.971+0000 7f82a2ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3376e6e80 0x55c3376eff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46544/0 (socket says 172.21.15.105:46544) 2024-04-15T15:21:01.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.971+0000 7f82a2ffd640 1 -- 172.21.15.105:0/1071192453 learned_addr learned my addr 172.21.15.105:0/1071192453 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:01.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.971+0000 7f82ab7fe640 1 --2- 172.21.15.105:0/1071192453 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55c3376e4b70 0x55c3376efa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:01.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.971+0000 7f82abfff640 1 --2- 172.21.15.105:0/1071192453 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c3376e7260 0x55c3376b8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:01.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.971+0000 7f82a2ffd640 1 -- 172.21.15.105:0/1071192453 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55c3376e4b70 msgr2=0x55c3376efa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:01.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.971+0000 7f82a2ffd640 1 --2- 172.21.15.105:0/1071192453 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55c3376e4b70 0x55c3376efa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:01.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.971+0000 7f82a2ffd640 1 -- 172.21.15.105:0/1071192453 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c3376e7260 msgr2=0x55c3376b8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:01.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.971+0000 7f82a2ffd640 1 --2- 172.21.15.105:0/1071192453 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c3376e7260 0x55c3376b8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:01.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.971+0000 7f82a2ffd640 1 -- 172.21.15.105:0/1071192453 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3376ef800 con 0x55c3376e6e80 2024-04-15T15:21:01.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.971+0000 7f82a2ffd640 1 --2- 172.21.15.105:0/1071192453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3376e6e80 0x55c3376eff40 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f8294009870 tx=0x7f829407b360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38a47ed00c71d586 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:01.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.971+0000 7f828a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-04-15T15:21:01.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.971+0000 7f828a7fc640 1 -- 172.21.15.105:0/1071192453 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82940ac1d0 con 0x55c3376e6e80 2024-04-15T15:21:01.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.971+0000 7f828a7fc640 1 -- 172.21.15.105:0/1071192453 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f8294077e90 con 0x55c3376e6e80 2024-04-15T15:21:01.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.971+0000 7f82b49a0740 1 -- 172.21.15.105:0/1071192453 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f829c003c10 con 0x55c3376e6e80 2024-04-15T15:21:01.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.972+0000 7f828a7fc640 1 -- 172.21.15.105:0/1071192453 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) ==== 24+0+0 (secure 0 0 0) 0x7f829407ac70 con 0x55c3376e6e80 2024-04-15T15:21:01.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.972+0000 7f82b49a0740 10 client.5670 fetch_fsmap learned FSMap version 14 2024-04-15T15:21:01.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.972+0000 7f82b49a0740 1 -- 172.21.15.105:0/1071192453 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) -- 0x55c3377068c0 con 0x55c3376e6e80 2024-04-15T15:21:01.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.972+0000 7f828a7fc640 1 -- 172.21.15.105:0/1071192453 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f829c003c10 con 0x55c3376e6e80 2024-04-15T15:21:01.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.972+0000 7f828a7fc640 1 -- 172.21.15.105:0/1071192453 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 14) ==== 3494+0+0 (secure 0 0 0) 0x7f829407ae10 con 0x55c3376e6e80 2024-04-15T15:21:01.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.972+0000 7f82b49a0740 10 client.5670 fetch_fsmap finished waiting for FSMap version 14 2024-04-15T15:21:01.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.972+0000 7f82b49a0740 10 client.5670 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-15T15:21:01.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.972+0000 7f82b49a0740 20 client.5670 populate_metadata read hostname 'smithi105' 2024-04-15T15:21:01.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.972+0000 7f82b49a0740 1 --2- 172.21.15.105:0/1071192453 >> [v2:172.21.15.105:6835/958066041,v1:172.21.15.105:6837/958066041] conn(0x55c337728670 0x55c3376b9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:01.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.972+0000 7f82b49a0740 4 client.5670 mds_command: new command op to 5196 tid=0 multi_id=0 [{"prefix": "quiesce path", "path": "/volumes/_nogroup/subvol_quiesce/3b88d86e-6ee6-481f-8fa4-d97e9f88b65d", "wait": true}] 2024-04-15T15:21:01.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.972+0000 7f82b49a0740 1 -- 172.21.15.105:0/1071192453 --> [v2:172.21.15.105:6835/958066041,v1:172.21.15.105:6837/958066041] -- command(tid 0: {"prefix": "quiesce path", "path": "/volumes/_nogroup/subvol_quiesce/3b88d86e-6ee6-481f-8fa4-d97e9f88b65d", "wait": true}) -- 0x55c33762cdf0 con 0x55c337728670 2024-04-15T15:21:01.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.972+0000 7f82ab7fe640 1 --2- 172.21.15.105:0/1071192453 >> [v2:172.21.15.105:6835/958066041,v1:172.21.15.105:6837/958066041] conn(0x55c337728670 0x55c3376b9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:01.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.972+0000 7f828a7fc640 10 client.5670 ms_handle_connect on v2:172.21.15.105:6835/958066041 2024-04-15T15:21:01.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.972+0000 7f82ab7fe640 1 --2- 172.21.15.105:0/1071192453 >> [v2:172.21.15.105:6835/958066041,v1:172.21.15.105:6837/958066041] conn(0x55c337728670 0x55c3376b9210 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61786411c8038dc8 server_cookie=43f5d17b6f41cadc in_seq=0 out_seq=0 2024-04-15T15:21:01.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.976+0000 7f828a7fc640 1 -- 172.21.15.105:0/1071192453 <== mds.0 v2:172.21.15.105:6835/958066041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2332 (crc 0 0 0) 0x55c33762cdf0 con 0x55c337728670 2024-04-15T15:21:01.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.976+0000 7f828a7fc640 10 client.5670 handle_command_reply: tid=0 2024-04-15T15:21:01.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.976+0000 7f82b49a0740 1 client.5670 shutdown 2024-04-15T15:21:01.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.977+0000 7f82b49a0740 1 -- 172.21.15.105:0/1071192453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3376e6e80 msgr2=0x55c3376eff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:01.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.977+0000 7f82b49a0740 1 --2- 172.21.15.105:0/1071192453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3376e6e80 0x55c3376eff40 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f8294009870 tx=0x7f829407b360 comp rx=0 tx=0).stop 2024-04-15T15:21:01.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.977+0000 7f82b49a0740 1 -- 172.21.15.105:0/1071192453 shutdown_connections 2024-04-15T15:21:01.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.977+0000 7f82b49a0740 1 --2- 172.21.15.105:0/1071192453 >> [v2:172.21.15.105:6835/958066041,v1:172.21.15.105:6837/958066041] conn(0x55c337728670 0x55c3376b9210 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:01.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.977+0000 7f82b49a0740 1 --2- 172.21.15.105:0/1071192453 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c3376e7260 0x55c3376b8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:01.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.977+0000 7f82b49a0740 1 --2- 172.21.15.105:0/1071192453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3376e6e80 0x55c3376eff40 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:01.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.977+0000 7f828a7fc640 0 client.5670 ms_handle_reset on v2:172.21.15.105:6835/958066041 2024-04-15T15:21:01.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.977+0000 7f82b49a0740 1 --2- 172.21.15.105:0/1071192453 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55c3376e4b70 0x55c3376efa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:01.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.977+0000 7f82b49a0740 1 -- 172.21.15.105:0/1071192453 >> 172.21.15.105:0/1071192453 conn(0x55c3376ba360 msgr2=0x55c3377046e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:01.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.977+0000 7f82b49a0740 1 -- 172.21.15.105:0/1071192453 shutdown_connections 2024-04-15T15:21:01.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.977+0000 7f82b49a0740 1 -- 172.21.15.105:0/1071192453 wait complete. 2024-04-15T15:21:01.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.977+0000 7f82b49a0740 20 client.5670 trim_cache size 0 max 16384 2024-04-15T15:21:01.982 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-04-15T15:21:01.982 INFO:teuthology.orchestra.run.smithi105.stdout: "op": { 2024-04-15T15:21:01.982 INFO:teuthology.orchestra.run.smithi105.stdout: "result": 0, 2024-04-15T15:21:01.982 INFO:teuthology.orchestra.run.smithi105.stdout: "flag_point": "quiesce complete", 2024-04-15T15:21:01.982 INFO:teuthology.orchestra.run.smithi105.stdout: "reqid": { 2024-04-15T15:21:01.982 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": { 2024-04-15T15:21:01.982 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "mds", 2024-04-15T15:21:01.982 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 0 2024-04-15T15:21:01.982 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-04-15T15:21:01.982 INFO:teuthology.orchestra.run.smithi105.stdout: "tid": 1 2024-04-15T15:21:01.982 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-04-15T15:21:01.983 INFO:teuthology.orchestra.run.smithi105.stdout: "op_type": "internal_op", 2024-04-15T15:21:01.983 INFO:teuthology.orchestra.run.smithi105.stdout: "internal_op": 5384, 2024-04-15T15:21:01.983 INFO:teuthology.orchestra.run.smithi105.stdout: "op_name": "quiesce_path", 2024-04-15T15:21:01.983 INFO:teuthology.orchestra.run.smithi105.stdout: "events": [ 2024-04-15T15:21:01.983 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-04-15T15:21:01.983 INFO:teuthology.orchestra.run.smithi105.stdout: "time": "2024-04-15T15:21:01.973697+0000", 2024-04-15T15:21:01.983 INFO:teuthology.orchestra.run.smithi105.stdout: "event": "initiated" 2024-04-15T15:21:01.983 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-04-15T15:21:01.983 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-04-15T15:21:01.983 INFO:teuthology.orchestra.run.smithi105.stdout: "time": "2024-04-15T15:21:01.973697+0000", 2024-04-15T15:21:01.983 INFO:teuthology.orchestra.run.smithi105.stdout: "event": "header_read" 2024-04-15T15:21:01.983 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-04-15T15:21:01.983 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-04-15T15:21:01.983 INFO:teuthology.orchestra.run.smithi105.stdout: "time": "2024-04-15T15:21:01.973697+0000", 2024-04-15T15:21:01.983 INFO:teuthology.orchestra.run.smithi105.stdout: "event": "throttled" 2024-04-15T15:21:01.984 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-04-15T15:21:01.984 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-04-15T15:21:01.984 INFO:teuthology.orchestra.run.smithi105.stdout: "time": "2024-04-15T15:21:01.973697+0000", 2024-04-15T15:21:01.984 INFO:teuthology.orchestra.run.smithi105.stdout: "event": "all_read" 2024-04-15T15:21:01.984 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-04-15T15:21:01.984 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-04-15T15:21:01.984 INFO:teuthology.orchestra.run.smithi105.stdout: "time": "2024-04-15T15:21:01.973697+0000", 2024-04-15T15:21:01.984 INFO:teuthology.orchestra.run.smithi105.stdout: "event": "dispatched" 2024-04-15T15:21:01.984 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-04-15T15:21:01.984 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-04-15T15:21:01.984 INFO:teuthology.orchestra.run.smithi105.stdout: "time": "2024-04-15T15:21:01.973765+0000", 2024-04-15T15:21:01.984 INFO:teuthology.orchestra.run.smithi105.stdout: "event": "acquired locks" 2024-04-15T15:21:01.984 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-04-15T15:21:01.984 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-04-15T15:21:01.985 INFO:teuthology.orchestra.run.smithi105.stdout: "time": "2024-04-15T15:21:01.973786+0000", 2024-04-15T15:21:01.985 INFO:teuthology.orchestra.run.smithi105.stdout: "event": "acquired locks" 2024-04-15T15:21:01.985 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-04-15T15:21:01.985 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-04-15T15:21:01.985 INFO:teuthology.orchestra.run.smithi105.stdout: "time": "2024-04-15T15:21:01.973806+0000", 2024-04-15T15:21:01.985 INFO:teuthology.orchestra.run.smithi105.stdout: "event": "acquired locks" 2024-04-15T15:21:01.985 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-04-15T15:21:01.985 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-04-15T15:21:01.985 INFO:teuthology.orchestra.run.smithi105.stdout: "time": "2024-04-15T15:21:01.973826+0000", 2024-04-15T15:21:01.985 INFO:teuthology.orchestra.run.smithi105.stdout: "event": "acquired locks" 2024-04-15T15:21:01.985 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-04-15T15:21:01.985 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-04-15T15:21:01.985 INFO:teuthology.orchestra.run.smithi105.stdout: "time": "2024-04-15T15:21:01.977655+0000", 2024-04-15T15:21:01.985 INFO:teuthology.orchestra.run.smithi105.stdout: "event": "acquired locks" 2024-04-15T15:21:01.985 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-04-15T15:21:01.986 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-04-15T15:21:01.986 INFO:teuthology.orchestra.run.smithi105.stdout: "time": "2024-04-15T15:21:01.977666+0000", 2024-04-15T15:21:01.986 INFO:teuthology.orchestra.run.smithi105.stdout: "event": "acquired locks" 2024-04-15T15:21:01.986 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-04-15T15:21:01.986 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-04-15T15:21:01.986 INFO:teuthology.orchestra.run.smithi105.stdout: "time": "2024-04-15T15:21:01.977675+0000", 2024-04-15T15:21:01.986 INFO:teuthology.orchestra.run.smithi105.stdout: "event": "acquired locks" 2024-04-15T15:21:01.986 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-04-15T15:21:01.986 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-04-15T15:21:01.986 INFO:teuthology.orchestra.run.smithi105.stdout: "time": "2024-04-15T15:21:01.977686+0000", 2024-04-15T15:21:01.986 INFO:teuthology.orchestra.run.smithi105.stdout: "event": "acquired locks" 2024-04-15T15:21:01.986 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-04-15T15:21:01.986 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-04-15T15:21:01.986 INFO:teuthology.orchestra.run.smithi105.stdout: "time": "2024-04-15T15:21:01.977697+0000", 2024-04-15T15:21:01.987 INFO:teuthology.orchestra.run.smithi105.stdout: "event": "quiesce complete" 2024-04-15T15:21:01.987 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-04-15T15:21:01.987 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-04-15T15:21:01.987 INFO:teuthology.orchestra.run.smithi105.stdout: "locks": null 2024-04-15T15:21:01.987 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-04-15T15:21:01.987 INFO:teuthology.orchestra.run.smithi105.stdout: "state": { 2024-04-15T15:21:01.987 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes": 2, 2024-04-15T15:21:01.987 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_quiesced": 1, 2024-04-15T15:21:01.987 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_blocked": 1, 2024-04-15T15:21:01.987 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_dropped": 0, 2024-04-15T15:21:01.987 INFO:teuthology.orchestra.run.smithi105.stdout: "failed": [] 2024-04-15T15:21:01.987 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-04-15T15:21:01.987 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-04-15T15:21:01.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.977+0000 7f82b1cd9640 1 -- 172.21.15.105:0/4152727552 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f828c06c980 msgr2=0x7f828c06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:01.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.977+0000 7f82b1cd9640 1 --2- 172.21.15.105:0/4152727552 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f828c06c980 0x7f828c06ee40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f829c001660 tx=0x7f829c009040 comp rx=0 tx=0).stop 2024-04-15T15:21:01.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.977+0000 7f82b1cd9640 1 -- 172.21.15.105:0/4152727552 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82ac0cf970 msgr2=0x7f82ac161250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:01.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.977+0000 7f82b1cd9640 1 --2- 172.21.15.105:0/4152727552 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82ac0cf970 0x7f82ac161250 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f8294000c00 tx=0x7f8294004730 comp rx=0 tx=0).stop 2024-04-15T15:21:01.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.977+0000 7f82b1cd9640 1 -- 172.21.15.105:0/4152727552 shutdown_connections 2024-04-15T15:21:01.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.977+0000 7f82b1cd9640 1 --2- 172.21.15.105:0/4152727552 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f828c06c980 0x7f828c06ee40 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:01.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.977+0000 7f82b1cd9640 1 --2- 172.21.15.105:0/4152727552 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82ac15e830 0x7f82ac1572f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:01.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.977+0000 7f82b1cd9640 1 --2- 172.21.15.105:0/4152727552 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82ac0cf970 0x7f82ac161250 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:01.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.977+0000 7f82b1cd9640 1 --2- 172.21.15.105:0/4152727552 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82ac0ba0f0 0x7f82ac0c7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:01.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.977+0000 7f82b1cd9640 1 -- 172.21.15.105:0/4152727552 >> 172.21.15.105:0/4152727552 conn(0x7f82ac06ccb0 msgr2=0x7f82ac0c8300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:01.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.978+0000 7f82b1cd9640 1 -- 172.21.15.105:0/4152727552 shutdown_connections 2024-04-15T15:21:01.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:01.978+0000 7f82b1cd9640 1 -- 172.21.15.105:0/4152727552 wait complete. 2024-04-15T15:21:01.990 DEBUG:tasks.cephfs.test_quiesce:{'op': {'result': 0, 'flag_point': 'quiesce complete', 'reqid': {'entity': {'type': 'mds', 'num': 0}, 'tid': 1}, 'op_type': 'internal_op', 'internal_op': 5384, 'op_name': 'quiesce_path', 'events': [{'time': '2024-04-15T15:21:01.973697+0000', 'event': 'initiated'}, {'time': '2024-04-15T15:21:01.973697+0000', 'event': 'header_read'}, {'time': '2024-04-15T15:21:01.973697+0000', 'event': 'throttled'}, {'time': '2024-04-15T15:21:01.973697+0000', 'event': 'all_read'}, {'time': '2024-04-15T15:21:01.973697+0000', 'event': 'dispatched'}, {'time': '2024-04-15T15:21:01.973765+0000', 'event': 'acquired locks'}, {'time': '2024-04-15T15:21:01.973786+0000', 'event': 'acquired locks'}, {'time': '2024-04-15T15:21:01.973806+0000', 'event': 'acquired locks'}, {'time': '2024-04-15T15:21:01.973826+0000', 'event': 'acquired locks'}, {'time': '2024-04-15T15:21:01.977655+0000', 'event': 'acquired locks'}, {'time': '2024-04-15T15:21:01.977666+0000', 'event': 'acquired locks'}, {'time': '2024-04-15T15:21:01.977675+0000', 'event': 'acquired locks'}, {'time': '2024-04-15T15:21:01.977686+0000', 'event': 'acquired locks'}, {'time': '2024-04-15T15:21:01.977697+0000', 'event': 'quiesce complete'}], 'locks': None}, 'state': {'inodes': 2, 'inodes_quiesced': 1, 'inodes_blocked': 1, 'inodes_dropped': 0, 'failed': []}} 2024-04-15T15:21:01.993 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-15T15:21:02.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.094+0000 7f057904a640 1 Processor -- start 2024-04-15T15:21:02.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.094+0000 7f057904a640 1 -- start start 2024-04-15T15:21:02.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.095+0000 7f057904a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f057413ec60 0x7f057406ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:02.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.095+0000 7f057904a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f057415bc90 con 0x7f057406f4d0 2024-04-15T15:21:02.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.095+0000 7f057904a640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f057415bf70 con 0x7f057413e680 2024-04-15T15:21:02.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.095+0000 7f057904a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f05740d0280 con 0x7f057413ec60 2024-04-15T15:21:02.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.095+0000 7f057259c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f057413ec60 0x7f057406ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:02.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.095+0000 7f057359e640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f057406f4d0 0x7f057415f1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:50770/0 (socket says 172.21.15.105:50770) 2024-04-15T15:21:02.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.095+0000 7f057259c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f057413ec60 0x7f057406ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46552/0 (socket says 172.21.15.105:46552) 2024-04-15T15:21:02.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.095+0000 7f057359e640 1 -- 172.21.15.105:0/2493593348 learned_addr learned my addr 172.21.15.105:0/2493593348 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:21:02.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.095+0000 7f057259c640 1 -- 172.21.15.105:0/2493593348 >> v1:172.21.15.154:6789/0 conn(0x7f057413e680 legacy=0x7f05740cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:02.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.095+0000 7f057259c640 1 -- 172.21.15.105:0/2493593348 >> v1:172.21.15.105:6789/0 conn(0x7f057406f4d0 legacy=0x7f057415f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:02.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.095+0000 7f0571d9b640 1 -- 172.21.15.105:0/2493593348 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3940400512 0 0) 0x7f057415bc90 con 0x7f057406f4d0 2024-04-15T15:21:02.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.095+0000 7f057259c640 1 -- 172.21.15.105:0/2493593348 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f057415c250 con 0x7f057413ec60 2024-04-15T15:21:02.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.096+0000 7f057259c640 1 --2- 172.21.15.105:0/2493593348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f057413ec60 0x7f057406ef90 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f0568009d90 tx=0x7f056802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c514bdcf1523257b server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:02.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.096+0000 7f0571d9b640 1 -- 172.21.15.105:0/2493593348 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0568039070 con 0x7f057413ec60 2024-04-15T15:21:02.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.096+0000 7f0571d9b640 1 -- 172.21.15.105:0/2493593348 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f056802cc60 con 0x7f057413ec60 2024-04-15T15:21:02.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.096+0000 7f0571d9b640 1 -- 172.21.15.105:0/2493593348 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0568034660 con 0x7f057413ec60 2024-04-15T15:21:02.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.096+0000 7f057904a640 1 -- 172.21.15.105:0/2493593348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f057413ec60 msgr2=0x7f057406ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:02.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.096+0000 7f057904a640 1 --2- 172.21.15.105:0/2493593348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f057413ec60 0x7f057406ef90 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f0568009d90 tx=0x7f056802beb0 comp rx=0 tx=0).stop 2024-04-15T15:21:02.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.096+0000 7f057904a640 1 -- 172.21.15.105:0/2493593348 shutdown_connections 2024-04-15T15:21:02.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.096+0000 7f057904a640 1 --2- 172.21.15.105:0/2493593348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f057413ec60 0x7f057406ef90 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:02.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.097+0000 7f057904a640 1 -- 172.21.15.105:0/2493593348 >> 172.21.15.105:0/2493593348 conn(0x7f0574067d50 msgr2=0x7f057406b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:02.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.097+0000 7f057904a640 1 -- 172.21.15.105:0/2493593348 shutdown_connections 2024-04-15T15:21:02.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.097+0000 7f057904a640 1 -- 172.21.15.105:0/2493593348 wait complete. 2024-04-15T15:21:02.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.098+0000 7f057904a640 1 Processor -- start 2024-04-15T15:21:02.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.098+0000 7f057904a640 1 -- start start 2024-04-15T15:21:02.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.099+0000 7f057904a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f057406f4d0 0x7f0574154900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:02.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.099+0000 7f057904a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f057413e680 0x7f057415a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:02.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.099+0000 7f057904a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f057413ec60 0x7f057415ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:02.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.099+0000 7f057904a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f057415c810 con 0x7f057413e680 2024-04-15T15:21:02.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.099+0000 7f057904a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f057406f900 con 0x7f057406f4d0 2024-04-15T15:21:02.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.099+0000 7f057904a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f05740cf590 con 0x7f057413ec60 2024-04-15T15:21:02.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.099+0000 7f057259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f057413e680 0x7f057415a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:02.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.099+0000 7f057259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f057413e680 0x7f057415a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46560/0 (socket says 172.21.15.105:46560) 2024-04-15T15:21:02.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.099+0000 7f057259c640 1 -- 172.21.15.105:0/3875889689 learned_addr learned my addr 172.21.15.105:0/3875889689 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:02.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.099+0000 7f057359e640 1 --2- 172.21.15.105:0/3875889689 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f057413ec60 0x7f057415ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:02.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.099+0000 7f0572d9d640 1 --2- 172.21.15.105:0/3875889689 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f057406f4d0 0x7f0574154900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:02.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.099+0000 7f057259c640 1 -- 172.21.15.105:0/3875889689 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f057413ec60 msgr2=0x7f057415ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:02.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.099+0000 7f057259c640 1 --2- 172.21.15.105:0/3875889689 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f057413ec60 0x7f057415ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:02.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.099+0000 7f057259c640 1 -- 172.21.15.105:0/3875889689 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f057406f4d0 msgr2=0x7f0574154900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:02.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.099+0000 7f057259c640 1 --2- 172.21.15.105:0/3875889689 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f057406f4d0 0x7f0574154900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:02.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.099+0000 7f057259c640 1 -- 172.21.15.105:0/3875889689 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0574154e40 con 0x7f057413e680 2024-04-15T15:21:02.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.099+0000 7f0572d9d640 1 --2- 172.21.15.105:0/3875889689 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f057406f4d0 0x7f0574154900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:21:02.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.099+0000 7f057259c640 1 --2- 172.21.15.105:0/3875889689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f057413e680 0x7f057415a5c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f0568009d90 tx=0x7f0568004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:02.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.099+0000 7f055b7fe640 1 -- 172.21.15.105:0/3875889689 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0568044020 con 0x7f057413e680 2024-04-15T15:21:02.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.099+0000 7f055b7fe640 1 -- 172.21.15.105:0/3875889689 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f056803f670 con 0x7f057413e680 2024-04-15T15:21:02.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.099+0000 7f057904a640 1 -- 172.21.15.105:0/3875889689 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0574143af0 con 0x7f057413e680 2024-04-15T15:21:02.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.099+0000 7f055b7fe640 1 -- 172.21.15.105:0/3875889689 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0568039040 con 0x7f057413e680 2024-04-15T15:21:02.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.100+0000 7f057904a640 1 -- 172.21.15.105:0/3875889689 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05741440b0 con 0x7f057413e680 2024-04-15T15:21:02.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.100+0000 7f057904a640 1 -- 172.21.15.105:0/3875889689 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0574068a40 con 0x7f057413e680 2024-04-15T15:21:02.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.101+0000 7f055b7fe640 1 -- 172.21.15.105:0/3875889689 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f0568047020 con 0x7f057413e680 2024-04-15T15:21:02.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.101+0000 7f055b7fe640 1 --2- 172.21.15.105:0/3875889689 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f056006c980 0x7f056006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:02.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.101+0000 7f055b7fe640 1 -- 172.21.15.105:0/3875889689 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f05680ae7b0 con 0x7f057413e680 2024-04-15T15:21:02.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.101+0000 7f0572d9d640 1 --2- 172.21.15.105:0/3875889689 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f056006c980 0x7f056006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:02.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.102+0000 7f0572d9d640 1 --2- 172.21.15.105:0/3875889689 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f056006c980 0x7f056006ee40 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0574155270 tx=0x7f056c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:02.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.106+0000 7f055b7fe640 1 -- 172.21.15.105:0/3875889689 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f056804e030 con 0x7f057413e680 2024-04-15T15:21:02.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.320+0000 7f057904a640 1 -- 172.21.15.105:0/3875889689 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f05740b3790 con 0x7f057413e680 2024-04-15T15:21:02.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.320+0000 7f055b7fe640 1 -- 172.21.15.105:0/3875889689 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+5180 (secure 0 0 0) 0x7f056807cc80 con 0x7f057413e680 2024-04-15T15:21:02.322 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:21:02.322 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4778,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/4848508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":4848508},{"type":"v1","addr":"172.21.15.154:6835","nonce":4848508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5005,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/1127070021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1127070021},{"type":"v1","addr":"172.21.15.154:6833","nonce":1127070021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5193,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6836/3825669416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3825669416},{"type":"v1","addr":"172.21.15.105:6836","nonce":3825669416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-15T15:20:29.000616+0000","modified":"2024-04-15T15:20:31.008997+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5196},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5196":{"gid":5196,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6837/958066041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":958066041},{"type":"v1","addr":"172.21.15.105:6837","nonce":958066041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5196,"qdb_cluster":[5196]},"id":2}]} 2024-04-15T15:21:02.323 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 14 2024-04-15T15:21:02.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.322+0000 7f057904a640 1 -- 172.21.15.105:0/3875889689 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f056006c980 msgr2=0x7f056006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:02.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.322+0000 7f057904a640 1 --2- 172.21.15.105:0/3875889689 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f056006c980 0x7f056006ee40 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0574155270 tx=0x7f056c009040 comp rx=0 tx=0).stop 2024-04-15T15:21:02.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.322+0000 7f057904a640 1 -- 172.21.15.105:0/3875889689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f057413e680 msgr2=0x7f057415a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:02.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.322+0000 7f057904a640 1 --2- 172.21.15.105:0/3875889689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f057413e680 0x7f057415a5c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f0568009d90 tx=0x7f0568004a40 comp rx=0 tx=0).stop 2024-04-15T15:21:02.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.322+0000 7f057904a640 1 -- 172.21.15.105:0/3875889689 shutdown_connections 2024-04-15T15:21:02.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.322+0000 7f057904a640 1 --2- 172.21.15.105:0/3875889689 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f057413ec60 0x7f057415ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:02.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.322+0000 7f057904a640 1 --2- 172.21.15.105:0/3875889689 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f056006c980 0x7f056006ee40 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:02.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.322+0000 7f057904a640 1 --2- 172.21.15.105:0/3875889689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f057413e680 0x7f057415a5c0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:02.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.322+0000 7f057904a640 1 --2- 172.21.15.105:0/3875889689 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f057406f4d0 0x7f0574154900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:02.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.322+0000 7f057904a640 1 -- 172.21.15.105:0/3875889689 >> 172.21.15.105:0/3875889689 conn(0x7f0574067d50 msgr2=0x7f057406b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:02.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.323+0000 7f057904a640 1 -- 172.21.15.105:0/3875889689 shutdown_connections 2024-04-15T15:21:02.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.323+0000 7f057904a640 1 -- 172.21.15.105:0/3875889689 wait complete. 2024-04-15T15:21:02.336 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json dump tree /volumes/_nogroup/subvol_quiesce/3b88d86e-6ee6-481f-8fa4-d97e9f88b65d 0 2024-04-15T15:21:02.455 INFO:teuthology.orchestra.run.smithi105.stdout:[{"ino":1099511627779,"rdev":0,"ctime":"2024-04-15T15:21:01.749878+0000","btime":"2024-04-15T15:20:42.757105+0000","mode":16895,"uid":0,"gid":0,"nlink":1,"dir_layout":{"dir_hash":2,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":4194304,"stripe_count":1,"object_size":4194304,"pool_id":5,"pool_ns":""},"old_pools":[],"size":0,"truncate_seq":1,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2024-04-15T15:21:01.749878+0000","atime":"2024-04-15T15:20:42.757105+0000","time_warp_seq":0,"change_attr":2,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"quiesce_block":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"2024-04-15T15:21:01.749878+0000","num_files":1,"num_subdirs":0,"change_attr":1},"rstat":{"version":0,"rbytes":0,"rfiles":1,"rsubdirs":1,"rsnaps":0,"rctime":"2024-04-15T15:21:01.782807+0000"},"accounted_rstat":{"version":0,"rbytes":0,"rfiles":1,"rsubdirs":1,"rsnaps":0,"rctime":"2024-04-15T15:21:01.782807+0000"},"version":20,"file_data_version":0,"xattr_version":1,"backtrace_version":2,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"0.000000","last_scrub_version":0,"symlink":"","xattrs":[],"dirfragtree":{"splits":[]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"request":1,"lock":2,"dirfrag":1,"caps":1,"dirtyparent":1,"dirty":1,"waiter":0,"authpin":1},"nref":8,"versionlock":{"gather_set":[],"state":"lock","type":"iversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":1,"num_xlocks":0,"xlock_by":{}},"authlock":{},"linklock":{},"dirfragtreelock":{},"filelock":{},"xattrlock":{},"snaplock":{},"nestlock":{"gather_set":[],"state":"lock","type":"inest","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"flocklock":{},"policylock":{},"quiescelock":{"gather_set":[],"state":"lock","type":"iquiesce","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":1,"xlock_by":{"reqid":{"entity":{"type":"mds","num":0},"tid":2}}},"states":["auth","dirty","dirtyparent","dirtypool"],"client_caps":[{"client_id":5003,"pending":"pAsLsXs","issued":"pAsLsXs","wanted":"-","last_sent":2},{"client_id":5021,"pending":"pAsLsXs","issued":"pAsLsXs","wanted":"-","last_sent":2},{"client_id":5460,"pending":"pAsLsXs","issued":"pAsLsXs","wanted":"-","last_sent":7},{"client_id":5619,"pending":"pAsLsXs","issued":"pAsLsXs","wanted":"-","last_sent":3},{"client_id":5640,"pending":"pAsLsXs","issued":"pAsLsXs","wanted":"-","last_sent":2}],"loner":-1,"want_loner":-1,"mds_caps_wanted":[],"dirfrags":[{"path":"/volumes/_nogroup/subvol_quiesce/3b88d86e-6ee6-481f-8fa4-d97e9f88b65d","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"7","version":"7","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"dirty":1,"authpin":0},"nref":2,"dentries":[{"path":"volumes/_nogroup/subvol_quiesce/3b88d86e-6ee6-481f-8fa4-d97e9f88b65d/file","path_ino":1,"snap_first":2,"snap_last":18446744073709551614,"is_primary":true,"is_remote":false,"is_null":false,"is_new":false,"inode":1099511629282,"remote_type":"","version":6,"projected_version":6,"auth_pins":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"lock":0,"inodepin":1,"dirty":1,"authpin":0},"nref":2,"lock":{},"versionlock":{"gather_set":[],"state":"lock","type":"dversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"]}]}],"is_subvolume":false}] 2024-04-15T15:21:02.456 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "accounted_rstat": { "rbytes": 0, "rctime": "2024-04-15T15:21:01.782807+0000", "rfiles": 1, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "atime": "2024-04-15T15:20:42.757105+0000", "auth_pins": 1, "auth_state": { "replicas": {} }, "authlock": {}, "backtrace_version": 2, "btime": "2024-04-15T15:20:42.757105+0000", "change_attr": 2, "client_caps": [ { "client_id": 5003, "issued": "pAsLsXs", "last_sent": 2, "pending": "pAsLsXs", "wanted": "-" }, { "client_id": 5021, "issued": "pAsLsXs", "last_sent": 2, "pending": "pAsLsXs", "wanted": "-" }, { "client_id": 5460, "issued": "pAsLsXs", "last_sent": 7, "pending": "pAsLsXs", "wanted": "-" }, { "client_id": 5619, "issued": "pAsLsXs", "last_sent": 3, "pending": "pAsLsXs", "wanted": "-" }, { "client_id": 5640, "issued": "pAsLsXs", "last_sent": 2, "pending": "pAsLsXs", "wanted": "-" } ], "client_ranges": [], "ctime": "2024-04-15T15:21:01.749878+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfrags": [ { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dentries": [ { "auth_pins": 0, "auth_state": { "replicas": {} }, "inode": 1099511629282, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_new": false, "is_null": false, "is_primary": true, "is_remote": false, "lock": {}, "nref": 2, "path": "volumes/_nogroup/subvol_quiesce/3b88d86e-6ee6-481f-8fa4-d97e9f88b65d/file", "path_ino": 1, "pins": { "authpin": 0, "dirty": 1, "inodepin": 1, "lock": 0, "request": 0 }, "projected_version": 6, "remote_type": "", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snap_first": 2, "snap_last": 18446744073709551614, "states": [ "auth", "dirty" ], "version": 6, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "dversion", "xlock_by": {} } } ], "dir_auth": "", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/volumes/_nogroup/subvol_quiesce/3b88d86e-6ee6-481f-8fa4-d97e9f88b65d", "pins": { "authpin": 0, "child": 1, "dirty": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" } ], "dirfragtree": { "splits": [] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 1, "mtime": "2024-04-15T15:21:01.749878+0000", "num_files": 1, "num_subdirs": 0, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": {}, "flocklock": {}, "gid": 0, "ino": 1099511627779, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_subvolume": false, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "layout": { "object_size": 4194304, "pool_id": 5, "pool_ns": "", "stripe_count": 1, "stripe_unit": 4194304 }, "linklock": {}, "loner": -1, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 16895, "mtime": "2024-04-15T15:21:01.749878+0000", "nestlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "inest", "xlock_by": {} }, "nlink": 1, "nref": 8, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "pins": { "authpin": 1, "caps": 1, "dirfrag": 1, "dirty": 1, "dirtyparent": 1, "lock": 2, "request": 1, "waiter": 0 }, "policylock": {}, "quiesce_block": false, "quiescelock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 1, "state": "lock", "type": "iquiesce", "xlock_by": { "reqid": { "entity": { "num": 0, "type": "mds" }, "tid": 2 } } }, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 0, "rctime": "2024-04-15T15:21:01.782807+0000", "rfiles": 1, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "size": 0, "snaplock": {}, "states": [ "auth", "dirty", "dirtyparent", "dirtypool" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "uid": 0, "version": 20, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 1, "num_xlocks": 0, "state": "lock", "type": "iversion", "xlock_by": {} }, "want_loner": -1, "xattr_version": 1, "xattrlock": {}, "xattrs": [] } ] 2024-04-15T15:21:02.456 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 ops --flags=locks --path=/tmp/mds.0-ops 2024-04-15T15:21:02.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.560+0000 7f80d4288640 1 Processor -- start 2024-04-15T15:21:02.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.560+0000 7f80d4288640 1 -- start start 2024-04-15T15:21:02.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.561+0000 7f80d4288640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f80cc0ba480 0x7f80cc0d1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:02.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.561+0000 7f80d4288640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80cc0d3400 con 0x7f80cc0d1650 2024-04-15T15:21:02.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.561+0000 7f80d4288640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80cc0d36e0 con 0x7f80cc15e8d0 2024-04-15T15:21:02.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.561+0000 7f80d4288640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f80cc0d31c0 con 0x7f80cc0ba480 2024-04-15T15:21:02.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.561+0000 7f80d1823640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f80cc0ba480 0x7f80cc0d1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:02.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.561+0000 7f80d2825640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f80cc0d1650 0x7f80cc0b7880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:50786/0 (socket says 172.21.15.105:50786) 2024-04-15T15:21:02.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.561+0000 7f80d2825640 1 -- 172.21.15.105:0/3709512542 learned_addr learned my addr 172.21.15.105:0/3709512542 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:21:02.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.562+0000 7f80d1823640 1 -- 172.21.15.105:0/3709512542 >> v1:172.21.15.154:6789/0 conn(0x7f80cc15e8d0 legacy=0x7f80cc0d2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:02.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.562+0000 7f80d1823640 1 -- 172.21.15.105:0/3709512542 >> v1:172.21.15.105:6789/0 conn(0x7f80cc0d1650 legacy=0x7f80cc0b7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:02.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.562+0000 7f80d1022640 1 -- 172.21.15.105:0/3709512542 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 50688120 0 0) 0x7f80cc0d3400 con 0x7f80cc0d1650 2024-04-15T15:21:02.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.562+0000 7f80d1823640 1 -- 172.21.15.105:0/3709512542 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80cc0d39c0 con 0x7f80cc0ba480 2024-04-15T15:21:02.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.562+0000 7f80d1823640 1 --2- 172.21.15.105:0/3709512542 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80cc0ba480 0x7f80cc0d1110 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f80bc0097d0 tx=0x7f80bc02b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c13727ce935383df server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:02.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.563+0000 7f80d1022640 1 -- 172.21.15.105:0/3709512542 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80bc034070 con 0x7f80cc0ba480 2024-04-15T15:21:02.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.563+0000 7f80d1022640 1 -- 172.21.15.105:0/3709512542 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80bc030890 con 0x7f80cc0ba480 2024-04-15T15:21:02.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.563+0000 7f80d1022640 1 -- 172.21.15.105:0/3709512542 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80bc030c60 con 0x7f80cc0ba480 2024-04-15T15:21:02.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.563+0000 7f80d4288640 1 -- 172.21.15.105:0/3709512542 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80cc0ba480 msgr2=0x7f80cc0d1110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:02.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.563+0000 7f80d4288640 1 --2- 172.21.15.105:0/3709512542 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80cc0ba480 0x7f80cc0d1110 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f80bc0097d0 tx=0x7f80bc02b9d0 comp rx=0 tx=0).stop 2024-04-15T15:21:02.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.563+0000 7f80d4288640 1 -- 172.21.15.105:0/3709512542 shutdown_connections 2024-04-15T15:21:02.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.563+0000 7f80d4288640 1 --2- 172.21.15.105:0/3709512542 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80cc0ba480 0x7f80cc0d1110 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:02.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.563+0000 7f80d4288640 1 -- 172.21.15.105:0/3709512542 >> 172.21.15.105:0/3709512542 conn(0x7f80cc06d460 msgr2=0x7f80cc067720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:02.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.563+0000 7f80d4288640 1 -- 172.21.15.105:0/3709512542 shutdown_connections 2024-04-15T15:21:02.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.563+0000 7f80d4288640 1 -- 172.21.15.105:0/3709512542 wait complete. 2024-04-15T15:21:02.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.564+0000 7f80d4288640 1 Processor -- start 2024-04-15T15:21:02.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.564+0000 7f80d4288640 1 -- start start 2024-04-15T15:21:02.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.565+0000 7f80d4288640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80cc0ba480 0x7f80cc0b82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:02.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.565+0000 7f80d4288640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80cc0d1650 0x7f80cc0b8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:02.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.565+0000 7f80d4288640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80cc15e8d0 0x7f80cc1565c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:02.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.565+0000 7f80d4288640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f80cc0d44a0 con 0x7f80cc15e8d0 2024-04-15T15:21:02.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.565+0000 7f80d4288640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f80cc0d3f80 con 0x7f80cc0d1650 2024-04-15T15:21:02.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.565+0000 7f80d4288640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f80cc15ecb0 con 0x7f80cc0ba480 2024-04-15T15:21:02.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.565+0000 7f80d2825640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80cc15e8d0 0x7f80cc1565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:02.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.565+0000 7f80d2825640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80cc15e8d0 0x7f80cc1565c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46572/0 (socket says 172.21.15.105:46572) 2024-04-15T15:21:02.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.565+0000 7f80d2825640 1 -- 172.21.15.105:0/3139632777 learned_addr learned my addr 172.21.15.105:0/3139632777 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:02.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.565+0000 7f80d1823640 1 --2- 172.21.15.105:0/3139632777 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80cc0d1650 0x7f80cc0b8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:02.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.565+0000 7f80d2825640 1 -- 172.21.15.105:0/3139632777 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80cc0ba480 msgr2=0x7f80cc0b82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:02.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.565+0000 7f80d2024640 1 --2- 172.21.15.105:0/3139632777 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80cc0ba480 0x7f80cc0b82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:02.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.566+0000 7f80d2825640 1 --2- 172.21.15.105:0/3139632777 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80cc0ba480 0x7f80cc0b82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:02.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.566+0000 7f80d2825640 1 -- 172.21.15.105:0/3139632777 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80cc0d1650 msgr2=0x7f80cc0b8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:02.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.566+0000 7f80d2825640 1 --2- 172.21.15.105:0/3139632777 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80cc0d1650 0x7f80cc0b8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:02.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.566+0000 7f80d2825640 1 -- 172.21.15.105:0/3139632777 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80cc157340 con 0x7f80cc15e8d0 2024-04-15T15:21:02.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.566+0000 7f80d1823640 1 --2- 172.21.15.105:0/3139632777 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80cc0d1650 0x7f80cc0b8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:21:02.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.566+0000 7f80d2024640 1 --2- 172.21.15.105:0/3139632777 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80cc0ba480 0x7f80cc0b82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:21:02.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.566+0000 7f80d2825640 1 --2- 172.21.15.105:0/3139632777 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80cc15e8d0 0x7f80cc1565c0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f80c400c7a0 tx=0x7f80c400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:02.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.566+0000 7f80c2ffd640 1 -- 172.21.15.105:0/3139632777 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c4018070 con 0x7f80cc15e8d0 2024-04-15T15:21:02.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.566+0000 7f80c2ffd640 1 -- 172.21.15.105:0/3139632777 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80c400a640 con 0x7f80cc15e8d0 2024-04-15T15:21:02.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.566+0000 7f80d4288640 1 -- 172.21.15.105:0/3139632777 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80cc157620 con 0x7f80cc15e8d0 2024-04-15T15:21:02.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.566+0000 7f80c2ffd640 1 -- 172.21.15.105:0/3139632777 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c400aab0 con 0x7f80cc15e8d0 2024-04-15T15:21:02.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.566+0000 7f80d4288640 1 -- 172.21.15.105:0/3139632777 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80cc157be0 con 0x7f80cc15e8d0 2024-04-15T15:21:02.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.567+0000 7f80c2ffd640 1 -- 172.21.15.105:0/3139632777 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f80c401b070 con 0x7f80cc15e8d0 2024-04-15T15:21:02.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.568+0000 7f80c2ffd640 1 --2- 172.21.15.105:0/3139632777 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f80b006c980 0x7f80b006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:02.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.568+0000 7f80d2024640 1 --2- 172.21.15.105:0/3139632777 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f80b006c980 0x7f80b006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:02.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.568+0000 7f80c2ffd640 1 -- 172.21.15.105:0/3139632777 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f80c408b1f0 con 0x7f80cc15e8d0 2024-04-15T15:21:02.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.568+0000 7f80d2024640 1 --2- 172.21.15.105:0/3139632777 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f80b006c980 0x7f80b006ee40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f80cc15e340 tx=0x7f80c8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:02.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.569+0000 7f80d6e76740 1 Processor -- start 2024-04-15T15:21:02.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.569+0000 7f80d6e76740 1 -- start start 2024-04-15T15:21:02.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.570+0000 7f80d6e76740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e1f6583da0 0x55e1f6584180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:02.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.570+0000 7f80d6e76740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55e1f6587b60 0x55e1f6589600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:02.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.570+0000 7f80d6e76740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1f65b38a0 0x55e1f65b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:02.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.570+0000 7f80d6e76740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55e1f64eea80 con 0x55e1f65b38a0 2024-04-15T15:21:02.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.570+0000 7f80d6e76740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55e1f6495580 con 0x55e1f6583da0 2024-04-15T15:21:02.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.570+0000 7f80d6e76740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x55e1f64eaa40 con 0x55e1f6587b60 2024-04-15T15:21:02.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.570+0000 7f80d2825640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1f65b38a0 0x55e1f65b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:02.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.570+0000 7f80d2825640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1f65b38a0 0x55e1f65b3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46580/0 (socket says 172.21.15.105:46580) 2024-04-15T15:21:02.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.570+0000 7f80d2825640 1 -- 172.21.15.105:0/1093247722 learned_addr learned my addr 172.21.15.105:0/1093247722 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:02.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.570+0000 7f80d2024640 1 --2- 172.21.15.105:0/1093247722 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e1f6583da0 0x55e1f6584180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:02.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.570+0000 7f80d1823640 1 --2- 172.21.15.105:0/1093247722 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55e1f6587b60 0x55e1f6589600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:02.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.570+0000 7f80d2825640 1 -- 172.21.15.105:0/1093247722 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55e1f6587b60 msgr2=0x55e1f6589600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:02.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.570+0000 7f80d2825640 1 --2- 172.21.15.105:0/1093247722 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55e1f6587b60 0x55e1f6589600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:02.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.570+0000 7f80d2825640 1 -- 172.21.15.105:0/1093247722 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e1f6583da0 msgr2=0x55e1f6584180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:02.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.570+0000 7f80d2825640 1 --2- 172.21.15.105:0/1093247722 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e1f6583da0 0x55e1f6584180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:02.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.570+0000 7f80d2825640 1 -- 172.21.15.105:0/1093247722 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1f64a9ad0 con 0x55e1f65b38a0 2024-04-15T15:21:02.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.570+0000 7f80d2024640 1 --2- 172.21.15.105:0/1093247722 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e1f6583da0 0x55e1f6584180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:21:02.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.570+0000 7f80d2825640 1 --2- 172.21.15.105:0/1093247722 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1f65b38a0 0x55e1f65b3d20 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f80c400c770 tx=0x7f80c4058e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f3f7b492471b875 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:02.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.570+0000 7f80affff640 1 -- 172.21.15.105:0/1093247722 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c4061b10 con 0x55e1f65b38a0 2024-04-15T15:21:02.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.571+0000 7f80affff640 1 -- 172.21.15.105:0/1093247722 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80c40555a0 con 0x55e1f65b38a0 2024-04-15T15:21:02.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.571+0000 7f80affff640 1 -- 172.21.15.105:0/1093247722 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c400ae50 con 0x55e1f65b38a0 2024-04-15T15:21:02.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.571+0000 7f80d6e76740 1 -- 172.21.15.105:0/1093247722 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1f65b38a0 msgr2=0x55e1f65b3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:02.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.571+0000 7f80d6e76740 1 --2- 172.21.15.105:0/1093247722 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1f65b38a0 0x55e1f65b3d20 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f80c400c770 tx=0x7f80c4058e30 comp rx=0 tx=0).stop 2024-04-15T15:21:02.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.571+0000 7f80d6e76740 1 -- 172.21.15.105:0/1093247722 shutdown_connections 2024-04-15T15:21:02.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.571+0000 7f80d6e76740 1 --2- 172.21.15.105:0/1093247722 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55e1f6587b60 0x55e1f6589600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:02.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.571+0000 7f80d6e76740 1 --2- 172.21.15.105:0/1093247722 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1f65b38a0 0x55e1f65b3d20 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:02.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.571+0000 7f80d6e76740 1 --2- 172.21.15.105:0/1093247722 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e1f6583da0 0x55e1f6584180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:02.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.571+0000 7f80d6e76740 1 -- 172.21.15.105:0/1093247722 >> 172.21.15.105:0/1093247722 conn(0x55e1f6585e20 msgr2=0x55e1f6587450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:02.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.571+0000 7f80d6e76740 1 -- 172.21.15.105:0/1093247722 shutdown_connections 2024-04-15T15:21:02.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.571+0000 7f80d6e76740 1 -- 172.21.15.105:0/1093247722 wait complete. 2024-04-15T15:21:02.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.572+0000 7f80d6e76740 1 -- start start 2024-04-15T15:21:02.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.572+0000 7f80d6e76740 1 Processor -- start 2024-04-15T15:21:02.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.573+0000 7f80d6e76740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e1f65b38a0 0x55e1f65b7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:02.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.573+0000 7f80d6e76740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1f65d6b90 0x55e1f65afcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:02.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.573+0000 7f80d6e76740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55e1f65b7740 0x55e1f65b01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:02.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.573+0000 7f80d6e76740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55e1f6587ff0 con 0x55e1f65d6b90 2024-04-15T15:21:02.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.573+0000 7f80d6e76740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55e1f64a8940 con 0x55e1f65b38a0 2024-04-15T15:21:02.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.573+0000 7f80d6e76740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x55e1f65882d0 con 0x55e1f65b7740 2024-04-15T15:21:02.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.573+0000 7f80d6e76740 10 client.0 _ll_register_callbacks cb 0x55e1f64a8350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-15T15:21:02.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.573+0000 7f80d1823640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1f65d6b90 0x55e1f65afcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:02.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.573+0000 7f80d1823640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1f65d6b90 0x55e1f65afcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46588/0 (socket says 172.21.15.105:46588) 2024-04-15T15:21:02.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.573+0000 7f80d1823640 1 -- 172.21.15.105:0/4147991081 learned_addr learned my addr 172.21.15.105:0/4147991081 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:02.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.573+0000 7f80d2825640 1 --2- 172.21.15.105:0/4147991081 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55e1f65b7740 0x55e1f65b01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:02.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.573+0000 7f80d2024640 1 --2- 172.21.15.105:0/4147991081 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e1f65b38a0 0x55e1f65b7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:02.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.573+0000 7f80d1823640 1 -- 172.21.15.105:0/4147991081 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55e1f65b7740 msgr2=0x55e1f65b01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:02.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.573+0000 7f80d1823640 1 --2- 172.21.15.105:0/4147991081 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55e1f65b7740 0x55e1f65b01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:02.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.573+0000 7f80d1823640 1 -- 172.21.15.105:0/4147991081 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e1f65b38a0 msgr2=0x55e1f65b7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:02.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.573+0000 7f80d1823640 1 --2- 172.21.15.105:0/4147991081 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e1f65b38a0 0x55e1f65b7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:02.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.573+0000 7f80d1823640 1 -- 172.21.15.105:0/4147991081 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1f6201420 con 0x55e1f65d6b90 2024-04-15T15:21:02.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.573+0000 7f80d1823640 1 --2- 172.21.15.105:0/4147991081 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1f65d6b90 0x55e1f65afcb0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f80bc02bfe0 tx=0x7f80bc02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=957c3dd9c992d3e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:02.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.573+0000 7f80ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-04-15T15:21:02.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.573+0000 7f80ad7fa640 1 -- 172.21.15.105:0/4147991081 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80bc034040 con 0x55e1f65d6b90 2024-04-15T15:21:02.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.574+0000 7f80ad7fa640 1 -- 172.21.15.105:0/4147991081 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f80bc004440 con 0x55e1f65d6b90 2024-04-15T15:21:02.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.574+0000 7f80d6e76740 1 -- 172.21.15.105:0/4147991081 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80c40555a0 con 0x55e1f65d6b90 2024-04-15T15:21:02.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.574+0000 7f80ad7fa640 1 -- 172.21.15.105:0/4147991081 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) ==== 24+0+0 (secure 0 0 0) 0x7f80bc050050 con 0x55e1f65d6b90 2024-04-15T15:21:02.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.574+0000 7f80d6e76740 10 client.5694 fetch_fsmap learned FSMap version 14 2024-04-15T15:21:02.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.574+0000 7f80d6e76740 1 -- 172.21.15.105:0/4147991081 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) -- 0x55e1f62cd830 con 0x55e1f65d6b90 2024-04-15T15:21:02.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.574+0000 7f80ad7fa640 1 -- 172.21.15.105:0/4147991081 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80c40555a0 con 0x55e1f65d6b90 2024-04-15T15:21:02.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.574+0000 7f80ad7fa640 1 -- 172.21.15.105:0/4147991081 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 14) ==== 3494+0+0 (secure 0 0 0) 0x7f80bc053020 con 0x55e1f65d6b90 2024-04-15T15:21:02.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.574+0000 7f80d6e76740 10 client.5694 fetch_fsmap finished waiting for FSMap version 14 2024-04-15T15:21:02.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.574+0000 7f80d6e76740 10 client.5694 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-15T15:21:02.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.574+0000 7f80d6e76740 20 client.5694 populate_metadata read hostname 'smithi105' 2024-04-15T15:21:02.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.574+0000 7f80d6e76740 1 --2- 172.21.15.105:0/4147991081 >> [v2:172.21.15.105:6835/958066041,v1:172.21.15.105:6837/958066041] conn(0x55e1f65cc920 0x55e1f65ccd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:02.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.574+0000 7f80d6e76740 4 client.5694 mds_command: new command op to 5196 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-15T15:21:02.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.574+0000 7f80d6e76740 1 -- 172.21.15.105:0/4147991081 --> [v2:172.21.15.105:6835/958066041,v1:172.21.15.105:6837/958066041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e1f64fc4a0 con 0x55e1f65cc920 2024-04-15T15:21:02.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.574+0000 7f80d2024640 1 --2- 172.21.15.105:0/4147991081 >> [v2:172.21.15.105:6835/958066041,v1:172.21.15.105:6837/958066041] conn(0x55e1f65cc920 0x55e1f65ccd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:02.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.574+0000 7f80ad7fa640 10 client.5694 ms_handle_connect on v2:172.21.15.105:6835/958066041 2024-04-15T15:21:02.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.574+0000 7f80d2024640 1 --2- 172.21.15.105:0/4147991081 >> [v2:172.21.15.105:6835/958066041,v1:172.21.15.105:6837/958066041] conn(0x55e1f65cc920 0x55e1f65ccd40 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a1a9e88aad351a0 server_cookie=97d371459617b5de in_seq=0 out_seq=0 2024-04-15T15:21:02.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.575+0000 7f80ad7fa640 1 -- 172.21.15.105:0/4147991081 <== mds.0 v2:172.21.15.105:6835/958066041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e1f64fc4a0 con 0x55e1f65cc920 2024-04-15T15:21:02.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.575+0000 7f80ad7fa640 10 client.5694 handle_command_reply: tid=0 2024-04-15T15:21:02.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.575+0000 7f80d6e76740 1 client.5694 shutdown 2024-04-15T15:21:02.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.575+0000 7f80d6e76740 1 -- 172.21.15.105:0/4147991081 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1f65d6b90 msgr2=0x55e1f65afcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:02.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.575+0000 7f80d6e76740 1 --2- 172.21.15.105:0/4147991081 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1f65d6b90 0x55e1f65afcb0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f80bc02bfe0 tx=0x7f80bc02e000 comp rx=0 tx=0).stop 2024-04-15T15:21:02.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.575+0000 7f80d6e76740 1 -- 172.21.15.105:0/4147991081 shutdown_connections 2024-04-15T15:21:02.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.575+0000 7f80d6e76740 1 --2- 172.21.15.105:0/4147991081 >> [v2:172.21.15.105:6835/958066041,v1:172.21.15.105:6837/958066041] conn(0x55e1f65cc920 0x55e1f65ccd40 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:02.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.575+0000 7f80d6e76740 1 --2- 172.21.15.105:0/4147991081 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55e1f65b7740 0x55e1f65b01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:02.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.575+0000 7f80d6e76740 1 --2- 172.21.15.105:0/4147991081 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1f65d6b90 0x55e1f65afcb0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:02.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.575+0000 7f80d6e76740 1 --2- 172.21.15.105:0/4147991081 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e1f65b38a0 0x55e1f65b7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:02.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.575+0000 7f80ad7fa640 0 client.5694 ms_handle_reset on v2:172.21.15.105:6835/958066041 2024-04-15T15:21:02.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.575+0000 7f80d6e76740 1 -- 172.21.15.105:0/4147991081 >> 172.21.15.105:0/4147991081 conn(0x55e1f6583da0 msgr2=0x55e1f65b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:02.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.575+0000 7f80d6e76740 1 -- 172.21.15.105:0/4147991081 shutdown_connections 2024-04-15T15:21:02.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.575+0000 7f80d6e76740 1 -- 172.21.15.105:0/4147991081 wait complete. 2024-04-15T15:21:02.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.575+0000 7f80d6e76740 20 client.5694 trim_cache size 0 max 16384 2024-04-15T15:21:02.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.591+0000 7f80d6e76740 1 Processor -- start 2024-04-15T15:21:02.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.591+0000 7f80d6e76740 1 -- start start 2024-04-15T15:21:02.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.592+0000 7f80d6e76740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55e1f65b38a0 0x55e1f65b1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:02.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.592+0000 7f80d6e76740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e1f65b7740 0x55e1f65b1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:02.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.592+0000 7f80d6e76740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1f65d6b90 0x55e1f65b3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:02.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.592+0000 7f80d6e76740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f80cc200e30 con 0x55e1f65d6b90 2024-04-15T15:21:02.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.592+0000 7f80d6e76740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f80cc1ffe50 con 0x55e1f65b7740 2024-04-15T15:21:02.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.592+0000 7f80d6e76740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f80cc1f6300 con 0x55e1f65b38a0 2024-04-15T15:21:02.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.592+0000 7f80d2825640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1f65d6b90 0x55e1f65b3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:02.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.592+0000 7f80d2825640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1f65d6b90 0x55e1f65b3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46602/0 (socket says 172.21.15.105:46602) 2024-04-15T15:21:02.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.592+0000 7f80d2825640 1 -- 172.21.15.105:0/3680878405 learned_addr learned my addr 172.21.15.105:0/3680878405 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:02.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.592+0000 7f80d2024640 1 --2- 172.21.15.105:0/3680878405 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55e1f65b38a0 0x55e1f65b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:02.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.592+0000 7f80d1823640 1 --2- 172.21.15.105:0/3680878405 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e1f65b7740 0x55e1f65b1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:02.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.592+0000 7f80d2825640 1 -- 172.21.15.105:0/3680878405 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55e1f65b38a0 msgr2=0x55e1f65b1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:02.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.592+0000 7f80d2825640 1 --2- 172.21.15.105:0/3680878405 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55e1f65b38a0 0x55e1f65b1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:02.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.592+0000 7f80d2825640 1 -- 172.21.15.105:0/3680878405 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e1f65b7740 msgr2=0x55e1f65b1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:02.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.592+0000 7f80d2825640 1 --2- 172.21.15.105:0/3680878405 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e1f65b7740 0x55e1f65b1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:02.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.592+0000 7f80d2825640 1 -- 172.21.15.105:0/3680878405 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1f657f3f0 con 0x55e1f65d6b90 2024-04-15T15:21:02.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.592+0000 7f80d2825640 1 --2- 172.21.15.105:0/3680878405 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1f65d6b90 0x55e1f65b3c80 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f80c40897c0 tx=0x7f80c4009070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6fb80a9978dbcf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:02.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.593+0000 7f80af7fe640 1 -- 172.21.15.105:0/3680878405 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c4076070 con 0x55e1f65d6b90 2024-04-15T15:21:02.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.593+0000 7f80af7fe640 1 -- 172.21.15.105:0/3680878405 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80c4087040 con 0x55e1f65d6b90 2024-04-15T15:21:02.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.593+0000 7f80af7fe640 1 -- 172.21.15.105:0/3680878405 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c405d2e0 con 0x55e1f65d6b90 2024-04-15T15:21:02.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.593+0000 7f80d6e76740 1 -- 172.21.15.105:0/3680878405 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1f65d6b90 msgr2=0x55e1f65b3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:02.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.593+0000 7f80d6e76740 1 --2- 172.21.15.105:0/3680878405 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1f65d6b90 0x55e1f65b3c80 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f80c40897c0 tx=0x7f80c4009070 comp rx=0 tx=0).stop 2024-04-15T15:21:02.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.593+0000 7f80d6e76740 1 -- 172.21.15.105:0/3680878405 shutdown_connections 2024-04-15T15:21:02.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.593+0000 7f80d6e76740 1 --2- 172.21.15.105:0/3680878405 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1f65d6b90 0x55e1f65b3c80 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:02.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.593+0000 7f80d6e76740 1 --2- 172.21.15.105:0/3680878405 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e1f65b7740 0x55e1f65b1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:02.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.593+0000 7f80d6e76740 1 --2- 172.21.15.105:0/3680878405 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55e1f65b38a0 0x55e1f65b1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:02.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.593+0000 7f80d6e76740 1 -- 172.21.15.105:0/3680878405 >> 172.21.15.105:0/3680878405 conn(0x55e1f6583da0 msgr2=0x55e1f6584560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:02.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.593+0000 7f80d6e76740 1 -- 172.21.15.105:0/3680878405 shutdown_connections 2024-04-15T15:21:02.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.593+0000 7f80d6e76740 1 -- 172.21.15.105:0/3680878405 wait complete. 2024-04-15T15:21:02.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.594+0000 7f80d6e76740 1 -- start start 2024-04-15T15:21:02.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.594+0000 7f80d6e76740 1 Processor -- start 2024-04-15T15:21:02.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.595+0000 7f80d6e76740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55e1f65b38a0 0x55e1f6648de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:02.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.595+0000 7f80d6e76740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1f65b7740 0x55e1f65b4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:02.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.595+0000 7f80d6e76740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e1f65d6b90 0x55e1f6582980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:02.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.595+0000 7f80d6e76740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f80cc1f6950 con 0x55e1f65b7740 2024-04-15T15:21:02.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.595+0000 7f80d6e76740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55e1f65cd0b0 con 0x55e1f65d6b90 2024-04-15T15:21:02.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.595+0000 7f80d6e76740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x55e1f65de190 con 0x55e1f65b38a0 2024-04-15T15:21:02.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.595+0000 7f80d6e76740 10 client.0 _ll_register_callbacks cb 0x7f80bc030b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-15T15:21:02.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.595+0000 7f80d1823640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1f65b7740 0x55e1f65b4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:02.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.595+0000 7f80d1823640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1f65b7740 0x55e1f65b4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46616/0 (socket says 172.21.15.105:46616) 2024-04-15T15:21:02.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.595+0000 7f80d1823640 1 -- 172.21.15.105:0/1893066436 learned_addr learned my addr 172.21.15.105:0/1893066436 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:02.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.595+0000 7f80d1823640 1 -- 172.21.15.105:0/1893066436 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55e1f65b38a0 msgr2=0x55e1f6648de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:02.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.595+0000 7f80d1823640 1 --2- 172.21.15.105:0/1893066436 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55e1f65b38a0 0x55e1f6648de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:02.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.595+0000 7f80d1823640 1 -- 172.21.15.105:0/1893066436 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e1f65d6b90 msgr2=0x55e1f6582980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:02.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.595+0000 7f80d2825640 1 --2- 172.21.15.105:0/1893066436 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e1f65d6b90 0x55e1f6582980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:02.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.595+0000 7f80d1823640 1 --2- 172.21.15.105:0/1893066436 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e1f65d6b90 0x55e1f6582980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:02.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.595+0000 7f80d1823640 1 -- 172.21.15.105:0/1893066436 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1f65b0480 con 0x55e1f65b7740 2024-04-15T15:21:02.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.595+0000 7f80d1823640 1 --2- 172.21.15.105:0/1893066436 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1f65b7740 0x55e1f65b4550 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f80bc006f80 tx=0x7f80bc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b25988c15f068cd server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:02.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.595+0000 7f80ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-04-15T15:21:02.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.595+0000 7f80ad7fa640 1 -- 172.21.15.105:0/1893066436 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80bc034040 con 0x55e1f65b7740 2024-04-15T15:21:02.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.595+0000 7f80ad7fa640 1 -- 172.21.15.105:0/1893066436 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f80bc005da0 con 0x55e1f65b7740 2024-04-15T15:21:02.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.595+0000 7f80d6e76740 1 -- 172.21.15.105:0/1893066436 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80c4087040 con 0x55e1f65b7740 2024-04-15T15:21:02.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.596+0000 7f80ad7fa640 1 -- 172.21.15.105:0/1893066436 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) ==== 24+0+0 (secure 0 0 0) 0x7f80bc065050 con 0x55e1f65b7740 2024-04-15T15:21:02.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.596+0000 7f80d6e76740 10 client.5700 fetch_fsmap learned FSMap version 14 2024-04-15T15:21:02.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.596+0000 7f80d6e76740 1 -- 172.21.15.105:0/1893066436 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) -- 0x55e1f6649320 con 0x55e1f65b7740 2024-04-15T15:21:02.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.596+0000 7f80ad7fa640 1 -- 172.21.15.105:0/1893066436 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80c4087040 con 0x55e1f65b7740 2024-04-15T15:21:02.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.596+0000 7f80ad7fa640 1 -- 172.21.15.105:0/1893066436 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 14) ==== 3494+0+0 (secure 0 0 0) 0x7f80bc068020 con 0x55e1f65b7740 2024-04-15T15:21:02.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.596+0000 7f80d6e76740 10 client.5700 fetch_fsmap finished waiting for FSMap version 14 2024-04-15T15:21:02.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.596+0000 7f80d6e76740 10 client.5700 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-15T15:21:02.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.596+0000 7f80d6e76740 20 client.5700 populate_metadata read hostname 'smithi105' 2024-04-15T15:21:02.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.596+0000 7f80d6e76740 1 --2- 172.21.15.105:0/1893066436 >> [v2:172.21.15.105:6835/958066041,v1:172.21.15.105:6837/958066041] conn(0x55e1f65b4a90 0x55e1f65d54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:02.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.596+0000 7f80d6e76740 4 client.5700 mds_command: new command op to 5196 tid=0 multi_id=0 [{"prefix": "ops", "flags": ["locks"], "path": "/tmp/mds.0-ops"}] 2024-04-15T15:21:02.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.596+0000 7f80d6e76740 1 -- 172.21.15.105:0/1893066436 --> [v2:172.21.15.105:6835/958066041,v1:172.21.15.105:6837/958066041] -- command(tid 0: {"prefix": "ops", "flags": ["locks"], "path": "/tmp/mds.0-ops"}) -- 0x55e1f6586580 con 0x55e1f65b4a90 2024-04-15T15:21:02.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.596+0000 7f80d2024640 1 --2- 172.21.15.105:0/1893066436 >> [v2:172.21.15.105:6835/958066041,v1:172.21.15.105:6837/958066041] conn(0x55e1f65b4a90 0x55e1f65d54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:02.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.596+0000 7f80ad7fa640 10 client.5700 ms_handle_connect on v2:172.21.15.105:6835/958066041 2024-04-15T15:21:02.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.596+0000 7f80d2024640 1 --2- 172.21.15.105:0/1893066436 >> [v2:172.21.15.105:6835/958066041,v1:172.21.15.105:6837/958066041] conn(0x55e1f65b4a90 0x55e1f65d54f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d77e4e116c78f317 server_cookie=d03f8523b3810cdc in_seq=0 out_seq=0 2024-04-15T15:21:02.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.597+0000 7f80ad7fa640 1 -- 172.21.15.105:0/1893066436 <== mds.0 v2:172.21.15.105:6835/958066041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+33 (crc 0 0 0) 0x55e1f6586580 con 0x55e1f65b4a90 2024-04-15T15:21:02.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.597+0000 7f80ad7fa640 10 client.5700 handle_command_reply: tid=0 2024-04-15T15:21:02.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.597+0000 7f80d6e76740 1 client.5700 shutdown 2024-04-15T15:21:02.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.597+0000 7f80d6e76740 1 -- 172.21.15.105:0/1893066436 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1f65b7740 msgr2=0x55e1f65b4550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:02.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.597+0000 7f80d6e76740 1 --2- 172.21.15.105:0/1893066436 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1f65b7740 0x55e1f65b4550 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f80bc006f80 tx=0x7f80bc004730 comp rx=0 tx=0).stop 2024-04-15T15:21:02.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.597+0000 7f80d6e76740 1 -- 172.21.15.105:0/1893066436 shutdown_connections 2024-04-15T15:21:02.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.597+0000 7f80d6e76740 1 --2- 172.21.15.105:0/1893066436 >> [v2:172.21.15.105:6835/958066041,v1:172.21.15.105:6837/958066041] conn(0x55e1f65b4a90 0x55e1f65d54f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:02.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.597+0000 7f80d6e76740 1 --2- 172.21.15.105:0/1893066436 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e1f65d6b90 0x55e1f6582980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:02.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.597+0000 7f80d6e76740 1 --2- 172.21.15.105:0/1893066436 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1f65b7740 0x55e1f65b4550 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:02.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.597+0000 7f80ad7fa640 0 client.5700 ms_handle_reset on v2:172.21.15.105:6835/958066041 2024-04-15T15:21:02.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.597+0000 7f80d6e76740 1 --2- 172.21.15.105:0/1893066436 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55e1f65b38a0 0x55e1f6648de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:02.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.597+0000 7f80d6e76740 1 -- 172.21.15.105:0/1893066436 >> 172.21.15.105:0/1893066436 conn(0x55e1f6583da0 msgr2=0x55e1f65b1720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:02.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.597+0000 7f80d6e76740 1 -- 172.21.15.105:0/1893066436 shutdown_connections 2024-04-15T15:21:02.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.597+0000 7f80d6e76740 1 -- 172.21.15.105:0/1893066436 wait complete. 2024-04-15T15:21:02.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.597+0000 7f80d6e76740 20 client.5700 trim_cache size 0 max 16384 2024-04-15T15:21:02.608 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-04-15T15:21:02.608 INFO:teuthology.orchestra.run.smithi105.stdout: "path": "/tmp/mds.0-ops" 2024-04-15T15:21:02.608 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-04-15T15:21:02.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.598+0000 7f80d4288640 1 -- 172.21.15.105:0/3139632777 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f80b006c980 msgr2=0x7f80b006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:02.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.598+0000 7f80d4288640 1 --2- 172.21.15.105:0/3139632777 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f80b006c980 0x7f80b006ee40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f80cc15e340 tx=0x7f80c8009040 comp rx=0 tx=0).stop 2024-04-15T15:21:02.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.598+0000 7f80d4288640 1 -- 172.21.15.105:0/3139632777 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80cc15e8d0 msgr2=0x7f80cc1565c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:02.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.598+0000 7f80d4288640 1 --2- 172.21.15.105:0/3139632777 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80cc15e8d0 0x7f80cc1565c0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f80c400c7a0 tx=0x7f80c400cc60 comp rx=0 tx=0).stop 2024-04-15T15:21:02.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.598+0000 7f80d4288640 1 -- 172.21.15.105:0/3139632777 shutdown_connections 2024-04-15T15:21:02.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.598+0000 7f80d4288640 1 --2- 172.21.15.105:0/3139632777 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f80b006c980 0x7f80b006ee40 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:02.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.598+0000 7f80d4288640 1 --2- 172.21.15.105:0/3139632777 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80cc15e8d0 0x7f80cc1565c0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:02.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.598+0000 7f80d4288640 1 --2- 172.21.15.105:0/3139632777 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80cc0d1650 0x7f80cc0b8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:02.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.598+0000 7f80d4288640 1 --2- 172.21.15.105:0/3139632777 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80cc0ba480 0x7f80cc0b82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:02.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.598+0000 7f80d4288640 1 -- 172.21.15.105:0/3139632777 >> 172.21.15.105:0/3139632777 conn(0x7f80cc06d460 msgr2=0x7f80cc0ca330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:02.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.598+0000 7f80d4288640 1 -- 172.21.15.105:0/3139632777 shutdown_connections 2024-04-15T15:21:02.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:02.598+0000 7f80d4288640 1 -- 172.21.15.105:0/3139632777 wait complete. 2024-04-15T15:21:02.611 DEBUG:teuthology.orchestra.run.smithi105:> mktemp 2024-04-15T15:21:02.626 INFO:teuthology.orchestra.run.smithi105.stdout:/tmp/tmp.JntQ2TEdUH 2024-04-15T15:21:02.627 DEBUG:teuthology.orchestra.run.smithi105:> sudo cp /tmp/mds.0-ops /tmp/tmp.JntQ2TEdUH 2024-04-15T15:21:02.695 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 0666 /tmp/tmp.JntQ2TEdUH 2024-04-15T15:21:02.824 DEBUG:teuthology.orchestra.remote:smithi105:/tmp/tmp.JntQ2TEdUH is 8KB 2024-04-15T15:21:02.872 DEBUG:teuthology.orchestra.run.smithi105:> rm -fr /tmp/tmp.JntQ2TEdUH 2024-04-15T15:21:02.888 DEBUG:tasks.cephfs.filesystem:read 7757B of ops 2024-04-15T15:21:02.888 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json dump tree /volumes/_nogroup/subvol_quiesce/3b88d86e-6ee6-481f-8fa4-d97e9f88b65d --path=/tmp/mds.0-cache 2024-04-15T15:21:03.045 INFO:teuthology.orchestra.run.smithi105.stdout:{"path":"/tmp/mds.0-cache"} 2024-04-15T15:21:03.045 DEBUG:tasks.cephfs.filesystem:_json_asok output { "path": "/tmp/mds.0-cache" } 2024-04-15T15:21:03.045 DEBUG:teuthology.orchestra.run.smithi105:> mktemp 2024-04-15T15:21:03.061 INFO:teuthology.orchestra.run.smithi105.stdout:/tmp/tmp.ZKHW0I5Gvv 2024-04-15T15:21:03.062 DEBUG:teuthology.orchestra.run.smithi105:> sudo cp /tmp/mds.0-cache /tmp/tmp.ZKHW0I5Gvv 2024-04-15T15:21:03.130 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 0666 /tmp/tmp.ZKHW0I5Gvv 2024-04-15T15:21:03.258 DEBUG:teuthology.orchestra.remote:smithi105:/tmp/tmp.ZKHW0I5Gvv is 6KB 2024-04-15T15:21:03.280 DEBUG:teuthology.orchestra.run.smithi105:> rm -fr /tmp/tmp.ZKHW0I5Gvv 2024-04-15T15:21:03.295 DEBUG:tasks.cephfs.filesystem:read 6360B of cache 2024-04-15T15:21:03.296 INFO:tasks.cephfs.test_quiesce:qp = 1; qi = 1; qib = 1; qina = 0 2024-04-15T15:21:03.296 ERROR:tasks.cephfs.test_quiesce:inode: { "ino": 1099511627779, "rdev": 0, "ctime": "2024-04-15T15:21:01.749878+0000", "btime": "2024-04-15T15:20:42.757105+0000", "mode": 16895, "uid": 0, "gid": 0, "nlink": 1, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "layout": { "stripe_unit": 4194304, "stripe_count": 1, "object_size": 4194304, "pool_id": 5, "pool_ns": "" }, "old_pools": [], "size": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "truncate_from": 0, "truncate_pending": 0, "mtime": "2024-04-15T15:21:01.749878+0000", "atime": "2024-04-15T15:20:42.757105+0000", "time_warp_seq": 0, "change_attr": 2, "export_pin": -1, "export_ephemeral_random_pin": 0, "export_ephemeral_distributed_pin": false, "quiesce_block": false, "client_ranges": [], "dirstat": { "version": 0, "mtime": "2024-04-15T15:21:01.749878+0000", "num_files": 1, "num_subdirs": 0, "change_attr": 1 }, "rstat": { "version": 0, "rbytes": 0, "rfiles": 1, "rsubdirs": 1, "rsnaps": 0, "rctime": "2024-04-15T15:21:01.782807+0000" }, "accounted_rstat": { "version": 0, "rbytes": 0, "rfiles": 1, "rsubdirs": 1, "rsnaps": 0, "rctime": "2024-04-15T15:21:01.782807+0000" }, "version": 20, "file_data_version": 0, "xattr_version": 1, "backtrace_version": 2, "stray_prior_path": "", "max_size_ever": 0, "quota": { "max_bytes": 0, "max_files": 0 }, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "symlink": "", "xattrs": [], "dirfragtree": { "splits": [] }, "old_inodes": [], "oldest_snap": 18446744073709551614, "damage_flags": 0, "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 1, "is_frozen": false, "is_freezing": false, "pins": { "request": 1, "lock": 2, "dirfrag": 1, "caps": 1, "dirtyparent": 1, "dirty": 1, "waiter": 0, "authpin": 1 }, "nref": 8, "versionlock": { "gather_set": [], "state": "lock", "type": "iversion", "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 1, "num_xlocks": 0, "xlock_by": {} }, "authlock": {}, "linklock": {}, "dirfragtreelock": {}, "filelock": {}, "xattrlock": {}, "snaplock": {}, "nestlock": { "gather_set": [], "state": "lock", "type": "inest", "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "xlock_by": {} }, "flocklock": {}, "policylock": {}, "quiescelock": { "gather_set": [], "state": "lock", "type": "iquiesce", "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 1, "xlock_by": { "reqid": { "entity": { "type": "mds", "num": 0 }, "tid": 2 } } }, "states": [ "auth", "dirty", "dirtyparent", "dirtypool" ], "client_caps": [ { "client_id": 5003, "pending": "pAsLsXs", "issued": "pAsLsXs", "wanted": "-", "last_sent": 2 }, { "client_id": 5021, "pending": "pAsLsXs", "issued": "pAsLsXs", "wanted": "-", "last_sent": 2 }, { "client_id": 5460, "pending": "pAsLsXs", "issued": "pAsLsXs", "wanted": "-", "last_sent": 7 }, { "client_id": 5619, "pending": "pAsLsXs", "issued": "pAsLsXs", "wanted": "-", "last_sent": 3 }, { "client_id": 5640, "pending": "pAsLsXs", "issued": "pAsLsXs", "wanted": "-", "last_sent": 2 } ], "loner": -1, "want_loner": -1, "mds_caps_wanted": [], "dirfrags": [ { "path": "/volumes/_nogroup/subvol_quiesce/3b88d86e-6ee6-481f-8fa4-d97e9f88b65d", "dirfrag": "0x10000000003", "snapid_first": 2, "projected_version": "7", "version": "7", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "dirty": 1, "authpin": 0 }, "nref": 2, "dentries": [ { "path": "volumes/_nogroup/subvol_quiesce/3b88d86e-6ee6-481f-8fa4-d97e9f88b65d/file", "path_ino": 1, "snap_first": 2, "snap_last": 18446744073709551614, "is_primary": true, "is_remote": false, "is_null": false, "is_new": false, "inode": 1099511629282, "remote_type": "", "version": 6, "projected_version": 6, "auth_pins": 0, "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "is_frozen": false, "is_freezing": false, "pins": { "request": 0, "lock": 0, "inodepin": 1, "dirty": 1, "authpin": 0 }, "nref": 2, "lock": {}, "versionlock": { "gather_set": [], "state": "lock", "type": "dversion", "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "xlock_by": {} }, "states": [ "auth", "dirty" ] } ] } ], "is_subvolume": false } 2024-04-15T15:21:03.296 ERROR:tasks.cephfs.test_quiesce:op: { "description": "internal op quiesce_inode:mds.0:2 fp=#0x10000000003", "initiated_at": "2024-04-15T15:21:01.973839+0000", "age": 0.624193013, "duration": 0.624324492, "continuous": true, "type_data": { "result": -2147483648, "flag_point": "quiesce complete", "reqid": { "entity": { "type": "mds", "num": 0 }, "tid": 2 }, "op_type": "internal_op", "internal_op": 5385, "op_name": "quiesce_inode", "events": [ { "time": "2024-04-15T15:21:01.973839+0000", "event": "initiated" }, { "time": "2024-04-15T15:21:01.973839+0000", "event": "header_read" }, { "time": "2024-04-15T15:21:01.973839+0000", "event": "throttled" }, { "time": "2024-04-15T15:21:01.973839+0000", "event": "all_read" }, { "time": "2024-04-15T15:21:01.973839+0000", "event": "dispatched" }, { "time": "2024-04-15T15:21:01.974035+0000", "event": "failed to xlock, waiting" }, { "time": "2024-04-15T15:21:01.974449+0000", "event": "acquired locks" }, { "time": "2024-04-15T15:21:01.977636+0000", "event": "quiesce complete" } ], "locks": [ { "object": { "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 1, "is_frozen": false, "is_freezing": false, "pins": { "request": 1, "lock": 2, "dirfrag": 1, "caps": 1, "dirtyparent": 1, "dirty": 1, "waiter": 0, "authpin": 1 }, "nref": 8 }, "object_string": "[inode 0x10000000003 [...2,head] /volumes/_nogroup/subvol_quiesce/3b88d86e-6ee6-481f-8fa4-d97e9f88b65d/ auth v20 ap=1 DIRTYPARENT f(v0 m2024-04-15T15:21:01.749878+0000 1=1+0) n(v0 rc2024-04-15T15:21:01.782807+0000 2=1+1) (inest lock) (iversion lock w=1) (iquiesce lock x=1 by request(mds.0:2 nref=3)) caps={5003=pAsLsXs/-@2,5021=pAsLsXs/-@2,5460=pAsLsXs/-@7,5619=pAsLsXs/-@3,5640=pAsLsXs/-@2} | request=1 lock=2 dirfrag=1 caps=1 dirtyparent=1 dirty=1 waiter=0 authpin=1 0x5572ffee9080]", "lock": { "gather_set": [], "state": "lock", "type": "iquiesce", "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 1, "xlock_by": { "reqid": { "entity": { "type": "mds", "num": 0 }, "tid": 2 } } }, "flags": 4, "wrlock_target": -1 }, { "object": { "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 1, "is_frozen": false, "is_freezing": false, "pins": { "request": 1, "lock": 2, "dirfrag": 1, "caps": 1, "dirtyparent": 1, "dirty": 1, "waiter": 0, "authpin": 1 }, "nref": 8 }, "object_string": "[inode 0x10000000003 [...2,head] /volumes/_nogroup/subvol_quiesce/3b88d86e-6ee6-481f-8fa4-d97e9f88b65d/ auth v20 ap=1 DIRTYPARENT f(v0 m2024-04-15T15:21:01.749878+0000 1=1+0) n(v0 rc2024-04-15T15:21:01.782807+0000 2=1+1) (inest lock) (iversion lock w=1) (iquiesce lock x=1 by request(mds.0:2 nref=3)) caps={5003=pAsLsXs/-@2,5021=pAsLsXs/-@2,5460=pAsLsXs/-@7,5619=pAsLsXs/-@3,5640=pAsLsXs/-@2} | request=1 lock=2 dirfrag=1 caps=1 dirtyparent=1 dirty=1 waiter=0 authpin=1 0x5572ffee9080]", "lock": { "gather_set": [], "state": "lock", "type": "iversion", "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 1, "num_xlocks": 0, "xlock_by": {} }, "flags": 2, "wrlock_target": -1 } ] } } 2024-04-15T15:21:03.297 ERROR:tasks.cephfs.test_quiesce:lock_type: iversion 2024-04-15T15:21:03.297 ERROR:tasks.cephfs.test_quiesce:cache: [ { "ino": 1099511629282, "rdev": 0, "ctime": "2024-04-15T15:21:01.782807+0000", "btime": "2024-04-15T15:21:01.749878+0000", "mode": 33188, "uid": 1000, "gid": 1301, "nlink": 1, "dir_layout": { "dir_hash": 0, "unused1": 0, "unused2": 0, "unused3": 0 }, "layout": { "stripe_unit": 4194304, "stripe_count": 1, "object_size": 4194304, "pool_id": 5, "pool_ns": "" }, "old_pools": [], "size": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "truncate_from": 0, "truncate_pending": 0, "mtime": "2024-04-15T15:21:01.753583+0000", "atime": "2024-04-15T15:21:01.753583+0000", "time_warp_seq": 2, "change_attr": 1, "export_pin": -1, "export_ephemeral_random_pin": 0, "export_ephemeral_distributed_pin": false, "quiesce_block": true, "client_ranges": [], "dirstat": { "version": 0, "mtime": "0.000000", "num_files": 0, "num_subdirs": 0, "change_attr": 0 }, "rstat": { "version": 0, "rbytes": 0, "rfiles": 1, "rsubdirs": 0, "rsnaps": 0, "rctime": "2024-04-15T15:21:01.782807+0000" }, "accounted_rstat": { "version": 0, "rbytes": 0, "rfiles": 1, "rsubdirs": 0, "rsnaps": 0, "rctime": "2024-04-15T15:21:01.782807+0000" }, "version": 6, "file_data_version": 0, "xattr_version": 1, "backtrace_version": 4, "stray_prior_path": "", "max_size_ever": 0, "quota": { "max_bytes": 0, "max_files": 0 }, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "symlink": "", "xattrs": [], "dirfragtree": { "splits": [] }, "old_inodes": [], "oldest_snap": 18446744073709551614, "damage_flags": 0, "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 1, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 1, "lock": 0, "caps": 1, "dirtyparent": 1, "dirty": 1, "waiter": 0, "authpin": 1 }, "nref": 5, "versionlock": { "gather_set": [], "state": "lock", "type": "iversion", "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "xlock_by": {} }, "authlock": {}, "linklock": {}, "dirfragtreelock": {}, "filelock": {}, "xattrlock": {}, "snaplock": {}, "nestlock": { "gather_set": [], "state": "lock", "type": "inest", "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "xlock_by": {} }, "flocklock": {}, "policylock": {}, "quiescelock": { "gather_set": [], "state": "lock", "type": "iquiesce", "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "xlock_by": {} }, "states": [ "auth", "dirty", "dirtyparent", "dirtypool" ], "client_caps": [ { "client_id": 5619, "pending": "pAsLsXs", "issued": "pAsLsXs", "wanted": "-", "last_sent": 3 } ], "loner": -1, "want_loner": -1, "mds_caps_wanted": [], "dirfrags": [], "is_subvolume": false }, { "ino": 1099511627779, "rdev": 0, "ctime": "2024-04-15T15:21:01.749878+0000", "btime": "2024-04-15T15:20:42.757105+0000", "mode": 16895, "uid": 0, "gid": 0, "nlink": 1, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "layout": { "stripe_unit": 4194304, "stripe_count": 1, "object_size": 4194304, "pool_id": 5, "pool_ns": "" }, "old_pools": [], "size": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "truncate_from": 0, "truncate_pending": 0, "mtime": "2024-04-15T15:21:01.749878+0000", "atime": "2024-04-15T15:20:42.757105+0000", "time_warp_seq": 0, "change_attr": 2, "export_pin": -1, "export_ephemeral_random_pin": 0, "export_ephemeral_distributed_pin": false, "quiesce_block": false, "client_ranges": [], "dirstat": { "version": 0, "mtime": "2024-04-15T15:21:01.749878+0000", "num_files": 1, "num_subdirs": 0, "change_attr": 1 }, "rstat": { "version": 0, "rbytes": 0, "rfiles": 1, "rsubdirs": 1, "rsnaps": 0, "rctime": "2024-04-15T15:21:01.782807+0000" }, "accounted_rstat": { "version": 0, "rbytes": 0, "rfiles": 1, "rsubdirs": 1, "rsnaps": 0, "rctime": "2024-04-15T15:21:01.782807+0000" }, "version": 20, "file_data_version": 0, "xattr_version": 1, "backtrace_version": 2, "stray_prior_path": "", "max_size_ever": 0, "quota": { "max_bytes": 0, "max_files": 0 }, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "symlink": "", "xattrs": [], "dirfragtree": { "splits": [] }, "old_inodes": [], "oldest_snap": 18446744073709551614, "damage_flags": 0, "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 1, "is_frozen": false, "is_freezing": false, "pins": { "request": 1, "lock": 2, "dirfrag": 1, "caps": 1, "dirtyparent": 1, "dirty": 1, "waiter": 0, "authpin": 1 }, "nref": 8, "versionlock": { "gather_set": [], "state": "lock", "type": "iversion", "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 1, "num_xlocks": 0, "xlock_by": {} }, "authlock": {}, "linklock": {}, "dirfragtreelock": {}, "filelock": {}, "xattrlock": {}, "snaplock": {}, "nestlock": { "gather_set": [], "state": "lock", "type": "inest", "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "xlock_by": {} }, "flocklock": {}, "policylock": {}, "quiescelock": { "gather_set": [], "state": "lock", "type": "iquiesce", "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 1, "xlock_by": { "reqid": { "entity": { "type": "mds", "num": 0 }, "tid": 2 } } }, "states": [ "auth", "dirty", "dirtyparent", "dirtypool" ], "client_caps": [ { "client_id": 5003, "pending": "pAsLsXs", "issued": "pAsLsXs", "wanted": "-", "last_sent": 2 }, { "client_id": 5021, "pending": "pAsLsXs", "issued": "pAsLsXs", "wanted": "-", "last_sent": 2 }, { "client_id": 5460, "pending": "pAsLsXs", "issued": "pAsLsXs", "wanted": "-", "last_sent": 7 }, { "client_id": 5619, "pending": "pAsLsXs", "issued": "pAsLsXs", "wanted": "-", "last_sent": 3 }, { "client_id": 5640, "pending": "pAsLsXs", "issued": "pAsLsXs", "wanted": "-", "last_sent": 2 } ], "loner": -1, "want_loner": -1, "mds_caps_wanted": [], "dirfrags": [ { "path": "/volumes/_nogroup/subvol_quiesce/3b88d86e-6ee6-481f-8fa4-d97e9f88b65d", "dirfrag": "0x10000000003", "snapid_first": 2, "projected_version": "7", "version": "7", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "dirty": 1, "authpin": 0 }, "nref": 2, "dentries": [ { "path": "volumes/_nogroup/subvol_quiesce/3b88d86e-6ee6-481f-8fa4-d97e9f88b65d/file", "path_ino": 1, "snap_first": 2, "snap_last": 18446744073709551614, "is_primary": true, "is_remote": false, "is_null": false, "is_new": false, "inode": 1099511629282, "remote_type": "", "version": 6, "projected_version": 6, "auth_pins": 0, "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "is_frozen": false, "is_freezing": false, "pins": { "request": 0, "lock": 0, "inodepin": 1, "dirty": 1, "authpin": 0 }, "nref": 2, "lock": {}, "versionlock": { "gather_set": [], "state": "lock", "type": "dversion", "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "xlock_by": {} }, "states": [ "auth", "dirty" ] } ] } ], "is_subvolume": false } ] 2024-04-15T15:21:03.297 ERROR:tasks.cephfs.test_quiesce:ops: { "1099511627779": { "description": "internal op quiesce_inode:mds.0:2 fp=#0x10000000003", "initiated_at": "2024-04-15T15:21:01.973839+0000", "age": 0.624193013, "duration": 0.624324492, "continuous": true, "type_data": { "result": -2147483648, "flag_point": "quiesce complete", "reqid": { "entity": { "type": "mds", "num": 0 }, "tid": 2 }, "op_type": "internal_op", "internal_op": 5385, "op_name": "quiesce_inode", "events": [ { "time": "2024-04-15T15:21:01.973839+0000", "event": "initiated" }, { "time": "2024-04-15T15:21:01.973839+0000", "event": "header_read" }, { "time": "2024-04-15T15:21:01.973839+0000", "event": "throttled" }, { "time": "2024-04-15T15:21:01.973839+0000", "event": "all_read" }, { "time": "2024-04-15T15:21:01.973839+0000", "event": "dispatched" }, { "time": "2024-04-15T15:21:01.974035+0000", "event": "failed to xlock, waiting" }, { "time": "2024-04-15T15:21:01.974449+0000", "event": "acquired locks" }, { "time": "2024-04-15T15:21:01.977636+0000", "event": "quiesce complete" } ], "locks": [ { "object": { "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 1, "is_frozen": false, "is_freezing": false, "pins": { "request": 1, "lock": 2, "dirfrag": 1, "caps": 1, "dirtyparent": 1, "dirty": 1, "waiter": 0, "authpin": 1 }, "nref": 8 }, "object_string": "[inode 0x10000000003 [...2,head] /volumes/_nogroup/subvol_quiesce/3b88d86e-6ee6-481f-8fa4-d97e9f88b65d/ auth v20 ap=1 DIRTYPARENT f(v0 m2024-04-15T15:21:01.749878+0000 1=1+0) n(v0 rc2024-04-15T15:21:01.782807+0000 2=1+1) (inest lock) (iversion lock w=1) (iquiesce lock x=1 by request(mds.0:2 nref=3)) caps={5003=pAsLsXs/-@2,5021=pAsLsXs/-@2,5460=pAsLsXs/-@7,5619=pAsLsXs/-@3,5640=pAsLsXs/-@2} | request=1 lock=2 dirfrag=1 caps=1 dirtyparent=1 dirty=1 waiter=0 authpin=1 0x5572ffee9080]", "lock": { "gather_set": [], "state": "lock", "type": "iquiesce", "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 1, "xlock_by": { "reqid": { "entity": { "type": "mds", "num": 0 }, "tid": 2 } } }, "flags": 4, "wrlock_target": -1 }, { "object": { "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 1, "is_frozen": false, "is_freezing": false, "pins": { "request": 1, "lock": 2, "dirfrag": 1, "caps": 1, "dirtyparent": 1, "dirty": 1, "waiter": 0, "authpin": 1 }, "nref": 8 }, "object_string": "[inode 0x10000000003 [...2,head] /volumes/_nogroup/subvol_quiesce/3b88d86e-6ee6-481f-8fa4-d97e9f88b65d/ auth v20 ap=1 DIRTYPARENT f(v0 m2024-04-15T15:21:01.749878+0000 1=1+0) n(v0 rc2024-04-15T15:21:01.782807+0000 2=1+1) (inest lock) (iversion lock w=1) (iquiesce lock x=1 by request(mds.0:2 nref=3)) caps={5003=pAsLsXs/-@2,5021=pAsLsXs/-@2,5460=pAsLsXs/-@7,5619=pAsLsXs/-@3,5640=pAsLsXs/-@2} | request=1 lock=2 dirfrag=1 caps=1 dirtyparent=1 dirty=1 waiter=0 authpin=1 0x5572ffee9080]", "lock": { "gather_set": [], "state": "lock", "type": "iversion", "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 1, "num_xlocks": 0, "xlock_by": {} }, "flags": 2, "wrlock_target": -1 } ] } }, "1099511629282": { "description": "internal op quiesce_inode:mds.0:3 fp=#0x100000005e2", "initiated_at": "2024-04-15T15:21:01.974585+0000", "age": 0.623446553, "duration": 0.623665645, "continuous": true, "type_data": { "result": -2147483648, "flag_point": "quiesce blocked", "reqid": { "entity": { "type": "mds", "num": 0 }, "tid": 3 }, "op_type": "internal_op", "internal_op": 5385, "op_name": "quiesce_inode", "events": [ { "time": "2024-04-15T15:21:01.974585+0000", "event": "initiated" }, { "time": "2024-04-15T15:21:01.974585+0000", "event": "header_read" }, { "time": "2024-04-15T15:21:01.974585+0000", "event": "throttled" }, { "time": "2024-04-15T15:21:01.974585+0000", "event": "all_read" }, { "time": "2024-04-15T15:21:01.974585+0000", "event": "dispatched" }, { "time": "2024-04-15T15:21:01.974694+0000", "event": "failed to xlock, waiting" }, { "time": "2024-04-15T15:21:01.975318+0000", "event": "failed to xlock, waiting" }, { "time": "2024-04-15T15:21:01.977517+0000", "event": "acquired locks" }, { "time": "2024-04-15T15:21:01.977518+0000", "event": "quiesce blocked" } ], "locks": [] } } } 2024-04-15T15:21:03.298 INFO:tasks.cephfs.test_quiesce:making archive directory /home/teuthworker/archive/leonidus-2024-04-15_14:54:08-fs-wip-lusov-quiesce-xlock-distro-default-smithi/7657132/quiesce 2024-04-15T15:21:03.300 ERROR:tasks.cephfs.test_quiesce:cache written to /home/teuthworker/archive/leonidus-2024-04-15_14:54:08-fs-wip-lusov-quiesce-xlock-distro-default-smithi/7657132/quiesce/cachei9z1t90a 2024-04-15T15:21:03.302 ERROR:tasks.cephfs.test_quiesce:ops written to /home/teuthworker/archive/leonidus-2024-04-15_14:54:08-fs-wip-lusov-quiesce-xlock-distro-default-smithi/7657132/quiesce/opsddbci77m 2024-04-15T15:21:03.302 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-15T15:21:03.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.435+0000 7f7f70cc8640 1 Processor -- start 2024-04-15T15:21:03.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.435+0000 7f7f70cc8640 1 -- start start 2024-04-15T15:21:03.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.436+0000 7f7f70cc8640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7f6c0722c0 0x7f7f6c06e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:03.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.436+0000 7f7f70cc8640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f6c0ccfb0 con 0x7f7f6c145e00 2024-04-15T15:21:03.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.436+0000 7f7f70cc8640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f6c0cd290 con 0x7f7f6c071e60 2024-04-15T15:21:03.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.436+0000 7f7f70cc8640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f7f6c0cdbc0 con 0x7f7f6c0722c0 2024-04-15T15:21:03.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.436+0000 7f7f6a59c640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f7f6c145e00 0x7f7f6c0ce7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:50794/0 (socket says 172.21.15.105:50794) 2024-04-15T15:21:03.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.436+0000 7f7f6a59c640 1 -- 172.21.15.105:0/40845370 learned_addr learned my addr 172.21.15.105:0/40845370 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:21:03.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.436+0000 7f7f69d9b640 1 -- 172.21.15.105:0/40845370 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 16026252 0 0) 0x7f7f6c0ccfb0 con 0x7f7f6c145e00 2024-04-15T15:21:03.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.436+0000 7f7f69d9b640 1 -- 172.21.15.105:0/40845370 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f58003520 con 0x7f7f6c145e00 2024-04-15T15:21:03.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.436+0000 7f7f6ad9d640 1 --2- 172.21.15.105:0/40845370 >> v2:172.21.15.154:3300/0 conn(0x7f7f6c0722c0 0x7f7f6c06e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:03.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.436+0000 7f7f69d9b640 1 -- 172.21.15.105:0/40845370 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3764860002 0 0) 0x7f7f58003520 con 0x7f7f6c145e00 2024-04-15T15:21:03.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.436+0000 7f7f69d9b640 1 -- 172.21.15.105:0/40845370 >> v2:172.21.15.154:3300/0 conn(0x7f7f6c0722c0 msgr2=0x7f7f6c06e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:03.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.436+0000 7f7f69d9b640 1 --2- 172.21.15.105:0/40845370 >> v2:172.21.15.154:3300/0 conn(0x7f7f6c0722c0 0x7f7f6c06e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:03.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.436+0000 7f7f69d9b640 1 -- 172.21.15.105:0/40845370 >> v1:172.21.15.154:6789/0 conn(0x7f7f6c071e60 legacy=0x7f7f6c06df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:03.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.436+0000 7f7f69d9b640 1 -- 172.21.15.105:0/40845370 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f6c0cd570 con 0x7f7f6c145e00 2024-04-15T15:21:03.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.436+0000 7f7f69d9b640 1 -- 172.21.15.105:0/40845370 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7f7f5c001670 con 0x7f7f6c145e00 2024-04-15T15:21:03.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.436+0000 7f7f6ad9d640 1 --2- 172.21.15.105:0/40845370 >> v2:172.21.15.154:3300/0 conn(0x7f7f6c0722c0 0x7f7f6c06e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:21:03.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.436+0000 7f7f69d9b640 1 -- 172.21.15.105:0/40845370 >> v1:172.21.15.105:6789/0 conn(0x7f7f6c145e00 legacy=0x7f7f6c0ce7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:03.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.436+0000 7f7f69d9b640 1 --2- 172.21.15.105:0/40845370 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f580039b0 0x7f7f58003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:03.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.436+0000 7f7f69d9b640 1 --2- 172.21.15.105:0/40845370 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f58004800 0x7f7f58004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:03.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.436+0000 7f7f69d9b640 1 --2- 172.21.15.105:0/40845370 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f58005210 0x7f7f5800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:03.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.436+0000 7f7f69d9b640 1 -- 172.21.15.105:0/40845370 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c0cdbc0 con 0x7f7f58004800 2024-04-15T15:21:03.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.436+0000 7f7f69d9b640 1 -- 172.21.15.105:0/40845370 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5800c140 con 0x7f7f58005210 2024-04-15T15:21:03.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.436+0000 7f7f69d9b640 1 -- 172.21.15.105:0/40845370 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f7f5800c340 con 0x7f7f580039b0 2024-04-15T15:21:03.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.437+0000 7f7f61d9b640 1 --2- 172.21.15.105:0/40845370 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f58004800 0x7f7f58004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:03.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.437+0000 7f7f6ad9d640 1 --2- 172.21.15.105:0/40845370 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f58005210 0x7f7f5800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:03.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.437+0000 7f7f6a59c640 1 --2- 172.21.15.105:0/40845370 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f580039b0 0x7f7f58003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:03.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.437+0000 7f7f61d9b640 1 -- 172.21.15.105:0/40845370 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f580039b0 msgr2=0x7f7f58003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:03.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.437+0000 7f7f61d9b640 1 --2- 172.21.15.105:0/40845370 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f580039b0 0x7f7f58003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:03.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.437+0000 7f7f61d9b640 1 -- 172.21.15.105:0/40845370 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f58005210 msgr2=0x7f7f5800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:03.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.437+0000 7f7f61d9b640 1 --2- 172.21.15.105:0/40845370 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f58005210 0x7f7f5800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:03.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.437+0000 7f7f6ad9d640 1 -- 172.21.15.105:0/40845370 reap_dead start 2024-04-15T15:21:03.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.437+0000 7f7f61d9b640 1 -- 172.21.15.105:0/40845370 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f58003520 con 0x7f7f58004800 2024-04-15T15:21:03.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.438+0000 7f7f61d9b640 1 --2- 172.21.15.105:0/40845370 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f58004800 0x7f7f58004c80 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f7f54001750 tx=0x7f7f58005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7533c2405d5ee89c server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:03.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.438+0000 7f7f69d9b640 1 -- 172.21.15.105:0/40845370 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f54001370 con 0x7f7f58004800 2024-04-15T15:21:03.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.438+0000 7f7f69d9b640 1 -- 172.21.15.105:0/40845370 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f540052d0 con 0x7f7f58004800 2024-04-15T15:21:03.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.438+0000 7f7f69d9b640 1 -- 172.21.15.105:0/40845370 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f54003830 con 0x7f7f58004800 2024-04-15T15:21:03.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.438+0000 7f7f70cc8640 1 -- 172.21.15.105:0/40845370 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f58004800 msgr2=0x7f7f58004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:03.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.438+0000 7f7f70cc8640 1 --2- 172.21.15.105:0/40845370 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f58004800 0x7f7f58004c80 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f7f54001750 tx=0x7f7f58005830 comp rx=0 tx=0).stop 2024-04-15T15:21:03.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.438+0000 7f7f70cc8640 1 -- 172.21.15.105:0/40845370 shutdown_connections 2024-04-15T15:21:03.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.438+0000 7f7f70cc8640 1 --2- 172.21.15.105:0/40845370 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f58004800 0x7f7f58004c80 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:03.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.438+0000 7f7f70cc8640 1 -- 172.21.15.105:0/40845370 >> 172.21.15.105:0/40845370 conn(0x7f7f6c067d50 msgr2=0x7f7f6c06b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:03.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.438+0000 7f7f70cc8640 1 -- 172.21.15.105:0/40845370 shutdown_connections 2024-04-15T15:21:03.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.438+0000 7f7f70cc8640 1 -- 172.21.15.105:0/40845370 wait complete. 2024-04-15T15:21:03.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.440+0000 7f7f70cc8640 1 Processor -- start 2024-04-15T15:21:03.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.440+0000 7f7f70cc8640 1 -- start start 2024-04-15T15:21:03.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.440+0000 7f7f70cc8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f58004800 0x7f7f6c1588b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:03.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.440+0000 7f7f70cc8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f6c144b90 0x7f7f6c157250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:03.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.440+0000 7f7f70cc8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f6c157790 0x7f7f6c157c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:03.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.440+0000 7f7f70cc8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c0ce370 con 0x7f7f6c157790 2024-04-15T15:21:03.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.440+0000 7f7f70cc8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c0726d0 con 0x7f7f58004800 2024-04-15T15:21:03.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.440+0000 7f7f70cc8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f7f6c1461e0 con 0x7f7f6c144b90 2024-04-15T15:21:03.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.441+0000 7f7f6ad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f6c157790 0x7f7f6c157c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:03.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.441+0000 7f7f6ad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f6c157790 0x7f7f6c157c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46626/0 (socket says 172.21.15.105:46626) 2024-04-15T15:21:03.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.441+0000 7f7f6ad9d640 1 -- 172.21.15.105:0/302951449 learned_addr learned my addr 172.21.15.105:0/302951449 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:03.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.441+0000 7f7f6a59c640 1 --2- 172.21.15.105:0/302951449 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f58004800 0x7f7f6c1588b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:03.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.441+0000 7f7f61d9b640 1 --2- 172.21.15.105:0/302951449 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f6c144b90 0x7f7f6c157250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:03.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.441+0000 7f7f6ad9d640 1 -- 172.21.15.105:0/302951449 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f6c144b90 msgr2=0x7f7f6c157250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:03.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.441+0000 7f7f6ad9d640 1 --2- 172.21.15.105:0/302951449 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f6c144b90 0x7f7f6c157250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:03.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.441+0000 7f7f6ad9d640 1 -- 172.21.15.105:0/302951449 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f58004800 msgr2=0x7f7f6c1588b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:03.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.441+0000 7f7f6ad9d640 1 --2- 172.21.15.105:0/302951449 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f58004800 0x7f7f6c1588b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:03.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.441+0000 7f7f6ad9d640 1 -- 172.21.15.105:0/302951449 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f6c0a9840 con 0x7f7f6c157790 2024-04-15T15:21:03.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.441+0000 7f7f6ad9d640 1 --2- 172.21.15.105:0/302951449 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f6c157790 0x7f7f6c157c40 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c072ef0 tx=0x7f7f64010530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:03.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.441+0000 7f7f637fe640 1 -- 172.21.15.105:0/302951449 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f64010cf0 con 0x7f7f6c157790 2024-04-15T15:21:03.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.441+0000 7f7f637fe640 1 -- 172.21.15.105:0/302951449 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f64003990 con 0x7f7f6c157790 2024-04-15T15:21:03.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.441+0000 7f7f70cc8640 1 -- 172.21.15.105:0/302951449 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f6c0a9b20 con 0x7f7f6c157790 2024-04-15T15:21:03.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.441+0000 7f7f637fe640 1 -- 172.21.15.105:0/302951449 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f64003b80 con 0x7f7f6c157790 2024-04-15T15:21:03.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.441+0000 7f7f70cc8640 1 -- 172.21.15.105:0/302951449 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f6c131d30 con 0x7f7f6c157790 2024-04-15T15:21:03.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.442+0000 7f7f70cc8640 1 -- 172.21.15.105:0/302951449 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f6c068a40 con 0x7f7f6c157790 2024-04-15T15:21:03.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.443+0000 7f7f637fe640 1 -- 172.21.15.105:0/302951449 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f7f6401e020 con 0x7f7f6c157790 2024-04-15T15:21:03.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.443+0000 7f7f637fe640 1 --2- 172.21.15.105:0/302951449 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f7f4c06c9b0 0x7f7f4c06ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:03.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.443+0000 7f7f6a59c640 1 --2- 172.21.15.105:0/302951449 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f7f4c06c9b0 0x7f7f4c06ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:03.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.443+0000 7f7f637fe640 1 -- 172.21.15.105:0/302951449 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f7f6408bf30 con 0x7f7f6c157790 2024-04-15T15:21:03.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.443+0000 7f7f6a59c640 1 --2- 172.21.15.105:0/302951449 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f7f4c06c9b0 0x7f7f4c06ee70 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c001050 tx=0x7f7f5c0078f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:03.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.447+0000 7f7f637fe640 1 -- 172.21.15.105:0/302951449 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7f6405a370 con 0x7f7f6c157790 2024-04-15T15:21:03.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.659+0000 7f7f70cc8640 1 -- 172.21.15.105:0/302951449 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f7f6c0aff30 con 0x7f7f6c157790 2024-04-15T15:21:03.661 INFO:tasks.ceph.mon.a.smithi105.stderr:2024-04-15T15:21:03.660+0000 7fd391944640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-15T15:21:03.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.661+0000 7f7f637fe640 1 -- 172.21.15.105:0/302951449 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v15) ==== 161+0+0 (secure 0 0 0) 0x7f7f64060070 con 0x7f7f6c157790 2024-04-15T15:21:03.663 INFO:teuthology.orchestra.run.smithi105.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-15T15:21:03.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.663+0000 7f7f70cc8640 1 -- 172.21.15.105:0/302951449 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f7f4c06c9b0 msgr2=0x7f7f4c06ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:03.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.663+0000 7f7f70cc8640 1 --2- 172.21.15.105:0/302951449 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f7f4c06c9b0 0x7f7f4c06ee70 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c001050 tx=0x7f7f5c0078f0 comp rx=0 tx=0).stop 2024-04-15T15:21:03.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.663+0000 7f7f70cc8640 1 -- 172.21.15.105:0/302951449 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f6c157790 msgr2=0x7f7f6c157c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:03.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.663+0000 7f7f70cc8640 1 --2- 172.21.15.105:0/302951449 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f6c157790 0x7f7f6c157c40 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c072ef0 tx=0x7f7f64010530 comp rx=0 tx=0).stop 2024-04-15T15:21:03.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.663+0000 7f7f70cc8640 1 -- 172.21.15.105:0/302951449 shutdown_connections 2024-04-15T15:21:03.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.664+0000 7f7f70cc8640 1 --2- 172.21.15.105:0/302951449 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f6c144b90 0x7f7f6c157250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:03.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.664+0000 7f7f70cc8640 1 --2- 172.21.15.105:0/302951449 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f7f4c06c9b0 0x7f7f4c06ee70 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:03.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.664+0000 7f7f70cc8640 1 --2- 172.21.15.105:0/302951449 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f6c157790 0x7f7f6c157c40 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:03.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.664+0000 7f7f70cc8640 1 --2- 172.21.15.105:0/302951449 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f58004800 0x7f7f6c1588b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:03.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.664+0000 7f7f70cc8640 1 -- 172.21.15.105:0/302951449 >> 172.21.15.105:0/302951449 conn(0x7f7f6c067d50 msgr2=0x7f7f6c15f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:03.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.664+0000 7f7f70cc8640 1 -- 172.21.15.105:0/302951449 shutdown_connections 2024-04-15T15:21:03.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.664+0000 7f7f70cc8640 1 -- 172.21.15.105:0/302951449 wait complete. 2024-04-15T15:21:03.677 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-15T15:21:03.703 INFO:tasks.ceph.mds.a.smithi105.stdout:starting mds.a at 2024-04-15T15:21:03.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.784+0000 7fa14890d640 1 Processor -- start 2024-04-15T15:21:03.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.784+0000 7fa14890d640 1 -- start start 2024-04-15T15:21:03.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.785+0000 7fa14890d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa1440ce330 0x7fa14415d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:03.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.785+0000 7fa14890d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa14415e320 0x7fa14415e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:03.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.785+0000 7fa14890d640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa14415dbd0 con 0x7fa144160230 2024-04-15T15:21:03.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.785+0000 7fa14890d640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fa14415de90 con 0x7fa1440ce330 2024-04-15T15:21:03.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.785+0000 7fa14890d640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fa1441607a0 con 0x7fa14415e320 2024-04-15T15:21:03.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.785+0000 7fa14259c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa1440ce330 0x7fa14415d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:03.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.785+0000 7fa14259c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa1440ce330 0x7fa14415d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46654/0 (socket says 172.21.15.105:46654) 2024-04-15T15:21:03.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.785+0000 7fa14259c640 1 -- 172.21.15.105:0/728591630 learned_addr learned my addr 172.21.15.105:0/728591630 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:03.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.785+0000 7fa13bfff640 1 --1- 172.21.15.105:0/728591630 >> v1:172.21.15.105:6789/0 conn(0x7fa144160230 0x7fa1440ce710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:50802/0 (socket says 172.21.15.105:50802) 2024-04-15T15:21:03.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.785+0000 7fa142d9d640 1 --2- 172.21.15.105:0/728591630 >> v2:172.21.15.154:3300/0 conn(0x7fa14415e320 0x7fa14415e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:03.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.785+0000 7fa14259c640 1 -- 172.21.15.105:0/728591630 >> v2:172.21.15.154:3300/0 conn(0x7fa14415e320 msgr2=0x7fa14415e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:03.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.785+0000 7fa14259c640 1 --2- 172.21.15.105:0/728591630 >> v2:172.21.15.154:3300/0 conn(0x7fa14415e320 0x7fa14415e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:03.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.785+0000 7fa14259c640 1 -- 172.21.15.105:0/728591630 >> v1:172.21.15.105:6789/0 conn(0x7fa144160230 legacy=0x7fa1440ce710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:03.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.785+0000 7fa141d9b640 1 -- 172.21.15.105:0/728591630 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 832987513 0 0) 0x7fa14415dbd0 con 0x7fa144160230 2024-04-15T15:21:03.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.785+0000 7fa14259c640 1 -- 172.21.15.105:0/728591630 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa144160a80 con 0x7fa1440ce330 2024-04-15T15:21:03.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.786+0000 7fa14259c640 1 --2- 172.21.15.105:0/728591630 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1440ce330 0x7fa14415d690 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fa144161320 tx=0x7fa13402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2af0091d3e0e2f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:03.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.786+0000 7fa141d9b640 1 -- 172.21.15.105:0/728591630 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa134034070 con 0x7fa1440ce330 2024-04-15T15:21:03.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.786+0000 7fa141d9b640 1 -- 172.21.15.105:0/728591630 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa134030860 con 0x7fa1440ce330 2024-04-15T15:21:03.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.786+0000 7fa141d9b640 1 -- 172.21.15.105:0/728591630 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa134030c30 con 0x7fa1440ce330 2024-04-15T15:21:03.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.786+0000 7fa14890d640 1 -- 172.21.15.105:0/728591630 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1440ce330 msgr2=0x7fa14415d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:03.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.786+0000 7fa14890d640 1 --2- 172.21.15.105:0/728591630 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1440ce330 0x7fa14415d690 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fa144161320 tx=0x7fa13402b9a0 comp rx=0 tx=0).stop 2024-04-15T15:21:03.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.787+0000 7fa14890d640 1 -- 172.21.15.105:0/728591630 shutdown_connections 2024-04-15T15:21:03.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.787+0000 7fa14890d640 1 --2- 172.21.15.105:0/728591630 >> v2:172.21.15.154:3300/0 conn(0x7fa14415e320 0x7fa14415e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:03.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.787+0000 7fa14890d640 1 --2- 172.21.15.105:0/728591630 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1440ce330 0x7fa14415d690 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:03.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.787+0000 7fa14890d640 1 -- 172.21.15.105:0/728591630 >> 172.21.15.105:0/728591630 conn(0x7fa14406b9c0 msgr2=0x7fa14406bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:03.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.787+0000 7fa14890d640 1 -- 172.21.15.105:0/728591630 shutdown_connections 2024-04-15T15:21:03.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.787+0000 7fa14890d640 1 -- 172.21.15.105:0/728591630 wait complete. 2024-04-15T15:21:03.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.788+0000 7fa14890d640 1 Processor -- start 2024-04-15T15:21:03.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.788+0000 7fa14890d640 1 -- start start 2024-04-15T15:21:03.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.789+0000 7fa14890d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1440ce330 0x7fa144155a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:03.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.789+0000 7fa14890d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa14415e320 0x7fa144159760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:03.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.789+0000 7fa14890d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa144160230 0x7fa14415a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:03.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.789+0000 7fa14890d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa144156f90 con 0x7fa144160230 2024-04-15T15:21:03.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.789+0000 7fa14890d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa144161040 con 0x7fa14415e320 2024-04-15T15:21:03.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.789+0000 7fa14890d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fa144157110 con 0x7fa1440ce330 2024-04-15T15:21:03.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.789+0000 7fa142d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa144160230 0x7fa14415a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:03.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.789+0000 7fa142d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa144160230 0x7fa14415a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46658/0 (socket says 172.21.15.105:46658) 2024-04-15T15:21:03.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.789+0000 7fa142d9d640 1 -- 172.21.15.105:0/3294898214 learned_addr learned my addr 172.21.15.105:0/3294898214 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:03.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.789+0000 7fa14259c640 1 --2- 172.21.15.105:0/3294898214 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1440ce330 0x7fa144155a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:03.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.789+0000 7fa142d9d640 1 -- 172.21.15.105:0/3294898214 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1440ce330 msgr2=0x7fa144155a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:03.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.789+0000 7fa13bfff640 1 --2- 172.21.15.105:0/3294898214 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa14415e320 0x7fa144159760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:03.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.789+0000 7fa142d9d640 1 --2- 172.21.15.105:0/3294898214 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1440ce330 0x7fa144155a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:03.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.789+0000 7fa142d9d640 1 -- 172.21.15.105:0/3294898214 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa14415e320 msgr2=0x7fa144159760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:03.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.789+0000 7fa142d9d640 1 --2- 172.21.15.105:0/3294898214 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa14415e320 0x7fa144159760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:03.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.789+0000 7fa142d9d640 1 -- 172.21.15.105:0/3294898214 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1441586c0 con 0x7fa144160230 2024-04-15T15:21:03.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.789+0000 7fa142d9d640 1 --2- 172.21.15.105:0/3294898214 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa144160230 0x7fa14415a410 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fa13c00cd70 tx=0x7fa13c00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:03.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.789+0000 7fa13affd640 1 -- 172.21.15.105:0/3294898214 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13c003b90 con 0x7fa144160230 2024-04-15T15:21:03.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.789+0000 7fa13affd640 1 -- 172.21.15.105:0/3294898214 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa13c003d30 con 0x7fa144160230 2024-04-15T15:21:03.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.789+0000 7fa14890d640 1 -- 172.21.15.105:0/3294898214 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1441589a0 con 0x7fa144160230 2024-04-15T15:21:03.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.789+0000 7fa13affd640 1 -- 172.21.15.105:0/3294898214 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13c004240 con 0x7fa144160230 2024-04-15T15:21:03.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.789+0000 7fa14890d640 1 -- 172.21.15.105:0/3294898214 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa144158f60 con 0x7fa144160230 2024-04-15T15:21:03.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.790+0000 7fa14890d640 1 -- 172.21.15.105:0/3294898214 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa144068a40 con 0x7fa144160230 2024-04-15T15:21:03.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.791+0000 7fa13affd640 1 -- 172.21.15.105:0/3294898214 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fa13c0043e0 con 0x7fa144160230 2024-04-15T15:21:03.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.791+0000 7fa13affd640 1 --2- 172.21.15.105:0/3294898214 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fa12406c9b0 0x7fa12406ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:03.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.791+0000 7fa13affd640 1 -- 172.21.15.105:0/3294898214 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5795+0+0 (secure 0 0 0) 0x7fa13c08c790 con 0x7fa144160230 2024-04-15T15:21:03.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.791+0000 7fa14259c640 1 --2- 172.21.15.105:0/3294898214 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fa12406c9b0 0x7fa12406ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:03.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.792+0000 7fa14259c640 1 --2- 172.21.15.105:0/3294898214 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fa12406c9b0 0x7fa12406ee70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa134036000 tx=0x7fa13403b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:03.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:03.795+0000 7fa13affd640 1 -- 172.21.15.105:0/3294898214 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa13c05aaf0 con 0x7fa144160230 2024-04-15T15:21:04.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.009+0000 7fa14890d640 1 -- 172.21.15.105:0/3294898214 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fa14415ff50 con 0x7fa144160230 2024-04-15T15:21:04.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.666+0000 7fa13affd640 1 -- 172.21.15.105:0/3294898214 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v16) ==== 162+0+0 (secure 0 0 0) 0x7fa13c05a3f0 con 0x7fa144160230 2024-04-15T15:21:04.668 INFO:teuthology.orchestra.run.smithi105.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-15T15:21:04.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.669+0000 7fa14890d640 1 -- 172.21.15.105:0/3294898214 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fa12406c9b0 msgr2=0x7fa12406ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:04.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.669+0000 7fa14890d640 1 --2- 172.21.15.105:0/3294898214 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fa12406c9b0 0x7fa12406ee70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa134036000 tx=0x7fa13403b040 comp rx=0 tx=0).stop 2024-04-15T15:21:04.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.669+0000 7fa14890d640 1 -- 172.21.15.105:0/3294898214 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa144160230 msgr2=0x7fa14415a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:04.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.669+0000 7fa14890d640 1 --2- 172.21.15.105:0/3294898214 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa144160230 0x7fa14415a410 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fa13c00cd70 tx=0x7fa13c00a660 comp rx=0 tx=0).stop 2024-04-15T15:21:04.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.669+0000 7fa14890d640 1 -- 172.21.15.105:0/3294898214 shutdown_connections 2024-04-15T15:21:04.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.669+0000 7fa14890d640 1 --2- 172.21.15.105:0/3294898214 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fa12406c9b0 0x7fa12406ee70 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:04.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.669+0000 7fa14890d640 1 --2- 172.21.15.105:0/3294898214 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa144160230 0x7fa14415a410 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:04.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.669+0000 7fa14890d640 1 --2- 172.21.15.105:0/3294898214 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa14415e320 0x7fa144159760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:04.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.669+0000 7fa14890d640 1 --2- 172.21.15.105:0/3294898214 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1440ce330 0x7fa144155a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:04.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.669+0000 7fa14890d640 1 -- 172.21.15.105:0/3294898214 >> 172.21.15.105:0/3294898214 conn(0x7fa14406b9c0 msgr2=0x7fa144145440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:04.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.669+0000 7fa14890d640 1 -- 172.21.15.105:0/3294898214 shutdown_connections 2024-04-15T15:21:04.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.669+0000 7fa14890d640 1 -- 172.21.15.105:0/3294898214 wait complete. 2024-04-15T15:21:04.687 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-15T15:21:04.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.831+0000 7f2e41952640 1 Processor -- start 2024-04-15T15:21:04.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.831+0000 7f2e41952640 1 -- start start 2024-04-15T15:21:04.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.832+0000 7f2e41952640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2e3c15eb60 0x7f2e3c0d11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:04.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.832+0000 7f2e41952640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e3c15ef40 con 0x7f2e3c162d40 2024-04-15T15:21:04.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.832+0000 7f2e41952640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e3c163260 con 0x7f2e3c0d1710 2024-04-15T15:21:04.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.832+0000 7f2e41952640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f2e3c0d1b60 con 0x7f2e3c15eb60 2024-04-15T15:21:04.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.832+0000 7f2e3affd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2e3c15eb60 0x7f2e3c0d11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:04.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.832+0000 7f2e3affd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2e3c15eb60 0x7f2e3c0d11d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46666/0 (socket says 172.21.15.105:46666) 2024-04-15T15:21:04.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.832+0000 7f2e3affd640 1 -- 172.21.15.105:0/1226178698 learned_addr learned my addr 172.21.15.105:0/1226178698 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:04.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.832+0000 7f2e3affd640 1 -- 172.21.15.105:0/1226178698 >> v1:172.21.15.154:6789/0 conn(0x7f2e3c0d1710 legacy=0x7f2e3c162630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:04.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.832+0000 7f2e3affd640 1 -- 172.21.15.105:0/1226178698 >> v1:172.21.15.105:6789/0 conn(0x7f2e3c162d40 legacy=0x7f2e3c0d3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:04.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.832+0000 7f2e3affd640 1 -- 172.21.15.105:0/1226178698 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e3c0d2970 con 0x7f2e3c15eb60 2024-04-15T15:21:04.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.833+0000 7f2e3affd640 1 --2- 172.21.15.105:0/1226178698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e3c15eb60 0x7f2e3c0d11d0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c009870 tx=0x7f2e2c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=881984f2e66b1e64 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:04.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.833+0000 7f2e3a7fc640 1 -- 172.21.15.105:0/1226178698 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c038070 con 0x7f2e3c15eb60 2024-04-15T15:21:04.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.833+0000 7f2e3a7fc640 1 -- 172.21.15.105:0/1226178698 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e2c031740 con 0x7f2e3c15eb60 2024-04-15T15:21:04.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.833+0000 7f2e3a7fc640 1 -- 172.21.15.105:0/1226178698 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c02f050 con 0x7f2e3c15eb60 2024-04-15T15:21:04.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.834+0000 7f2e41952640 1 -- 172.21.15.105:0/1226178698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e3c15eb60 msgr2=0x7f2e3c0d11d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:04.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.834+0000 7f2e41952640 1 --2- 172.21.15.105:0/1226178698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e3c15eb60 0x7f2e3c0d11d0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c009870 tx=0x7f2e2c02b860 comp rx=0 tx=0).stop 2024-04-15T15:21:04.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.834+0000 7f2e41952640 1 -- 172.21.15.105:0/1226178698 shutdown_connections 2024-04-15T15:21:04.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.834+0000 7f2e41952640 1 --2- 172.21.15.105:0/1226178698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e3c15eb60 0x7f2e3c0d11d0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:04.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.834+0000 7f2e41952640 1 -- 172.21.15.105:0/1226178698 >> 172.21.15.105:0/1226178698 conn(0x7f2e3c067720 msgr2=0x7f2e3c067b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:04.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.834+0000 7f2e41952640 1 -- 172.21.15.105:0/1226178698 shutdown_connections 2024-04-15T15:21:04.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.834+0000 7f2e41952640 1 -- 172.21.15.105:0/1226178698 wait complete. 2024-04-15T15:21:04.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.835+0000 7f2e41952640 1 Processor -- start 2024-04-15T15:21:04.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.835+0000 7f2e41952640 1 -- start start 2024-04-15T15:21:04.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.836+0000 7f2e41952640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e3c0d1710 0x7f2e3c0d49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:04.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.836+0000 7f2e41952640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e3c15eb60 0x7f2e3c15fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:04.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.836+0000 7f2e41952640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e3c162d40 0x7f2e3c1f7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:04.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.836+0000 7f2e41952640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2e3c0d2fc0 con 0x7f2e3c0d1710 2024-04-15T15:21:04.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.836+0000 7f2e41952640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2e3c163540 con 0x7f2e3c15eb60 2024-04-15T15:21:04.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.836+0000 7f2e41952640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f2e3c0b78f0 con 0x7f2e3c162d40 2024-04-15T15:21:04.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.836+0000 7f2e3affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e3c0d1710 0x7f2e3c0d49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:04.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.836+0000 7f2e3affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e3c0d1710 0x7f2e3c0d49a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46668/0 (socket says 172.21.15.105:46668) 2024-04-15T15:21:04.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.836+0000 7f2e3affd640 1 -- 172.21.15.105:0/3095761046 learned_addr learned my addr 172.21.15.105:0/3095761046 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:04.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.836+0000 7f2e3affd640 1 -- 172.21.15.105:0/3095761046 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e3c162d40 msgr2=0x7f2e3c1f7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:04.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.836+0000 7f2e3affd640 1 --2- 172.21.15.105:0/3095761046 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e3c162d40 0x7f2e3c1f7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:04.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.836+0000 7f2e3affd640 1 -- 172.21.15.105:0/3095761046 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e3c15eb60 msgr2=0x7f2e3c15fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:04.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.836+0000 7f2e3affd640 1 --2- 172.21.15.105:0/3095761046 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e3c15eb60 0x7f2e3c15fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:04.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.836+0000 7f2e3affd640 1 -- 172.21.15.105:0/3095761046 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e3c1604b0 con 0x7f2e3c0d1710 2024-04-15T15:21:04.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.836+0000 7f2e3affd640 1 --2- 172.21.15.105:0/3095761046 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e3c0d1710 0x7f2e3c0d49a0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c000c00 tx=0x7f2e2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:04.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.836+0000 7f2e38ff9640 1 -- 172.21.15.105:0/3095761046 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c042070 con 0x7f2e3c0d1710 2024-04-15T15:21:04.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.837+0000 7f2e38ff9640 1 -- 172.21.15.105:0/3095761046 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e2c033a70 con 0x7f2e3c0d1710 2024-04-15T15:21:04.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.837+0000 7f2e41952640 1 -- 172.21.15.105:0/3095761046 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e3c160790 con 0x7f2e3c0d1710 2024-04-15T15:21:04.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.837+0000 7f2e38ff9640 1 -- 172.21.15.105:0/3095761046 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c038040 con 0x7f2e3c0d1710 2024-04-15T15:21:04.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.837+0000 7f2e41952640 1 -- 172.21.15.105:0/3095761046 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e3c160d50 con 0x7f2e3c0d1710 2024-04-15T15:21:04.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.837+0000 7f2e41952640 1 -- 172.21.15.105:0/3095761046 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e3c068a40 con 0x7f2e3c0d1710 2024-04-15T15:21:04.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.838+0000 7f2e38ff9640 1 -- 172.21.15.105:0/3095761046 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f2e2c045070 con 0x7f2e3c0d1710 2024-04-15T15:21:04.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.838+0000 7f2e38ff9640 1 --2- 172.21.15.105:0/3095761046 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f2e1c06c980 0x7f2e1c06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:04.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.838+0000 7f2e38ff9640 1 -- 172.21.15.105:0/3095761046 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5795+0+0 (secure 0 0 0) 0x7f2e2c0adaa0 con 0x7f2e3c0d1710 2024-04-15T15:21:04.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.839+0000 7f2e33fff640 1 --2- 172.21.15.105:0/3095761046 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f2e1c06c980 0x7f2e1c06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:04.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.839+0000 7f2e33fff640 1 --2- 172.21.15.105:0/3095761046 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f2e1c06c980 0x7f2e1c06ee40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2e24000f60 tx=0x7f2e240079d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:04.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:04.843+0000 7f2e38ff9640 1 -- 172.21.15.105:0/3095761046 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2e2c04c030 con 0x7f2e3c0d1710 2024-04-15T15:21:05.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:05.057+0000 7f2e41952640 1 -- 172.21.15.105:0/3095761046 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2e3c15d5d0 con 0x7f2e3c0d1710 2024-04-15T15:21:05.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:05.057+0000 7f2e38ff9640 1 -- 172.21.15.105:0/3095761046 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5177 (secure 0 0 0) 0x7f2e2c045e40 con 0x7f2e3c0d1710 2024-04-15T15:21:05.059 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:21:05.059 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4778,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/4848508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":4848508},{"type":"v1","addr":"172.21.15.154:6835","nonce":4848508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5005,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/1127070021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1127070021},{"type":"v1","addr":"172.21.15.154:6833","nonce":1127070021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5193,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6836/3825669416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3825669416},{"type":"v1","addr":"172.21.15.105:6836","nonce":3825669416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-15T15:20:29.000616+0000","modified":"2024-04-15T15:21:04.669831+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5712":{"gid":5712,"name":"a","rank":0,"incarnation":17,"state":"up:replay","state_seq":1,"addr":"172.21.15.105:6837/4212477415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":4212477415},{"type":"v1","addr":"172.21.15.105:6837","nonce":4212477415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-04-15T15:21:05.060 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 17 2024-04-15T15:21:05.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:05.059+0000 7f2e41952640 1 -- 172.21.15.105:0/3095761046 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f2e1c06c980 msgr2=0x7f2e1c06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:05.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:05.059+0000 7f2e41952640 1 --2- 172.21.15.105:0/3095761046 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f2e1c06c980 0x7f2e1c06ee40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2e24000f60 tx=0x7f2e240079d0 comp rx=0 tx=0).stop 2024-04-15T15:21:05.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:05.059+0000 7f2e41952640 1 -- 172.21.15.105:0/3095761046 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e3c0d1710 msgr2=0x7f2e3c0d49a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:05.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:05.059+0000 7f2e41952640 1 --2- 172.21.15.105:0/3095761046 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e3c0d1710 0x7f2e3c0d49a0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c000c00 tx=0x7f2e2c0046c0 comp rx=0 tx=0).stop 2024-04-15T15:21:05.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:05.059+0000 7f2e41952640 1 -- 172.21.15.105:0/3095761046 shutdown_connections 2024-04-15T15:21:05.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:05.059+0000 7f2e41952640 1 --2- 172.21.15.105:0/3095761046 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e3c162d40 0x7f2e3c1f7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:05.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:05.059+0000 7f2e41952640 1 --2- 172.21.15.105:0/3095761046 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f2e1c06c980 0x7f2e1c06ee40 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:05.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:05.059+0000 7f2e41952640 1 --2- 172.21.15.105:0/3095761046 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e3c15eb60 0x7f2e3c15fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:05.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:05.059+0000 7f2e41952640 1 --2- 172.21.15.105:0/3095761046 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e3c0d1710 0x7f2e3c0d49a0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:05.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:05.060+0000 7f2e41952640 1 -- 172.21.15.105:0/3095761046 >> 172.21.15.105:0/3095761046 conn(0x7f2e3c067720 msgr2=0x7f2e3c0c87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:05.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:05.060+0000 7f2e41952640 1 -- 172.21.15.105:0/3095761046 shutdown_connections 2024-04-15T15:21:05.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:05.060+0000 7f2e41952640 1 -- 172.21.15.105:0/3095761046 wait complete. 2024-04-15T15:21:05.476 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-15T15:20:29.000616+0000', 'modified': '2024-04-15T15:21:04.669831+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 35, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5712}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5712': {'gid': 5712, 'name': 'a', 'rank': 0, 'incarnation': 17, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.105:6837/4212477415', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6835', 'nonce': 4212477415}, {'type': 'v1', 'addr': '172.21.15.105:6837', 'nonce': 4212477415}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-15T15:21:05.476 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5712:up:replay 2024-04-15T15:21:06.477 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-15T15:21:06.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.595+0000 7f29f735f640 1 Processor -- start 2024-04-15T15:21:06.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.595+0000 7f29f735f640 1 -- start start 2024-04-15T15:21:06.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.596+0000 7f29f735f640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f29f0145e00 0x7f29f00ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:06.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.596+0000 7f29f735f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f29f00ced30 0x7f29f0071e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:06.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.596+0000 7f29f735f640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29f00727b0 con 0x7f29f00723a0 2024-04-15T15:21:06.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.596+0000 7f29f735f640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f29f00cdbc0 con 0x7f29f00ced30 2024-04-15T15:21:06.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.596+0000 7f29f735f640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f29f00ccfb0 con 0x7f29f0145e00 2024-04-15T15:21:06.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.596+0000 7f29f48fa640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f29f00ced30 0x7f29f0071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:06.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.596+0000 7f29f58fc640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f29f00723a0 0x7f29f0160230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:50824/0 (socket says 172.21.15.105:50824) 2024-04-15T15:21:06.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.596+0000 7f29f48fa640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f29f00ced30 0x7f29f0071e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46680/0 (socket says 172.21.15.105:46680) 2024-04-15T15:21:06.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.596+0000 7f29f58fc640 1 -- 172.21.15.105:0/4023888104 learned_addr learned my addr 172.21.15.105:0/4023888104 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:21:06.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.596+0000 7f29f48fa640 1 -- 172.21.15.105:0/4023888104 >> v2:172.21.15.154:3300/0 conn(0x7f29f0145e00 msgr2=0x7f29f00ce7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:06.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.596+0000 7f29f48fa640 1 --2- 172.21.15.105:0/4023888104 >> v2:172.21.15.154:3300/0 conn(0x7f29f0145e00 0x7f29f00ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:06.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.596+0000 7f29f48fa640 1 -- 172.21.15.105:0/4023888104 >> v1:172.21.15.105:6789/0 conn(0x7f29f00723a0 legacy=0x7f29f0160230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:06.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.596+0000 7f29f48fa640 1 -- 172.21.15.105:0/4023888104 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29f00cd290 con 0x7f29f00ced30 2024-04-15T15:21:06.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.596+0000 7f29e7fff640 1 -- 172.21.15.105:0/4023888104 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2289400846 0 0) 0x7f29e8001490 con 0x7f29f00723a0 2024-04-15T15:21:06.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.597+0000 7f29f48fa640 1 --2- 172.21.15.105:0/4023888104 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29f00ced30 0x7f29f0071e60 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f29f00b5960 tx=0x7f29e002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abf8a1387061416c server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:06.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.597+0000 7f29e7fff640 1 -- 172.21.15.105:0/4023888104 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e0035020 con 0x7f29f00ced30 2024-04-15T15:21:06.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.597+0000 7f29e7fff640 1 -- 172.21.15.105:0/4023888104 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29e0031830 con 0x7f29f00ced30 2024-04-15T15:21:06.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.597+0000 7f29e7fff640 1 -- 172.21.15.105:0/4023888104 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e0031c00 con 0x7f29f00ced30 2024-04-15T15:21:06.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.598+0000 7f29f735f640 1 -- 172.21.15.105:0/4023888104 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29f00ced30 msgr2=0x7f29f0071e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:06.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.598+0000 7f29f735f640 1 --2- 172.21.15.105:0/4023888104 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29f00ced30 0x7f29f0071e60 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f29f00b5960 tx=0x7f29e002b8d0 comp rx=0 tx=0).stop 2024-04-15T15:21:06.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.598+0000 7f29f735f640 1 -- 172.21.15.105:0/4023888104 shutdown_connections 2024-04-15T15:21:06.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.598+0000 7f29f735f640 1 --2- 172.21.15.105:0/4023888104 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29f00ced30 0x7f29f0071e60 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:06.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.598+0000 7f29f735f640 1 --2- 172.21.15.105:0/4023888104 >> v2:172.21.15.154:3300/0 conn(0x7f29f0145e00 0x7f29f00ce7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:06.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.598+0000 7f29f735f640 1 -- 172.21.15.105:0/4023888104 >> 172.21.15.105:0/4023888104 conn(0x7f29f0067d50 msgr2=0x7f29f006b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:06.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.598+0000 7f29f735f640 1 -- 172.21.15.105:0/4023888104 shutdown_connections 2024-04-15T15:21:06.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.598+0000 7f29f735f640 1 -- 172.21.15.105:0/4023888104 wait complete. 2024-04-15T15:21:06.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.599+0000 7f29f735f640 1 Processor -- start 2024-04-15T15:21:06.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.599+0000 7f29f735f640 1 -- start start 2024-04-15T15:21:06.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.600+0000 7f29f735f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f29f00723a0 0x7f29f00a9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:06.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.600+0000 7f29f735f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f29f00ced30 0x7f29f015a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:06.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.600+0000 7f29f735f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29f0145e00 0x7f29f01448f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:06.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.600+0000 7f29f735f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f29f00ce370 con 0x7f29f0145e00 2024-04-15T15:21:06.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.600+0000 7f29f735f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f29f00cd850 con 0x7f29f00ced30 2024-04-15T15:21:06.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.600+0000 7f29f735f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f29f01461e0 con 0x7f29f00723a0 2024-04-15T15:21:06.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.600+0000 7f29f58fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29f0145e00 0x7f29f01448f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:06.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.600+0000 7f29f58fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29f0145e00 0x7f29f01448f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46686/0 (socket says 172.21.15.105:46686) 2024-04-15T15:21:06.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.600+0000 7f29f58fc640 1 -- 172.21.15.105:0/4216223070 learned_addr learned my addr 172.21.15.105:0/4216223070 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:06.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.600+0000 7f29f48fa640 1 --2- 172.21.15.105:0/4216223070 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f29f00ced30 0x7f29f015a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:06.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.600+0000 7f29f50fb640 1 --2- 172.21.15.105:0/4216223070 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f29f00723a0 0x7f29f00a9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:06.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.600+0000 7f29f58fc640 1 -- 172.21.15.105:0/4216223070 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f29f00723a0 msgr2=0x7f29f00a9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:06.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.600+0000 7f29f58fc640 1 --2- 172.21.15.105:0/4216223070 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f29f00723a0 0x7f29f00a9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:06.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.600+0000 7f29f58fc640 1 -- 172.21.15.105:0/4216223070 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f29f00ced30 msgr2=0x7f29f015a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:06.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.600+0000 7f29f58fc640 1 --2- 172.21.15.105:0/4216223070 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f29f00ced30 0x7f29f015a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:06.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.600+0000 7f29f58fc640 1 -- 172.21.15.105:0/4216223070 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29f015a7f0 con 0x7f29f0145e00 2024-04-15T15:21:06.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.600+0000 7f29f50fb640 1 --2- 172.21.15.105:0/4216223070 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f29f00723a0 0x7f29f00a9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:21:06.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.601+0000 7f29f58fc640 1 --2- 172.21.15.105:0/4216223070 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29f0145e00 0x7f29f01448f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f29e800c730 tx=0x7f29e800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:06.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.601+0000 7f29e5ffb640 1 -- 172.21.15.105:0/4216223070 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e8018070 con 0x7f29f0145e00 2024-04-15T15:21:06.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.601+0000 7f29e5ffb640 1 -- 172.21.15.105:0/4216223070 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29e800ce40 con 0x7f29f0145e00 2024-04-15T15:21:06.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.601+0000 7f29f735f640 1 -- 172.21.15.105:0/4216223070 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29f015aad0 con 0x7f29f0145e00 2024-04-15T15:21:06.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.601+0000 7f29e5ffb640 1 -- 172.21.15.105:0/4216223070 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e800a860 con 0x7f29f0145e00 2024-04-15T15:21:06.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.601+0000 7f29f735f640 1 -- 172.21.15.105:0/4216223070 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29f015b090 con 0x7f29f0145e00 2024-04-15T15:21:06.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.602+0000 7f29f735f640 1 -- 172.21.15.105:0/4216223070 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29f0068a40 con 0x7f29f0145e00 2024-04-15T15:21:06.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.602+0000 7f29e5ffb640 1 -- 172.21.15.105:0/4216223070 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f29e801b070 con 0x7f29f0145e00 2024-04-15T15:21:06.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.603+0000 7f29e5ffb640 1 --2- 172.21.15.105:0/4216223070 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f29d406c980 0x7f29d406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:06.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.603+0000 7f29f50fb640 1 --2- 172.21.15.105:0/4216223070 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f29d406c980 0x7f29d406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:06.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.603+0000 7f29e5ffb640 1 -- 172.21.15.105:0/4216223070 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5795+0+0 (secure 0 0 0) 0x7f29e808b1e0 con 0x7f29f0145e00 2024-04-15T15:21:06.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.604+0000 7f29f50fb640 1 --2- 172.21.15.105:0/4216223070 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f29d406c980 0x7f29d406ee40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f29ec006fd0 tx=0x7f29ec008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:06.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.608+0000 7f29e5ffb640 1 -- 172.21.15.105:0/4216223070 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f29f00727b0 con 0x7f29f0145e00 2024-04-15T15:21:06.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.823+0000 7f29f735f640 1 -- 172.21.15.105:0/4216223070 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f29f006fd50 con 0x7f29f0145e00 2024-04-15T15:21:06.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.823+0000 7f29e5ffb640 1 -- 172.21.15.105:0/4216223070 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+5177 (secure 0 0 0) 0x7f29e80595c0 con 0x7f29f0145e00 2024-04-15T15:21:06.825 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:21:06.825 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4778,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/4848508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":4848508},{"type":"v1","addr":"172.21.15.154:6835","nonce":4848508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5005,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/1127070021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1127070021},{"type":"v1","addr":"172.21.15.154:6833","nonce":1127070021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5193,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6836/3825669416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3825669416},{"type":"v1","addr":"172.21.15.105:6836","nonce":3825669416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-15T15:20:29.000616+0000","modified":"2024-04-15T15:21:05.675125+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5712":{"gid":5712,"name":"a","rank":0,"incarnation":17,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.105:6837/4212477415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":4212477415},{"type":"v1","addr":"172.21.15.105:6837","nonce":4212477415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-04-15T15:21:06.826 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 19 2024-04-15T15:21:06.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.825+0000 7f29f735f640 1 -- 172.21.15.105:0/4216223070 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f29d406c980 msgr2=0x7f29d406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:06.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.825+0000 7f29f735f640 1 --2- 172.21.15.105:0/4216223070 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f29d406c980 0x7f29d406ee40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f29ec006fd0 tx=0x7f29ec008040 comp rx=0 tx=0).stop 2024-04-15T15:21:06.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.825+0000 7f29f735f640 1 -- 172.21.15.105:0/4216223070 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29f0145e00 msgr2=0x7f29f01448f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:06.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.825+0000 7f29f735f640 1 --2- 172.21.15.105:0/4216223070 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29f0145e00 0x7f29f01448f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f29e800c730 tx=0x7f29e800cbf0 comp rx=0 tx=0).stop 2024-04-15T15:21:06.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.825+0000 7f29f735f640 1 -- 172.21.15.105:0/4216223070 shutdown_connections 2024-04-15T15:21:06.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.825+0000 7f29f735f640 1 --2- 172.21.15.105:0/4216223070 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f29d406c980 0x7f29d406ee40 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:06.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.825+0000 7f29f735f640 1 --2- 172.21.15.105:0/4216223070 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29f0145e00 0x7f29f01448f0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:06.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.825+0000 7f29f735f640 1 --2- 172.21.15.105:0/4216223070 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f29f00ced30 0x7f29f015a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:06.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.825+0000 7f29f735f640 1 --2- 172.21.15.105:0/4216223070 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f29f00723a0 0x7f29f00a9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:06.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.825+0000 7f29f735f640 1 -- 172.21.15.105:0/4216223070 >> 172.21.15.105:0/4216223070 conn(0x7f29f0067d50 msgr2=0x7f29f006df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:06.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.825+0000 7f29f735f640 1 -- 172.21.15.105:0/4216223070 shutdown_connections 2024-04-15T15:21:06.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:06.825+0000 7f29f735f640 1 -- 172.21.15.105:0/4216223070 wait complete. 2024-04-15T15:21:07.240 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 19, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-15T15:20:29.000616+0000', 'modified': '2024-04-15T15:21:05.675125+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 35, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5712}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5712': {'gid': 5712, 'name': 'a', 'rank': 0, 'incarnation': 17, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.105:6837/4212477415', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6835', 'nonce': 4212477415}, {'type': 'v1', 'addr': '172.21.15.105:6837', 'nonce': 4212477415}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-15T15:21:07.240 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5712:up:rejoin 2024-04-15T15:21:08.241 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-15T15:21:08.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.354+0000 7f3df27e9640 1 Processor -- start 2024-04-15T15:21:08.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.354+0000 7f3df27e9640 1 -- start start 2024-04-15T15:21:08.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.355+0000 7f3df27e9640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3dec15ebf0 0x7f3dec0b58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:08.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.355+0000 7f3df27e9640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3dec15efd0 con 0x7f3dec072d00 2024-04-15T15:21:08.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.355+0000 7f3df27e9640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3dec06baf0 con 0x7f3dec0b5e00 2024-04-15T15:21:08.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.355+0000 7f3df27e9640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f3dec06bd90 con 0x7f3dec15ebf0 2024-04-15T15:21:08.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.355+0000 7f3df0d86640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f3dec072d00 0x7f3dec0b3dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:50838/0 (socket says 172.21.15.105:50838) 2024-04-15T15:21:08.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.355+0000 7f3df0d86640 1 -- 172.21.15.105:0/1629095659 learned_addr learned my addr 172.21.15.105:0/1629095659 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:21:08.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.355+0000 7f3deaffd640 1 -- 172.21.15.105:0/1629095659 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1175963775 0 0) 0x7f3dec15efd0 con 0x7f3dec072d00 2024-04-15T15:21:08.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.355+0000 7f3deaffd640 1 -- 172.21.15.105:0/1629095659 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3dd4003520 con 0x7f3dec072d00 2024-04-15T15:21:08.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.355+0000 7f3debfff640 1 --2- 172.21.15.105:0/1629095659 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3dec15ebf0 0x7f3dec0b58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:08.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.356+0000 7f3deaffd640 1 -- 172.21.15.105:0/1629095659 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1603703826 0 0) 0x7f3dd4003520 con 0x7f3dec072d00 2024-04-15T15:21:08.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.356+0000 7f3deaffd640 1 -- 172.21.15.105:0/1629095659 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3dec15ebf0 msgr2=0x7f3dec0b58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:08.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.356+0000 7f3deaffd640 1 --2- 172.21.15.105:0/1629095659 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3dec15ebf0 0x7f3dec0b58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:08.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.356+0000 7f3deaffd640 1 -- 172.21.15.105:0/1629095659 >> v1:172.21.15.154:6789/0 conn(0x7f3dec0b5e00 legacy=0x7f3dec0725f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:08.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.356+0000 7f3deaffd640 1 -- 172.21.15.105:0/1629095659 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dec06c070 con 0x7f3dec072d00 2024-04-15T15:21:08.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.356+0000 7f3deaffd640 1 -- 172.21.15.105:0/1629095659 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7f3de40019f0 con 0x7f3dec072d00 2024-04-15T15:21:08.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.356+0000 7f3deaffd640 1 -- 172.21.15.105:0/1629095659 >> v1:172.21.15.105:6789/0 conn(0x7f3dec072d00 legacy=0x7f3dec0b3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:08.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.356+0000 7f3deaffd640 1 --2- 172.21.15.105:0/1629095659 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3dd40039e0 0x7f3dd4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:08.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.356+0000 7f3deaffd640 1 --2- 172.21.15.105:0/1629095659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3dd4004800 0x7f3dd4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:08.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.356+0000 7f3deaffd640 1 --2- 172.21.15.105:0/1629095659 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3dec15ebf0 0x7f3dd4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:08.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.356+0000 7f3deaffd640 1 -- 172.21.15.105:0/1629095659 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3dec06bd90 con 0x7f3dd40039e0 2024-04-15T15:21:08.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.356+0000 7f3deaffd640 1 -- 172.21.15.105:0/1629095659 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd400baf0 con 0x7f3dd4004800 2024-04-15T15:21:08.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.356+0000 7f3deaffd640 1 -- 172.21.15.105:0/1629095659 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f3dd400bcf0 con 0x7f3dec15ebf0 2024-04-15T15:21:08.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.356+0000 7f3debfff640 1 --2- 172.21.15.105:0/1629095659 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3dd40039e0 0x7f3dd4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:08.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.356+0000 7f3df0d86640 1 --2- 172.21.15.105:0/1629095659 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3dec15ebf0 0x7f3dd4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:08.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.356+0000 7f3deb7fe640 1 --2- 172.21.15.105:0/1629095659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3dd4004800 0x7f3dd4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:08.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.356+0000 7f3debfff640 1 -- 172.21.15.105:0/1629095659 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3dec15ebf0 msgr2=0x7f3dd4005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:08.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.356+0000 7f3debfff640 1 --2- 172.21.15.105:0/1629095659 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3dec15ebf0 0x7f3dd4005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:08.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.356+0000 7f3debfff640 1 -- 172.21.15.105:0/1629095659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3dd4004800 msgr2=0x7f3dd4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:08.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.356+0000 7f3debfff640 1 --2- 172.21.15.105:0/1629095659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3dd4004800 0x7f3dd4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:08.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.356+0000 7f3debfff640 1 -- 172.21.15.105:0/1629095659 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dd4003520 con 0x7f3dd40039e0 2024-04-15T15:21:08.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.357+0000 7f3debfff640 1 --2- 172.21.15.105:0/1629095659 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3dd40039e0 0x7f3dd4003e00 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc009ac0 tx=0x7f3dd4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8cb6c51e33e0009 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:08.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.357+0000 7f3deaffd640 1 -- 172.21.15.105:0/1629095659 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc039070 con 0x7f3dd40039e0 2024-04-15T15:21:08.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.357+0000 7f3deaffd640 1 -- 172.21.15.105:0/1629095659 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ddc02c6b0 con 0x7f3dd40039e0 2024-04-15T15:21:08.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.357+0000 7f3deaffd640 1 -- 172.21.15.105:0/1629095659 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc009690 con 0x7f3dd40039e0 2024-04-15T15:21:08.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.358+0000 7f3df27e9640 1 -- 172.21.15.105:0/1629095659 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3dd40039e0 msgr2=0x7f3dd4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:08.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.358+0000 7f3df27e9640 1 --2- 172.21.15.105:0/1629095659 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3dd40039e0 0x7f3dd4003e00 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc009ac0 tx=0x7f3dd4005830 comp rx=0 tx=0).stop 2024-04-15T15:21:08.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.358+0000 7f3df0d86640 1 -- 172.21.15.105:0/1629095659 reap_dead start 2024-04-15T15:21:08.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.358+0000 7f3df27e9640 1 -- 172.21.15.105:0/1629095659 shutdown_connections 2024-04-15T15:21:08.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.358+0000 7f3df27e9640 1 -- 172.21.15.105:0/1629095659 >> 172.21.15.105:0/1629095659 conn(0x7f3dec0d0d00 msgr2=0x7f3dec0d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:08.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.358+0000 7f3df27e9640 1 -- 172.21.15.105:0/1629095659 shutdown_connections 2024-04-15T15:21:08.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.358+0000 7f3df27e9640 1 -- 172.21.15.105:0/1629095659 wait complete. 2024-04-15T15:21:08.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.359+0000 7f3df27e9640 1 Processor -- start 2024-04-15T15:21:08.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.359+0000 7f3df27e9640 1 -- start start 2024-04-15T15:21:08.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.360+0000 7f3df27e9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3dd40039e0 0x7f3dec15bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:08.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.360+0000 7f3df27e9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3dec14c560 0x7f3dec161410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:08.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.360+0000 7f3df27e9640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3dec0cc1e0 0x7f3dec0cc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:08.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.360+0000 7f3df27e9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3dec073870 con 0x7f3dec14c560 2024-04-15T15:21:08.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.360+0000 7f3df27e9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3dec06c630 con 0x7f3dd40039e0 2024-04-15T15:21:08.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.360+0000 7f3df27e9640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f3dec0b58c0 con 0x7f3dec0cc1e0 2024-04-15T15:21:08.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.360+0000 7f3deb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3dec14c560 0x7f3dec161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:08.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.360+0000 7f3deb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3dec14c560 0x7f3dec161410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46706/0 (socket says 172.21.15.105:46706) 2024-04-15T15:21:08.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.360+0000 7f3deb7fe640 1 -- 172.21.15.105:0/3852616456 learned_addr learned my addr 172.21.15.105:0/3852616456 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:08.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.360+0000 7f3df0d86640 1 --2- 172.21.15.105:0/3852616456 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3dec0cc1e0 0x7f3dec0cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:08.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.360+0000 7f3debfff640 1 --2- 172.21.15.105:0/3852616456 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3dd40039e0 0x7f3dec15bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:08.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.360+0000 7f3deb7fe640 1 -- 172.21.15.105:0/3852616456 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3dec0cc1e0 msgr2=0x7f3dec0cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:08.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.360+0000 7f3deb7fe640 1 --2- 172.21.15.105:0/3852616456 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3dec0cc1e0 0x7f3dec0cc640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:08.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.360+0000 7f3deb7fe640 1 -- 172.21.15.105:0/3852616456 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3dd40039e0 msgr2=0x7f3dec15bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:08.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.360+0000 7f3deb7fe640 1 --2- 172.21.15.105:0/3852616456 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3dd40039e0 0x7f3dec15bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:08.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.360+0000 7f3deb7fe640 1 -- 172.21.15.105:0/3852616456 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dec14ca30 con 0x7f3dec14c560 2024-04-15T15:21:08.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.360+0000 7f3df0d86640 1 --2- 172.21.15.105:0/3852616456 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3dec0cc1e0 0x7f3dec0cc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:21:08.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.360+0000 7f3debfff640 1 --2- 172.21.15.105:0/3852616456 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3dd40039e0 0x7f3dec15bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:21:08.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.360+0000 7f3deb7fe640 1 --2- 172.21.15.105:0/3852616456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3dec14c560 0x7f3dec161410 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f3de000e490 tx=0x7f3de000e950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:08.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.361+0000 7f3de8ff9640 1 -- 172.21.15.105:0/3852616456 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de00052e0 con 0x7f3dec14c560 2024-04-15T15:21:08.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.361+0000 7f3de8ff9640 1 -- 172.21.15.105:0/3852616456 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3de0018070 con 0x7f3dec14c560 2024-04-15T15:21:08.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.361+0000 7f3df27e9640 1 -- 172.21.15.105:0/3852616456 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dec14cd10 con 0x7f3dec14c560 2024-04-15T15:21:08.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.361+0000 7f3de8ff9640 1 -- 172.21.15.105:0/3852616456 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de0003810 con 0x7f3dec14c560 2024-04-15T15:21:08.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.361+0000 7f3df27e9640 1 -- 172.21.15.105:0/3852616456 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dec14d2d0 con 0x7f3dec14c560 2024-04-15T15:21:08.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.362+0000 7f3df27e9640 1 -- 172.21.15.105:0/3852616456 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dec14a590 con 0x7f3dec14c560 2024-04-15T15:21:08.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.362+0000 7f3de8ff9640 1 -- 172.21.15.105:0/3852616456 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f3de00039b0 con 0x7f3dec14c560 2024-04-15T15:21:08.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.363+0000 7f3de8ff9640 1 --2- 172.21.15.105:0/3852616456 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f3dd806c980 0x7f3dd806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:08.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.363+0000 7f3de8ff9640 1 -- 172.21.15.105:0/3852616456 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5795+0+0 (secure 0 0 0) 0x7f3de008b850 con 0x7f3dec14c560 2024-04-15T15:21:08.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.363+0000 7f3debfff640 1 --2- 172.21.15.105:0/3852616456 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f3dd806c980 0x7f3dd806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:08.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.363+0000 7f3debfff640 1 --2- 172.21.15.105:0/3852616456 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f3dd806c980 0x7f3dd806ee40 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc036000 tx=0x7f3ddc030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:08.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.367+0000 7f3de8ff9640 1 -- 172.21.15.105:0/3852616456 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3dec14a590 con 0x7f3dec14c560 2024-04-15T15:21:08.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.577+0000 7f3df27e9640 1 -- 172.21.15.105:0/3852616456 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3dec068a40 con 0x7f3dec14c560 2024-04-15T15:21:08.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.578+0000 7f3de8ff9640 1 -- 172.21.15.105:0/3852616456 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+5184 (secure 0 0 0) 0x7f3de0059c30 con 0x7f3dec14c560 2024-04-15T15:21:08.580 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:21:08.580 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4778,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/4848508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":4848508},{"type":"v1","addr":"172.21.15.154:6835","nonce":4848508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5005,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/1127070021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1127070021},{"type":"v1","addr":"172.21.15.154:6833","nonce":1127070021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5193,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6836/3825669416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3825669416},{"type":"v1","addr":"172.21.15.105:6836","nonce":3825669416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-15T15:20:29.000616+0000","modified":"2024-04-15T15:21:07.677485+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5712":{"gid":5712,"name":"a","rank":0,"incarnation":17,"state":"up:active","state_seq":4,"addr":"172.21.15.105:6837/4212477415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":4212477415},{"type":"v1","addr":"172.21.15.105:6837","nonce":4212477415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5712,"qdb_cluster":[5712]},"id":2}]} 2024-04-15T15:21:08.580 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 20 2024-04-15T15:21:08.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.580+0000 7f3df27e9640 1 -- 172.21.15.105:0/3852616456 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f3dd806c980 msgr2=0x7f3dd806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:08.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.580+0000 7f3df27e9640 1 --2- 172.21.15.105:0/3852616456 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f3dd806c980 0x7f3dd806ee40 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc036000 tx=0x7f3ddc030000 comp rx=0 tx=0).stop 2024-04-15T15:21:08.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.580+0000 7f3df27e9640 1 -- 172.21.15.105:0/3852616456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3dec14c560 msgr2=0x7f3dec161410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:08.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.580+0000 7f3df27e9640 1 --2- 172.21.15.105:0/3852616456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3dec14c560 0x7f3dec161410 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f3de000e490 tx=0x7f3de000e950 comp rx=0 tx=0).stop 2024-04-15T15:21:08.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.580+0000 7f3df27e9640 1 -- 172.21.15.105:0/3852616456 shutdown_connections 2024-04-15T15:21:08.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.580+0000 7f3df27e9640 1 --2- 172.21.15.105:0/3852616456 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3dec0cc1e0 0x7f3dec0cc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:08.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.580+0000 7f3df27e9640 1 --2- 172.21.15.105:0/3852616456 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f3dd806c980 0x7f3dd806ee40 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:08.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.580+0000 7f3df27e9640 1 --2- 172.21.15.105:0/3852616456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3dec14c560 0x7f3dec161410 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:08.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.580+0000 7f3df27e9640 1 --2- 172.21.15.105:0/3852616456 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3dd40039e0 0x7f3dec15bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:08.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.580+0000 7f3df27e9640 1 -- 172.21.15.105:0/3852616456 >> 172.21.15.105:0/3852616456 conn(0x7f3dec0d0d00 msgr2=0x7f3dec0b0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:08.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.580+0000 7f3df27e9640 1 -- 172.21.15.105:0/3852616456 shutdown_connections 2024-04-15T15:21:08.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:08.580+0000 7f3df27e9640 1 -- 172.21.15.105:0/3852616456 wait complete. 2024-04-15T15:21:08.995 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-15T15:20:29.000616+0000', 'modified': '2024-04-15T15:21:07.677485+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 35, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5712}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5712': {'gid': 5712, 'name': 'a', 'rank': 0, 'incarnation': 17, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.105:6837/4212477415', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6835', 'nonce': 4212477415}, {'type': 'v1', 'addr': '172.21.15.105:6837', 'nonce': 4212477415}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5712, 'qdb_cluster': [5712]} 2024-04-15T15:21:08.995 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-15T15:21:08.995 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-15T15:21:09.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.097+0000 7f9e502d9640 1 Processor -- start 2024-04-15T15:21:09.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.097+0000 7f9e502d9640 1 -- start start 2024-04-15T15:21:09.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.098+0000 7f9e502d9640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9e4815eb60 0x7f9e480d11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:09.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.098+0000 7f9e502d9640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e48162d40 0x7f9e480d3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:09.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.098+0000 7f9e502d9640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e4815ef40 con 0x7f9e480d1710 2024-04-15T15:21:09.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.098+0000 7f9e502d9640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f9e480d1b60 con 0x7f9e4815eb60 2024-04-15T15:21:09.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.098+0000 7f9e502d9640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f9e48163540 con 0x7f9e48162d40 2024-04-15T15:21:09.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.099+0000 7f9e4e075640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9e4815eb60 0x7f9e480d11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:09.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.099+0000 7f9e4d874640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f9e480d1710 0x7f9e48162630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:50850/0 (socket says 172.21.15.105:50850) 2024-04-15T15:21:09.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.099+0000 7f9e4d874640 1 -- 172.21.15.105:0/3125569323 learned_addr learned my addr 172.21.15.105:0/3125569323 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:21:09.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.099+0000 7f9e4e876640 1 --2- 172.21.15.105:0/3125569323 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e48162d40 0x7f9e480d3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:09.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.099+0000 7f9e4e075640 1 -- 172.21.15.105:0/3125569323 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e48162d40 msgr2=0x7f9e480d3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:09.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.099+0000 7f9e4e075640 1 --2- 172.21.15.105:0/3125569323 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e48162d40 0x7f9e480d3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:09.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.099+0000 7f9e4e075640 1 -- 172.21.15.105:0/3125569323 >> v1:172.21.15.105:6789/0 conn(0x7f9e480d1710 legacy=0x7f9e48162630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:09.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.099+0000 7f9e4d073640 1 -- 172.21.15.105:0/3125569323 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4082882197 0 0) 0x7f9e4815ef40 con 0x7f9e480d1710 2024-04-15T15:21:09.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.099+0000 7f9e4e075640 1 -- 172.21.15.105:0/3125569323 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e48163260 con 0x7f9e4815eb60 2024-04-15T15:21:09.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.099+0000 7f9e4e876640 1 --2- 172.21.15.105:0/3125569323 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e48162d40 0x7f9e480d3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:21:09.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.100+0000 7f9e4e075640 1 --2- 172.21.15.105:0/3125569323 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e4815eb60 0x7f9e480d11d0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f9e440097a0 tx=0x7f9e4402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7062693cd884fd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:09.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.100+0000 7f9e4d073640 1 -- 172.21.15.105:0/3125569323 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e44035020 con 0x7f9e4815eb60 2024-04-15T15:21:09.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.100+0000 7f9e4d073640 1 -- 172.21.15.105:0/3125569323 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e44031830 con 0x7f9e4815eb60 2024-04-15T15:21:09.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.100+0000 7f9e4d073640 1 -- 172.21.15.105:0/3125569323 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e44031c00 con 0x7f9e4815eb60 2024-04-15T15:21:09.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.100+0000 7f9e502d9640 1 -- 172.21.15.105:0/3125569323 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e4815eb60 msgr2=0x7f9e480d11d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:09.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.100+0000 7f9e502d9640 1 --2- 172.21.15.105:0/3125569323 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e4815eb60 0x7f9e480d11d0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f9e440097a0 tx=0x7f9e4402b8d0 comp rx=0 tx=0).stop 2024-04-15T15:21:09.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.100+0000 7f9e502d9640 1 -- 172.21.15.105:0/3125569323 shutdown_connections 2024-04-15T15:21:09.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.100+0000 7f9e502d9640 1 --2- 172.21.15.105:0/3125569323 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e48162d40 0x7f9e480d3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:09.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.100+0000 7f9e502d9640 1 --2- 172.21.15.105:0/3125569323 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e4815eb60 0x7f9e480d11d0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:09.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.100+0000 7f9e502d9640 1 -- 172.21.15.105:0/3125569323 >> 172.21.15.105:0/3125569323 conn(0x7f9e48067720 msgr2=0x7f9e48067b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:09.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.100+0000 7f9e502d9640 1 -- 172.21.15.105:0/3125569323 shutdown_connections 2024-04-15T15:21:09.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.101+0000 7f9e502d9640 1 -- 172.21.15.105:0/3125569323 wait complete. 2024-04-15T15:21:09.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.102+0000 7f9e502d9640 1 Processor -- start 2024-04-15T15:21:09.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.102+0000 7f9e502d9640 1 -- start start 2024-04-15T15:21:09.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.102+0000 7f9e502d9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e480d1710 0x7f9e4815bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:09.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.102+0000 7f9e502d9640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e4815eb60 0x7f9e480cb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:09.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.102+0000 7f9e502d9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e48162d40 0x7f9e480cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:09.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.102+0000 7f9e502d9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9e480d2c50 con 0x7f9e480d1710 2024-04-15T15:21:09.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.102+0000 7f9e502d9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9e480d2fc0 con 0x7f9e48162d40 2024-04-15T15:21:09.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.102+0000 7f9e502d9640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f9e480b78f0 con 0x7f9e4815eb60 2024-04-15T15:21:09.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.102+0000 7f9e4e075640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e480d1710 0x7f9e4815bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:09.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.102+0000 7f9e4e075640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e480d1710 0x7f9e4815bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46716/0 (socket says 172.21.15.105:46716) 2024-04-15T15:21:09.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.102+0000 7f9e4e075640 1 -- 172.21.15.105:0/3800729023 learned_addr learned my addr 172.21.15.105:0/3800729023 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:09.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.103+0000 7f9e4d874640 1 --2- 172.21.15.105:0/3800729023 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e4815eb60 0x7f9e480cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:09.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.103+0000 7f9e4e876640 1 --2- 172.21.15.105:0/3800729023 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e48162d40 0x7f9e480cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:09.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.103+0000 7f9e4e075640 1 -- 172.21.15.105:0/3800729023 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e4815eb60 msgr2=0x7f9e480cb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:09.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.103+0000 7f9e4e075640 1 --2- 172.21.15.105:0/3800729023 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e4815eb60 0x7f9e480cb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:09.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.103+0000 7f9e4e075640 1 -- 172.21.15.105:0/3800729023 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e48162d40 msgr2=0x7f9e480cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:09.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.103+0000 7f9e4e075640 1 --2- 172.21.15.105:0/3800729023 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e48162d40 0x7f9e480cbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:09.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.103+0000 7f9e4e075640 1 -- 172.21.15.105:0/3800729023 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e480b9410 con 0x7f9e480d1710 2024-04-15T15:21:09.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.103+0000 7f9e4e075640 1 --2- 172.21.15.105:0/3800729023 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e480d1710 0x7f9e4815bc90 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f9e44009890 tx=0x7f9e440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:09.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.103+0000 7f9e3affd640 1 -- 172.21.15.105:0/3800729023 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e44041070 con 0x7f9e480d1710 2024-04-15T15:21:09.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.103+0000 7f9e3affd640 1 -- 172.21.15.105:0/3800729023 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e4403b920 con 0x7f9e480d1710 2024-04-15T15:21:09.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.103+0000 7f9e3affd640 1 -- 172.21.15.105:0/3800729023 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e44035040 con 0x7f9e480d1710 2024-04-15T15:21:09.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.103+0000 7f9e502d9640 1 -- 172.21.15.105:0/3800729023 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e480cc410 con 0x7f9e480d1710 2024-04-15T15:21:09.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.103+0000 7f9e502d9640 1 -- 172.21.15.105:0/3800729023 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e480cc9d0 con 0x7f9e480d1710 2024-04-15T15:21:09.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.105+0000 7f9e3affd640 1 -- 172.21.15.105:0/3800729023 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f9e44038020 con 0x7f9e480d1710 2024-04-15T15:21:09.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.105+0000 7f9e3affd640 1 --2- 172.21.15.105:0/3800729023 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f9e2c06c9d0 0x7f9e2c06ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:09.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.105+0000 7f9e3affd640 1 -- 172.21.15.105:0/3800729023 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5795+0+0 (secure 0 0 0) 0x7f9e440ad520 con 0x7f9e480d1710 2024-04-15T15:21:09.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.105+0000 7f9e4d874640 1 --2- 172.21.15.105:0/3800729023 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f9e2c06c9d0 0x7f9e2c06ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:09.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.106+0000 7f9e4d874640 1 --2- 172.21.15.105:0/3800729023 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f9e2c06c9d0 0x7f9e2c06ee90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9e480b2980 tx=0x7f9e3c007b70 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:09.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.106+0000 7f9e52f9e740 1 Processor -- start 2024-04-15T15:21:09.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.107+0000 7f9e52f9e740 1 -- start start 2024-04-15T15:21:09.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.107+0000 7f9e52f9e740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5606d4983e10 0x5606d4956db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:09.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.107+0000 7f9e52f9e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5606d49841f0 0x5606d4986ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:09.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.107+0000 7f9e52f9e740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5606d49572f0 0x5606d49870e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:09.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.107+0000 7f9e52f9e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5606d48c19c0 con 0x5606d49841f0 2024-04-15T15:21:09.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.107+0000 7f9e52f9e740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5606d4868650 con 0x5606d4983e10 2024-04-15T15:21:09.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.107+0000 7f9e52f9e740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x5606d48bd980 con 0x5606d49572f0 2024-04-15T15:21:09.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.107+0000 7f9e4d874640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5606d49841f0 0x5606d4986ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:09.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.107+0000 7f9e4d874640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5606d49841f0 0x5606d4986ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46728/0 (socket says 172.21.15.105:46728) 2024-04-15T15:21:09.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.107+0000 7f9e4d874640 1 -- 172.21.15.105:0/596992561 learned_addr learned my addr 172.21.15.105:0/596992561 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:09.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.108+0000 7f9e4d874640 1 -- 172.21.15.105:0/596992561 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5606d49572f0 msgr2=0x5606d49870e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:09.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.108+0000 7f9e4d874640 1 --2- 172.21.15.105:0/596992561 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5606d49572f0 0x5606d49870e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:09.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.108+0000 7f9e4d874640 1 -- 172.21.15.105:0/596992561 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5606d4983e10 msgr2=0x5606d4956db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:09.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.108+0000 7f9e4e075640 1 --2- 172.21.15.105:0/596992561 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5606d4983e10 0x5606d4956db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:09.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.108+0000 7f9e4d874640 1 --2- 172.21.15.105:0/596992561 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5606d4983e10 0x5606d4956db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:09.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.108+0000 7f9e4d874640 1 -- 172.21.15.105:0/596992561 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606d45d42d0 con 0x5606d49841f0 2024-04-15T15:21:09.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.108+0000 7f9e4d874640 1 --2- 172.21.15.105:0/596992561 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5606d49841f0 0x5606d4986ba0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c005cd0 tx=0x7f9e3c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8126ce94b0f00472 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:09.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.108+0000 7f9e2bfff640 1 -- 172.21.15.105:0/596992561 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c015110 con 0x5606d49841f0 2024-04-15T15:21:09.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.108+0000 7f9e2bfff640 1 -- 172.21.15.105:0/596992561 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e3c0159c0 con 0x5606d49841f0 2024-04-15T15:21:09.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.108+0000 7f9e2bfff640 1 -- 172.21.15.105:0/596992561 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c015d70 con 0x5606d49841f0 2024-04-15T15:21:09.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.109+0000 7f9e52f9e740 1 -- 172.21.15.105:0/596992561 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5606d49841f0 msgr2=0x5606d4986ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:09.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.109+0000 7f9e52f9e740 1 --2- 172.21.15.105:0/596992561 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5606d49841f0 0x5606d4986ba0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c005cd0 tx=0x7f9e3c012000 comp rx=0 tx=0).stop 2024-04-15T15:21:09.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.109+0000 7f9e52f9e740 1 -- 172.21.15.105:0/596992561 shutdown_connections 2024-04-15T15:21:09.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.109+0000 7f9e52f9e740 1 --2- 172.21.15.105:0/596992561 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5606d49572f0 0x5606d49870e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:09.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.109+0000 7f9e52f9e740 1 --2- 172.21.15.105:0/596992561 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5606d49841f0 0x5606d4986ba0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:09.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.109+0000 7f9e52f9e740 1 --2- 172.21.15.105:0/596992561 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5606d4983e10 0x5606d4956db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:09.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.109+0000 7f9e52f9e740 1 -- 172.21.15.105:0/596992561 >> 172.21.15.105:0/596992561 conn(0x5606d4959020 msgr2=0x5606d49a1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:09.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.109+0000 7f9e52f9e740 1 -- 172.21.15.105:0/596992561 shutdown_connections 2024-04-15T15:21:09.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.109+0000 7f9e52f9e740 1 -- 172.21.15.105:0/596992561 wait complete. 2024-04-15T15:21:09.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.109+0000 7f9e52f9e740 1 -- start start 2024-04-15T15:21:09.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.109+0000 7f9e52f9e740 1 Processor -- start 2024-04-15T15:21:09.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.110+0000 7f9e52f9e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5606d4983e10 0x5606d4999ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:09.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.110+0000 7f9e52f9e740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5606d49841f0 0x5606d499c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:09.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.110+0000 7f9e52f9e740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5606d4982e20 0x5606d499b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:09.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.110+0000 7f9e52f9e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5606d495b5c0 con 0x5606d4983e10 2024-04-15T15:21:09.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.110+0000 7f9e52f9e740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5606d487ba10 con 0x5606d4982e20 2024-04-15T15:21:09.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.110+0000 7f9e52f9e740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x5606d495a9b0 con 0x5606d49841f0 2024-04-15T15:21:09.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.110+0000 7f9e52f9e740 10 client.0 _ll_register_callbacks cb 0x5606d48b7250 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-15T15:21:09.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.110+0000 7f9e4e075640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5606d4983e10 0x5606d4999ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:09.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.111+0000 7f9e4e075640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5606d4983e10 0x5606d4999ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46742/0 (socket says 172.21.15.105:46742) 2024-04-15T15:21:09.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.111+0000 7f9e4e075640 1 -- 172.21.15.105:0/2294292497 learned_addr learned my addr 172.21.15.105:0/2294292497 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:09.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.111+0000 7f9e4d874640 1 --2- 172.21.15.105:0/2294292497 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5606d49841f0 0x5606d499c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:09.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.111+0000 7f9e4e876640 1 --2- 172.21.15.105:0/2294292497 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5606d4982e20 0x5606d499b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:09.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.111+0000 7f9e4e075640 1 -- 172.21.15.105:0/2294292497 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5606d49841f0 msgr2=0x5606d499c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:09.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.111+0000 7f9e4e075640 1 --2- 172.21.15.105:0/2294292497 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5606d49841f0 0x5606d499c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:09.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.111+0000 7f9e4e075640 1 -- 172.21.15.105:0/2294292497 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5606d4982e20 msgr2=0x5606d499b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:09.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.111+0000 7f9e4e075640 1 --2- 172.21.15.105:0/2294292497 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5606d4982e20 0x5606d499b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:09.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.111+0000 7f9e4e075640 1 -- 172.21.15.105:0/2294292497 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606d469e6b0 con 0x5606d4983e10 2024-04-15T15:21:09.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.111+0000 7f9e4d874640 1 --2- 172.21.15.105:0/2294292497 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5606d49841f0 0x5606d499c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:21:09.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.111+0000 7f9e4e876640 1 --2- 172.21.15.105:0/2294292497 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5606d4982e20 0x5606d499b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:21:09.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.111+0000 7f9e4e075640 1 --2- 172.21.15.105:0/2294292497 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5606d4983e10 0x5606d4999ea0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f9e44006ce0 tx=0x7f9e4407ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42fdd80ac2ef1dd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:09.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.111+0000 7f9e297fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-04-15T15:21:09.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.111+0000 7f9e297fa640 1 -- 172.21.15.105:0/2294292497 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e44083e70 con 0x5606d4983e10 2024-04-15T15:21:09.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.111+0000 7f9e297fa640 1 -- 172.21.15.105:0/2294292497 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(35..35 src has 1..35) ==== 5795+0+0 (secure 0 0 0) 0x7f9e4407dcd0 con 0x5606d4983e10 2024-04-15T15:21:09.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.111+0000 7f9e52f9e740 1 -- 172.21.15.105:0/2294292497 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e3c0159c0 con 0x5606d4983e10 2024-04-15T15:21:09.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.111+0000 7f9e297fa640 1 -- 172.21.15.105:0/2294292497 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f9e440ad0b0 con 0x5606d4983e10 2024-04-15T15:21:09.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.111+0000 7f9e52f9e740 10 client.5760 fetch_fsmap learned FSMap version 20 2024-04-15T15:21:09.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.111+0000 7f9e52f9e740 1 -- 172.21.15.105:0/2294292497 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x5606d499bdb0 con 0x5606d4983e10 2024-04-15T15:21:09.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.111+0000 7f9e297fa640 1 -- 172.21.15.105:0/2294292497 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e3c0159c0 con 0x5606d4983e10 2024-04-15T15:21:09.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.111+0000 7f9e297fa640 1 -- 172.21.15.105:0/2294292497 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7f9e440abb50 con 0x5606d4983e10 2024-04-15T15:21:09.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.111+0000 7f9e52f9e740 10 client.5760 fetch_fsmap finished waiting for FSMap version 20 2024-04-15T15:21:09.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.111+0000 7f9e52f9e740 10 client.5760 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-15T15:21:09.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.111+0000 7f9e52f9e740 20 client.5760 populate_metadata read hostname 'smithi105' 2024-04-15T15:21:09.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.112+0000 7f9e52f9e740 1 --2- 172.21.15.105:0/2294292497 >> [v2:172.21.15.105:6835/4212477415,v1:172.21.15.105:6837/4212477415] conn(0x5606d49891f0 0x5606d49bfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:09.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.112+0000 7f9e52f9e740 4 client.5760 mds_command: new command op to 5712 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-15T15:21:09.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.112+0000 7f9e52f9e740 1 -- 172.21.15.105:0/2294292497 --> [v2:172.21.15.105:6835/4212477415,v1:172.21.15.105:6837/4212477415] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606d48cf3e0 con 0x5606d49891f0 2024-04-15T15:21:09.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.112+0000 7f9e4d874640 1 --2- 172.21.15.105:0/2294292497 >> [v2:172.21.15.105:6835/4212477415,v1:172.21.15.105:6837/4212477415] conn(0x5606d49891f0 0x5606d49bfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:09.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.112+0000 7f9e297fa640 10 client.5760 ms_handle_connect on v2:172.21.15.105:6835/4212477415 2024-04-15T15:21:09.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.112+0000 7f9e4d874640 1 --2- 172.21.15.105:0/2294292497 >> [v2:172.21.15.105:6835/4212477415,v1:172.21.15.105:6837/4212477415] conn(0x5606d49891f0 0x5606d49bfa20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cf1ada889bbd8bc server_cookie=2f21fbda03b40ea9 in_seq=0 out_seq=0 2024-04-15T15:21:09.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.112+0000 7f9e297fa640 1 -- 172.21.15.105:0/2294292497 <== mds.0 v2:172.21.15.105:6835/4212477415 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5606d48cf3e0 con 0x5606d49891f0 2024-04-15T15:21:09.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.112+0000 7f9e297fa640 10 client.5760 handle_command_reply: tid=0 2024-04-15T15:21:09.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.113+0000 7f9e52f9e740 1 client.5760 shutdown 2024-04-15T15:21:09.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.113+0000 7f9e52f9e740 1 -- 172.21.15.105:0/2294292497 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5606d4983e10 msgr2=0x5606d4999ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:09.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.113+0000 7f9e52f9e740 1 --2- 172.21.15.105:0/2294292497 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5606d4983e10 0x5606d4999ea0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f9e44006ce0 tx=0x7f9e4407ace0 comp rx=0 tx=0).stop 2024-04-15T15:21:09.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.113+0000 7f9e52f9e740 1 -- 172.21.15.105:0/2294292497 shutdown_connections 2024-04-15T15:21:09.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.113+0000 7f9e52f9e740 1 --2- 172.21.15.105:0/2294292497 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5606d49841f0 0x5606d499c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:09.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.113+0000 7f9e52f9e740 1 --2- 172.21.15.105:0/2294292497 >> [v2:172.21.15.105:6835/4212477415,v1:172.21.15.105:6837/4212477415] conn(0x5606d49891f0 0x5606d49bfa20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:09.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.113+0000 7f9e52f9e740 1 --2- 172.21.15.105:0/2294292497 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5606d4982e20 0x5606d499b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:09.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.113+0000 7f9e52f9e740 1 --2- 172.21.15.105:0/2294292497 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5606d4983e10 0x5606d4999ea0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:09.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.113+0000 7f9e297fa640 0 client.5760 ms_handle_reset on v2:172.21.15.105:6835/4212477415 2024-04-15T15:21:09.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.113+0000 7f9e52f9e740 1 -- 172.21.15.105:0/2294292497 >> 172.21.15.105:0/2294292497 conn(0x5606d49572f0 msgr2=0x5606d4986b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:09.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.113+0000 7f9e52f9e740 1 -- 172.21.15.105:0/2294292497 shutdown_connections 2024-04-15T15:21:09.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.113+0000 7f9e52f9e740 1 -- 172.21.15.105:0/2294292497 wait complete. 2024-04-15T15:21:09.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.113+0000 7f9e52f9e740 20 client.5760 trim_cache size 0 max 16384 2024-04-15T15:21:09.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.131+0000 7f9e52f9e740 1 Processor -- start 2024-04-15T15:21:09.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.131+0000 7f9e52f9e740 1 -- start start 2024-04-15T15:21:09.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.132+0000 7f9e52f9e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5606d4982e20 0x5606d495ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:09.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.132+0000 7f9e52f9e740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5606d4983e10 0x5606d49ba2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:09.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.132+0000 7f9e52f9e740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5606d49841f0 0x5606d49ba800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:09.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.132+0000 7f9e52f9e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9e48200c90 con 0x5606d4982e20 2024-04-15T15:21:09.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.132+0000 7f9e52f9e740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9e481ffd80 con 0x5606d4983e10 2024-04-15T15:21:09.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.132+0000 7f9e52f9e740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f9e480b5050 con 0x5606d49841f0 2024-04-15T15:21:09.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.132+0000 7f9e4e075640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5606d4982e20 0x5606d495ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:09.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.132+0000 7f9e4e075640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5606d4982e20 0x5606d495ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46754/0 (socket says 172.21.15.105:46754) 2024-04-15T15:21:09.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.132+0000 7f9e4e075640 1 -- 172.21.15.105:0/249312232 learned_addr learned my addr 172.21.15.105:0/249312232 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:09.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.132+0000 7f9e4e075640 1 -- 172.21.15.105:0/249312232 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5606d49841f0 msgr2=0x5606d49ba800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:09.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.132+0000 7f9e4d874640 1 --2- 172.21.15.105:0/249312232 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5606d4983e10 0x5606d49ba2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:09.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.132+0000 7f9e4e876640 1 --2- 172.21.15.105:0/249312232 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5606d49841f0 0x5606d49ba800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:09.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.132+0000 7f9e4e075640 1 --2- 172.21.15.105:0/249312232 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5606d49841f0 0x5606d49ba800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:09.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.132+0000 7f9e4e075640 1 -- 172.21.15.105:0/249312232 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5606d4983e10 msgr2=0x5606d49ba2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:09.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.132+0000 7f9e4e075640 1 --2- 172.21.15.105:0/249312232 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5606d4983e10 0x5606d49ba2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:09.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.132+0000 7f9e4e075640 1 -- 172.21.15.105:0/249312232 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606d4958780 con 0x5606d4982e20 2024-04-15T15:21:09.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.132+0000 7f9e4e876640 1 --2- 172.21.15.105:0/249312232 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5606d49841f0 0x5606d49ba800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:21:09.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.132+0000 7f9e4d874640 1 --2- 172.21.15.105:0/249312232 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5606d4983e10 0x5606d49ba2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:21:09.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.132+0000 7f9e4e075640 1 --2- 172.21.15.105:0/249312232 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5606d4982e20 0x5606d495ab00 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f9e440ad890 tx=0x7f9e4403d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=881db96a8e7b6f73 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:09.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.132+0000 7f9e2b7fe640 1 -- 172.21.15.105:0/249312232 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4407b290 con 0x5606d4982e20 2024-04-15T15:21:09.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.132+0000 7f9e2b7fe640 1 -- 172.21.15.105:0/249312232 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e44077900 con 0x5606d4982e20 2024-04-15T15:21:09.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.132+0000 7f9e2b7fe640 1 -- 172.21.15.105:0/249312232 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4407f120 con 0x5606d4982e20 2024-04-15T15:21:09.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.133+0000 7f9e52f9e740 1 -- 172.21.15.105:0/249312232 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5606d4982e20 msgr2=0x5606d495ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:09.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.133+0000 7f9e52f9e740 1 --2- 172.21.15.105:0/249312232 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5606d4982e20 0x5606d495ab00 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f9e440ad890 tx=0x7f9e4403d070 comp rx=0 tx=0).stop 2024-04-15T15:21:09.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.133+0000 7f9e52f9e740 1 -- 172.21.15.105:0/249312232 shutdown_connections 2024-04-15T15:21:09.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.133+0000 7f9e52f9e740 1 --2- 172.21.15.105:0/249312232 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5606d49841f0 0x5606d49ba800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:09.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.133+0000 7f9e52f9e740 1 --2- 172.21.15.105:0/249312232 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5606d4983e10 0x5606d49ba2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:09.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.133+0000 7f9e52f9e740 1 --2- 172.21.15.105:0/249312232 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5606d4982e20 0x5606d495ab00 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:09.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.133+0000 7f9e52f9e740 1 -- 172.21.15.105:0/249312232 >> 172.21.15.105:0/249312232 conn(0x5606d49572f0 msgr2=0x5606d499ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:09.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.133+0000 7f9e52f9e740 1 -- 172.21.15.105:0/249312232 shutdown_connections 2024-04-15T15:21:09.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.133+0000 7f9e52f9e740 1 -- 172.21.15.105:0/249312232 wait complete. 2024-04-15T15:21:09.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.133+0000 7f9e52f9e740 1 -- start start 2024-04-15T15:21:09.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.133+0000 7f9e52f9e740 1 Processor -- start 2024-04-15T15:21:09.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.134+0000 7f9e52f9e740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5606d4982e20 0x5606d49c2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:09.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.134+0000 7f9e52f9e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5606d4983e10 0x5606d4a1b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:09.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.134+0000 7f9e52f9e740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5606d49841f0 0x5606d49c0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:09.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.134+0000 7f9e52f9e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5606d487ba10 con 0x5606d4983e10 2024-04-15T15:21:09.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.134+0000 7f9e52f9e740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9e480ce290 con 0x5606d4982e20 2024-04-15T15:21:09.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.134+0000 7f9e52f9e740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x5606d495b5c0 con 0x5606d49841f0 2024-04-15T15:21:09.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.134+0000 7f9e52f9e740 10 client.0 _ll_register_callbacks cb 0x7f9e4407ae40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-15T15:21:09.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.134+0000 7f9e4d874640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5606d4983e10 0x5606d4a1b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:09.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.134+0000 7f9e4d874640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5606d4983e10 0x5606d4a1b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46756/0 (socket says 172.21.15.105:46756) 2024-04-15T15:21:09.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.134+0000 7f9e4d874640 1 -- 172.21.15.105:0/3884901787 learned_addr learned my addr 172.21.15.105:0/3884901787 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:09.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.135+0000 7f9e4e876640 1 --2- 172.21.15.105:0/3884901787 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5606d49841f0 0x5606d49c0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:09.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.135+0000 7f9e4e075640 1 --2- 172.21.15.105:0/3884901787 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5606d4982e20 0x5606d49c2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:09.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.135+0000 7f9e4d874640 1 -- 172.21.15.105:0/3884901787 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5606d49841f0 msgr2=0x5606d49c0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:09.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.135+0000 7f9e4d874640 1 --2- 172.21.15.105:0/3884901787 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5606d49841f0 0x5606d49c0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:09.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.135+0000 7f9e4d874640 1 -- 172.21.15.105:0/3884901787 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5606d4982e20 msgr2=0x5606d49c2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:09.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.135+0000 7f9e4d874640 1 --2- 172.21.15.105:0/3884901787 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5606d4982e20 0x5606d49c2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:09.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.135+0000 7f9e4d874640 1 -- 172.21.15.105:0/3884901787 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606d49bb000 con 0x5606d4983e10 2024-04-15T15:21:09.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.135+0000 7f9e4e075640 1 --2- 172.21.15.105:0/3884901787 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5606d4982e20 0x5606d49c2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:21:09.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.135+0000 7f9e4d874640 1 --2- 172.21.15.105:0/3884901787 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5606d4983e10 0x5606d4a1b4f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c0011e0 tx=0x7f9e3c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c44c91f36cd4e315 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:09.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.135+0000 7f9e297fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-04-15T15:21:09.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.135+0000 7f9e297fa640 1 -- 172.21.15.105:0/3884901787 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c035060 con 0x5606d4983e10 2024-04-15T15:21:09.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.135+0000 7f9e297fa640 1 -- 172.21.15.105:0/3884901787 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(35..35 src has 1..35) ==== 5795+0+0 (secure 0 0 0) 0x7f9e3c015bd0 con 0x5606d4983e10 2024-04-15T15:21:09.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.135+0000 7f9e52f9e740 1 -- 172.21.15.105:0/3884901787 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e44077900 con 0x5606d4983e10 2024-04-15T15:21:09.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.135+0000 7f9e297fa640 1 -- 172.21.15.105:0/3884901787 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f9e3c03b050 con 0x5606d4983e10 2024-04-15T15:21:09.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.135+0000 7f9e52f9e740 10 client.5766 fetch_fsmap learned FSMap version 20 2024-04-15T15:21:09.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.135+0000 7f9e52f9e740 1 -- 172.21.15.105:0/3884901787 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x5606d4987400 con 0x5606d4983e10 2024-04-15T15:21:09.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.135+0000 7f9e297fa640 1 -- 172.21.15.105:0/3884901787 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e44077900 con 0x5606d4983e10 2024-04-15T15:21:09.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.135+0000 7f9e297fa640 1 -- 172.21.15.105:0/3884901787 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7f9e3c03e020 con 0x5606d4983e10 2024-04-15T15:21:09.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.135+0000 7f9e52f9e740 10 client.5766 fetch_fsmap finished waiting for FSMap version 20 2024-04-15T15:21:09.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.135+0000 7f9e52f9e740 10 client.5766 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-15T15:21:09.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.135+0000 7f9e52f9e740 20 client.5766 populate_metadata read hostname 'smithi105' 2024-04-15T15:21:09.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.135+0000 7f9e52f9e740 1 --2- 172.21.15.105:0/3884901787 >> [v2:172.21.15.105:6835/4212477415,v1:172.21.15.105:6837/4212477415] conn(0x5606d49ba910 0x5606d49a2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:09.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.135+0000 7f9e52f9e740 4 client.5766 mds_command: new command op to 5712 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-15T15:21:09.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.135+0000 7f9e52f9e740 1 -- 172.21.15.105:0/3884901787 --> [v2:172.21.15.105:6835/4212477415,v1:172.21.15.105:6837/4212477415] -- command(tid 0: {"prefix": "status"}) -- 0x5606d48c9250 con 0x5606d49ba910 2024-04-15T15:21:09.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.136+0000 7f9e4e075640 1 --2- 172.21.15.105:0/3884901787 >> [v2:172.21.15.105:6835/4212477415,v1:172.21.15.105:6837/4212477415] conn(0x5606d49ba910 0x5606d49a2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:09.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.136+0000 7f9e297fa640 10 client.5766 ms_handle_connect on v2:172.21.15.105:6835/4212477415 2024-04-15T15:21:09.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.136+0000 7f9e4e075640 1 --2- 172.21.15.105:0/3884901787 >> [v2:172.21.15.105:6835/4212477415,v1:172.21.15.105:6837/4212477415] conn(0x5606d49ba910 0x5606d49a2df0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e43b39cc47c579d server_cookie=562fbfc058ee4e26 in_seq=0 out_seq=0 2024-04-15T15:21:09.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.136+0000 7f9e297fa640 1 -- 172.21.15.105:0/3884901787 <== mds.0 v2:172.21.15.105:6835/4212477415 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x5606d48c9250 con 0x5606d49ba910 2024-04-15T15:21:09.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.136+0000 7f9e297fa640 10 client.5766 handle_command_reply: tid=0 2024-04-15T15:21:09.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.136+0000 7f9e52f9e740 1 client.5766 shutdown 2024-04-15T15:21:09.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.136+0000 7f9e52f9e740 1 -- 172.21.15.105:0/3884901787 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5606d4983e10 msgr2=0x5606d4a1b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:09.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.136+0000 7f9e52f9e740 1 --2- 172.21.15.105:0/3884901787 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5606d4983e10 0x5606d4a1b4f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c0011e0 tx=0x7f9e3c011000 comp rx=0 tx=0).stop 2024-04-15T15:21:09.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.136+0000 7f9e52f9e740 1 -- 172.21.15.105:0/3884901787 shutdown_connections 2024-04-15T15:21:09.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.136+0000 7f9e52f9e740 1 --2- 172.21.15.105:0/3884901787 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5606d49841f0 0x5606d49c0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:09.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.136+0000 7f9e52f9e740 1 --2- 172.21.15.105:0/3884901787 >> [v2:172.21.15.105:6835/4212477415,v1:172.21.15.105:6837/4212477415] conn(0x5606d49ba910 0x5606d49a2df0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:09.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.136+0000 7f9e52f9e740 1 --2- 172.21.15.105:0/3884901787 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5606d4983e10 0x5606d4a1b4f0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:09.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.136+0000 7f9e52f9e740 1 --2- 172.21.15.105:0/3884901787 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5606d4982e20 0x5606d49c2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:09.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.136+0000 7f9e297fa640 0 client.5766 ms_handle_reset on v2:172.21.15.105:6835/4212477415 2024-04-15T15:21:09.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.136+0000 7f9e52f9e740 1 -- 172.21.15.105:0/3884901787 >> 172.21.15.105:0/3884901787 conn(0x5606d49572f0 msgr2=0x5606d499ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:09.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.136+0000 7f9e52f9e740 1 -- 172.21.15.105:0/3884901787 shutdown_connections 2024-04-15T15:21:09.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.136+0000 7f9e52f9e740 1 -- 172.21.15.105:0/3884901787 wait complete. 2024-04-15T15:21:09.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.136+0000 7f9e52f9e740 20 client.5766 trim_cache size 0 max 16384 2024-04-15T15:21:09.145 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-04-15T15:21:09.145 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "09e4d2a4-a324-4793-af77-644b25e520e1", 2024-04-15T15:21:09.145 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 0, 2024-04-15T15:21:09.145 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 5712, 2024-04-15T15:21:09.146 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2024-04-15T15:21:09.146 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2024-04-15T15:21:09.146 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2024-04-15T15:21:09.146 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 4.4632041850000004, 2024-04-15T15:21:09.146 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 20, 2024-04-15T15:21:09.146 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 35, 2024-04-15T15:21:09.146 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 35, 2024-04-15T15:21:09.146 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 5.4329705830000004 2024-04-15T15:21:09.146 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-04-15T15:21:09.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.137+0000 7f9e502d9640 1 -- 172.21.15.105:0/3800729023 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f9e2c06c9d0 msgr2=0x7f9e2c06ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:09.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.137+0000 7f9e502d9640 1 --2- 172.21.15.105:0/3800729023 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f9e2c06c9d0 0x7f9e2c06ee90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9e480b2980 tx=0x7f9e3c007b70 comp rx=0 tx=0).stop 2024-04-15T15:21:09.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.137+0000 7f9e502d9640 1 -- 172.21.15.105:0/3800729023 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e480d1710 msgr2=0x7f9e4815bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:09.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.137+0000 7f9e502d9640 1 --2- 172.21.15.105:0/3800729023 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e480d1710 0x7f9e4815bc90 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f9e44009890 tx=0x7f9e440046c0 comp rx=0 tx=0).stop 2024-04-15T15:21:09.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.137+0000 7f9e502d9640 1 -- 172.21.15.105:0/3800729023 shutdown_connections 2024-04-15T15:21:09.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.137+0000 7f9e502d9640 1 --2- 172.21.15.105:0/3800729023 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e4815eb60 0x7f9e480cb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:09.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.137+0000 7f9e502d9640 1 --2- 172.21.15.105:0/3800729023 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f9e2c06c9d0 0x7f9e2c06ee90 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:09.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.137+0000 7f9e502d9640 1 --2- 172.21.15.105:0/3800729023 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e48162d40 0x7f9e480cbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:09.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.137+0000 7f9e502d9640 1 --2- 172.21.15.105:0/3800729023 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e480d1710 0x7f9e4815bc90 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:09.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.137+0000 7f9e502d9640 1 -- 172.21.15.105:0/3800729023 >> 172.21.15.105:0/3800729023 conn(0x7f9e48067720 msgr2=0x7f9e480c95a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:09.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.137+0000 7f9e502d9640 1 -- 172.21.15.105:0/3800729023 shutdown_connections 2024-04-15T15:21:09.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.137+0000 7f9e502d9640 1 -- 172.21.15.105:0/3800729023 wait complete. 2024-04-15T15:21:09.149 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-15T15:21:09.149 DEBUG:teuthology.orchestra.run.smithi105:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-15T15:21:09.197 DEBUG:teuthology.orchestra.run.smithi154:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-15T15:21:09.241 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-15T15:21:09.241 INFO:tasks.cephfs.mount:Terminating background process 2024-04-15T15:21:09.242 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-04-15T15:21:09.242 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:21:09.258 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:21:09.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.400+0000 7fd089d82640 1 Processor -- start 2024-04-15T15:21:09.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.400+0000 7fd089d82640 1 -- start start 2024-04-15T15:21:09.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.401+0000 7fd089d82640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd0840b3c80 0x7fd084169ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:09.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.401+0000 7fd089d82640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0840c8270 con 0x7fd0840d0540 2024-04-15T15:21:09.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.401+0000 7fd089d82640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0840d0920 con 0x7fd08406c0d0 2024-04-15T15:21:09.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.401+0000 7fd089d82640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fd0840c7cb0 con 0x7fd0840b3c80 2024-04-15T15:21:09.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.401+0000 7fd0837fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fd0840d0540 0x7fd08406b9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:50860/0 (socket says 172.21.15.105:50860) 2024-04-15T15:21:09.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.401+0000 7fd0837fe640 1 -- 172.21.15.105:0/3170064644 learned_addr learned my addr 172.21.15.105:0/3170064644 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:21:09.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.402+0000 7fd083fff640 1 --2- 172.21.15.105:0/3170064644 >> v2:172.21.15.154:3300/0 conn(0x7fd0840b3c80 0x7fd084169ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:09.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.402+0000 7fd0827fc640 1 -- 172.21.15.105:0/3170064644 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3199980180 0 0) 0x7fd0840c8270 con 0x7fd0840d0540 2024-04-15T15:21:09.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.402+0000 7fd0827fc640 1 -- 172.21.15.105:0/3170064644 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd06c003520 con 0x7fd0840d0540 2024-04-15T15:21:09.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.402+0000 7fd0827fc640 1 -- 172.21.15.105:0/3170064644 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2479653952 0 0) 0x7fd06c003520 con 0x7fd0840d0540 2024-04-15T15:21:09.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.402+0000 7fd0827fc640 1 -- 172.21.15.105:0/3170064644 >> v2:172.21.15.154:3300/0 conn(0x7fd0840b3c80 msgr2=0x7fd084169ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:09.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.402+0000 7fd0827fc640 1 --2- 172.21.15.105:0/3170064644 >> v2:172.21.15.154:3300/0 conn(0x7fd0840b3c80 0x7fd084169ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:09.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.402+0000 7fd0827fc640 1 -- 172.21.15.105:0/3170064644 >> v1:172.21.15.154:6789/0 conn(0x7fd08406c0d0 legacy=0x7fd084166570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:09.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.402+0000 7fd0827fc640 1 -- 172.21.15.105:0/3170064644 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0840c7f90 con 0x7fd0840d0540 2024-04-15T15:21:09.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.402+0000 7fd0827fc640 1 -- 172.21.15.105:0/3170064644 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7fd0740019f0 con 0x7fd0840d0540 2024-04-15T15:21:09.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.402+0000 7fd0827fc640 1 -- 172.21.15.105:0/3170064644 >> v1:172.21.15.105:6789/0 conn(0x7fd0840d0540 legacy=0x7fd08406b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:09.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.402+0000 7fd0827fc640 1 --2- 172.21.15.105:0/3170064644 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd06c0039e0 0x7fd06c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:09.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.402+0000 7fd0827fc640 1 --2- 172.21.15.105:0/3170064644 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd06c004800 0x7fd06c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:09.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.402+0000 7fd0827fc640 1 --2- 172.21.15.105:0/3170064644 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd06c0051c0 0x7fd06c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:09.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.402+0000 7fd0827fc640 1 -- 172.21.15.105:0/3170064644 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd0840c7cb0 con 0x7fd06c0051c0 2024-04-15T15:21:09.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.402+0000 7fd0827fc640 1 -- 172.21.15.105:0/3170064644 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd06c00c090 con 0x7fd06c0039e0 2024-04-15T15:21:09.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.402+0000 7fd0827fc640 1 -- 172.21.15.105:0/3170064644 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fd06c00c290 con 0x7fd06c004800 2024-04-15T15:21:09.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.402+0000 7fd083fff640 1 --2- 172.21.15.105:0/3170064644 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd06c0051c0 0x7fd06c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:09.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.402+0000 7fd0837fe640 1 --2- 172.21.15.105:0/3170064644 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd06c0039e0 0x7fd06c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:09.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.402+0000 7fd082ffd640 1 --2- 172.21.15.105:0/3170064644 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd06c004800 0x7fd06c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:09.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.402+0000 7fd083fff640 1 -- 172.21.15.105:0/3170064644 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd06c004800 msgr2=0x7fd06c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:09.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.402+0000 7fd083fff640 1 --2- 172.21.15.105:0/3170064644 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd06c004800 0x7fd06c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:09.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.402+0000 7fd083fff640 1 -- 172.21.15.105:0/3170064644 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd06c0039e0 msgr2=0x7fd06c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:09.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.402+0000 7fd083fff640 1 --2- 172.21.15.105:0/3170064644 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd06c0039e0 0x7fd06c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:09.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.402+0000 7fd083fff640 1 -- 172.21.15.105:0/3170064644 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd06c003520 con 0x7fd06c0051c0 2024-04-15T15:21:09.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.403+0000 7fd083fff640 1 -- 172.21.15.105:0/3170064644 reap_dead start 2024-04-15T15:21:09.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.403+0000 7fd083fff640 1 --2- 172.21.15.105:0/3170064644 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd06c0051c0 0x7fd06c00b9c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fd07c009ac0 tx=0x7fd06c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54bb5f28dfdcda5d server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:09.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.403+0000 7fd0827fc640 1 -- 172.21.15.105:0/3170064644 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07c009690 con 0x7fd06c0051c0 2024-04-15T15:21:09.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.403+0000 7fd0827fc640 1 -- 172.21.15.105:0/3170064644 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd07c004020 con 0x7fd06c0051c0 2024-04-15T15:21:09.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.403+0000 7fd0827fc640 1 -- 172.21.15.105:0/3170064644 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07c004410 con 0x7fd06c0051c0 2024-04-15T15:21:09.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.404+0000 7fd089d82640 1 -- 172.21.15.105:0/3170064644 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd06c0051c0 msgr2=0x7fd06c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:09.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.404+0000 7fd089d82640 1 --2- 172.21.15.105:0/3170064644 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd06c0051c0 0x7fd06c00b9c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fd07c009ac0 tx=0x7fd06c005830 comp rx=0 tx=0).stop 2024-04-15T15:21:09.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.404+0000 7fd089d82640 1 -- 172.21.15.105:0/3170064644 shutdown_connections 2024-04-15T15:21:09.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.404+0000 7fd089d82640 1 --2- 172.21.15.105:0/3170064644 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd06c0051c0 0x7fd06c00b9c0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:09.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.404+0000 7fd089d82640 1 -- 172.21.15.105:0/3170064644 >> 172.21.15.105:0/3170064644 conn(0x7fd08409aa40 msgr2=0x7fd0840b30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:09.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.404+0000 7fd089d82640 1 -- 172.21.15.105:0/3170064644 shutdown_connections 2024-04-15T15:21:09.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.404+0000 7fd089d82640 1 -- 172.21.15.105:0/3170064644 wait complete. 2024-04-15T15:21:09.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.405+0000 7fd089d82640 1 Processor -- start 2024-04-15T15:21:09.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.405+0000 7fd089d82640 1 -- start start 2024-04-15T15:21:09.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.406+0000 7fd089d82640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd06c0051c0 0x7fd0840b6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:09.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.406+0000 7fd089d82640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd08409deb0 0x7fd08414f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:09.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.406+0000 7fd089d82640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd08409b490 0x7fd08409b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:09.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.406+0000 7fd089d82640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd084070e40 con 0x7fd08409b490 2024-04-15T15:21:09.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.406+0000 7fd089d82640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd08406f180 con 0x7fd08409deb0 2024-04-15T15:21:09.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.406+0000 7fd089d82640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fd084070fc0 con 0x7fd06c0051c0 2024-04-15T15:21:09.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.406+0000 7fd083fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd08409b490 0x7fd08409b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:09.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.406+0000 7fd083fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd08409b490 0x7fd08409b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46772/0 (socket says 172.21.15.105:46772) 2024-04-15T15:21:09.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.406+0000 7fd083fff640 1 -- 172.21.15.105:0/781497484 learned_addr learned my addr 172.21.15.105:0/781497484 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:09.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.406+0000 7fd0837fe640 1 --2- 172.21.15.105:0/781497484 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd06c0051c0 0x7fd0840b6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:09.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.406+0000 7fd082ffd640 1 --2- 172.21.15.105:0/781497484 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd08409deb0 0x7fd08414f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:09.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.406+0000 7fd083fff640 1 -- 172.21.15.105:0/781497484 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd06c0051c0 msgr2=0x7fd0840b6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:09.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.406+0000 7fd083fff640 1 --2- 172.21.15.105:0/781497484 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd06c0051c0 0x7fd0840b6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:09.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.406+0000 7fd083fff640 1 -- 172.21.15.105:0/781497484 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd08409deb0 msgr2=0x7fd08414f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:09.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.406+0000 7fd083fff640 1 --2- 172.21.15.105:0/781497484 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd08409deb0 0x7fd08414f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:09.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.406+0000 7fd083fff640 1 -- 172.21.15.105:0/781497484 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd08414f8c0 con 0x7fd08409b490 2024-04-15T15:21:09.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.407+0000 7fd083fff640 1 --2- 172.21.15.105:0/781497484 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd08409b490 0x7fd08409b910 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fd07c02c380 tx=0x7fd07c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:09.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.407+0000 7fd06bfff640 1 -- 172.21.15.105:0/781497484 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07c02cbd0 con 0x7fd08409b490 2024-04-15T15:21:09.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.407+0000 7fd06bfff640 1 -- 172.21.15.105:0/781497484 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd07c02cd90 con 0x7fd08409b490 2024-04-15T15:21:09.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.407+0000 7fd089d82640 1 -- 172.21.15.105:0/781497484 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd08409e620 con 0x7fd08409b490 2024-04-15T15:21:09.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.407+0000 7fd06bfff640 1 -- 172.21.15.105:0/781497484 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07c003850 con 0x7fd08409b490 2024-04-15T15:21:09.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.407+0000 7fd089d82640 1 -- 172.21.15.105:0/781497484 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd08409ebe0 con 0x7fd08409b490 2024-04-15T15:21:09.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.408+0000 7fd089d82640 1 -- 172.21.15.105:0/781497484 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd084068a40 con 0x7fd08409b490 2024-04-15T15:21:09.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.408+0000 7fd06bfff640 1 -- 172.21.15.105:0/781497484 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fd07c0039f0 con 0x7fd08409b490 2024-04-15T15:21:09.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.409+0000 7fd06bfff640 1 --2- 172.21.15.105:0/781497484 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fd07006c9b0 0x7fd07006ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:09.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.409+0000 7fd06bfff640 1 -- 172.21.15.105:0/781497484 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5795+0+0 (secure 0 0 0) 0x7fd07c0ac930 con 0x7fd08409b490 2024-04-15T15:21:09.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.409+0000 7fd0837fe640 1 --2- 172.21.15.105:0/781497484 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fd07006c9b0 0x7fd07006ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:09.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.409+0000 7fd0837fe640 1 --2- 172.21.15.105:0/781497484 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fd07006c9b0 0x7fd07006ee70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd0740013d0 tx=0x7fd074006040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:09.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.413+0000 7fd06bfff640 1 -- 172.21.15.105:0/781497484 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd07c07ada0 con 0x7fd08409b490 2024-04-15T15:21:09.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.587+0000 7fd089d82640 1 -- 172.21.15.105:0/781497484 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd0840d0090 con 0x7fd08409b490 2024-04-15T15:21:09.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.589+0000 7fd06bfff640 1 -- 172.21.15.105:0/781497484 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16846 (secure 0 0 0) 0x7fd07c07a6a0 con 0x7fd08409b490 2024-04-15T15:21:09.591 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:21:09.591 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":35,"fsid":"09e4d2a4-a324-4793-af77-644b25e520e1","created":"2024-04-15T15:19:01.524543+0000","modified":"2024-04-15T15:21:03.660870+0000","last_up_change":"2024-04-15T15:19:12.825815+0000","last_in_change":"2024-04-15T15:19:03.788126+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:19:13.748585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:20:26.983353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-15T15:20:27.961802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6f6292b-2d7e-4aa0-9d1c-6ddd0f97287c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6801","nonce":2873040054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6803","nonce":2873040054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6807","nonce":2873040054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6805","nonce":2873040054}]},"public_addr":"172.21.15.105:6801/2873040054","cluster_addr":"172.21.15.105:6803/2873040054","heartbeat_back_addr":"172.21.15.105:6807/2873040054","heartbeat_front_addr":"172.21.15.105:6805/2873040054","state":["exists","up"]},{"osd":1,"uuid":"8900966a-b393-41e3-8f93-e5a43bd23622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6818","nonce":2494994479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6821","nonce":2494994479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6828","nonce":2494994479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6823","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6825","nonce":2494994479}]},"public_addr":"172.21.15.105:6818/2494994479","cluster_addr":"172.21.15.105:6821/2494994479","heartbeat_back_addr":"172.21.15.105:6828/2494994479","heartbeat_front_addr":"172.21.15.105:6825/2494994479","state":["exists","up"]},{"osd":2,"uuid":"81a5dc7a-9325-4ff5-941d-a2d5fba21f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6809","nonce":3798408699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6811","nonce":3798408699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6815","nonce":3798408699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6813","nonce":3798408699}]},"public_addr":"172.21.15.105:6809/3798408699","cluster_addr":"172.21.15.105:6811/3798408699","heartbeat_back_addr":"172.21.15.105:6815/3798408699","heartbeat_front_addr":"172.21.15.105:6813/3798408699","state":["exists","up"]},{"osd":3,"uuid":"df766ed2-5c71-4805-96f4-3d4970edc960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6819","nonce":1920720186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6824","nonce":1920720186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6831","nonce":1920720186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6827","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6829","nonce":1920720186}]},"public_addr":"172.21.15.105:6819/1920720186","cluster_addr":"172.21.15.105:6824/1920720186","heartbeat_back_addr":"172.21.15.105:6831/1920720186","heartbeat_front_addr":"172.21.15.105:6829/1920720186","state":["exists","up"]},{"osd":4,"uuid":"e1039e03-28b8-49df-b951-c91930b93c9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6810","nonce":3516494456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6813","nonce":3516494456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6820","nonce":3516494456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6817","nonce":3516494456}]},"public_addr":"172.21.15.154:6810/3516494456","cluster_addr":"172.21.15.154:6813/3516494456","heartbeat_back_addr":"172.21.15.154:6820/3516494456","heartbeat_front_addr":"172.21.15.154:6817/3516494456","state":["exists","up"]},{"osd":5,"uuid":"a9f5eaf1-66ff-405a-a9e4-ec3340812a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6801","nonce":540367751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6803","nonce":540367751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6807","nonce":540367751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6805","nonce":540367751}]},"public_addr":"172.21.15.154:6801/540367751","cluster_addr":"172.21.15.154:6803/540367751","heartbeat_back_addr":"172.21.15.154:6807/540367751","heartbeat_front_addr":"172.21.15.154:6805/540367751","state":["exists","up"]},{"osd":6,"uuid":"267d02ea-675f-428f-aae8-e0375e4d4feb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6825","nonce":1550829005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6827","nonce":1550829005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6831","nonce":1550829005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6829","nonce":1550829005}]},"public_addr":"172.21.15.154:6825/1550829005","cluster_addr":"172.21.15.154:6827/1550829005","heartbeat_back_addr":"172.21.15.154:6831/1550829005","heartbeat_front_addr":"172.21.15.154:6829/1550829005","state":["exists","up"]},{"osd":7,"uuid":"759ac6cc-896a-423c-885f-9fe90d8dcb14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6809","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6811","nonce":3920089903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6816","nonce":3920089903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6823","nonce":3920089903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6819","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6821","nonce":3920089903}]},"public_addr":"172.21.15.154:6811/3920089903","cluster_addr":"172.21.15.154:6816/3920089903","heartbeat_back_addr":"172.21.15.154:6823/3920089903","heartbeat_front_addr":"172.21.15.154:6821/3920089903","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.949509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.984072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.907278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.947391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.975447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.998896+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:09.028739+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.894119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.105:6837/958066041":"2024-04-16T15:21:03.660841+0000","172.21.15.105:6835/958066041":"2024-04-16T15:21:03.660841+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:21:09.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.590+0000 7fd089d82640 1 -- 172.21.15.105:0/781497484 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fd07006c9b0 msgr2=0x7fd07006ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:09.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.590+0000 7fd089d82640 1 --2- 172.21.15.105:0/781497484 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fd07006c9b0 0x7fd07006ee70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd0740013d0 tx=0x7fd074006040 comp rx=0 tx=0).stop 2024-04-15T15:21:09.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.590+0000 7fd089d82640 1 -- 172.21.15.105:0/781497484 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd08409b490 msgr2=0x7fd08409b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:09.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.590+0000 7fd089d82640 1 --2- 172.21.15.105:0/781497484 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd08409b490 0x7fd08409b910 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fd07c02c380 tx=0x7fd07c004730 comp rx=0 tx=0).stop 2024-04-15T15:21:09.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.591+0000 7fd089d82640 1 -- 172.21.15.105:0/781497484 shutdown_connections 2024-04-15T15:21:09.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.591+0000 7fd089d82640 1 --2- 172.21.15.105:0/781497484 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fd07006c9b0 0x7fd07006ee70 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:09.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.591+0000 7fd089d82640 1 --2- 172.21.15.105:0/781497484 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd08409b490 0x7fd08409b910 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:09.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.591+0000 7fd089d82640 1 --2- 172.21.15.105:0/781497484 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd08409deb0 0x7fd08414f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:09.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.591+0000 7fd089d82640 1 --2- 172.21.15.105:0/781497484 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd06c0051c0 0x7fd0840b6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:09.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.591+0000 7fd089d82640 1 -- 172.21.15.105:0/781497484 >> 172.21.15.105:0/781497484 conn(0x7fd08409aa40 msgr2=0x7fd0840aee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:09.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.591+0000 7fd089d82640 1 -- 172.21.15.105:0/781497484 shutdown_connections 2024-04-15T15:21:09.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.591+0000 7fd089d82640 1 -- 172.21.15.105:0/781497484 wait complete. 2024-04-15T15:21:09.604 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:19:13.748585+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-15T15:20:26.983353+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-15T15:20:27.961802+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:21:09.604 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:21:09.676 INFO:tasks.ceph.mds.a.smithi105.stderr:2024-04-15T15:21:09.674+0000 7f48b9f2d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-15T15:21:09.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.711+0000 7f863439d640 1 Processor -- start 2024-04-15T15:21:09.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.711+0000 7f863439d640 1 -- start start 2024-04-15T15:21:09.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.712+0000 7f863439d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f862c15e8d0 0x7f862c0d2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:09.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.712+0000 7f863439d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f862c0d1820 0x7f862c0b7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:09.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.712+0000 7f863439d640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f862c0d3400 con 0x7f862c0ba480 2024-04-15T15:21:09.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.712+0000 7f863439d640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f862c0d36e0 con 0x7f862c0d1820 2024-04-15T15:21:09.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.712+0000 7f863439d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f862c0d39c0 con 0x7f862c15e8d0 2024-04-15T15:21:09.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.713+0000 7f8631938640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f862c0ba480 0x7f862c0d1110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:50870/0 (socket says 172.21.15.105:50870) 2024-04-15T15:21:09.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.713+0000 7f8631938640 1 -- 172.21.15.105:0/2299722026 learned_addr learned my addr 172.21.15.105:0/2299722026 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:21:09.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.713+0000 7f863293a640 1 --2- 172.21.15.105:0/2299722026 >> v2:172.21.15.154:3300/0 conn(0x7f862c0d1820 0x7f862c0b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:09.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.713+0000 7f8632139640 1 --2- 172.21.15.105:0/2299722026 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f862c15e8d0 0x7f862c0d2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:09.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.713+0000 7f8631137640 1 -- 172.21.15.105:0/2299722026 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 134106494 0 0) 0x7f862c0d3400 con 0x7f862c0ba480 2024-04-15T15:21:09.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.713+0000 7f8631137640 1 -- 172.21.15.105:0/2299722026 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8614003520 con 0x7f862c0ba480 2024-04-15T15:21:09.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.713+0000 7f8632139640 1 -- 172.21.15.105:0/2299722026 >> v2:172.21.15.154:3300/0 conn(0x7f862c0d1820 msgr2=0x7f862c0b7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:09.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.713+0000 7f8632139640 1 --2- 172.21.15.105:0/2299722026 >> v2:172.21.15.154:3300/0 conn(0x7f862c0d1820 0x7f862c0b7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:09.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.713+0000 7f8632139640 1 -- 172.21.15.105:0/2299722026 >> v1:172.21.15.105:6789/0 conn(0x7f862c0ba480 legacy=0x7f862c0d1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:09.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.713+0000 7f8632139640 1 -- 172.21.15.105:0/2299722026 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f862c0d3ca0 con 0x7f862c15e8d0 2024-04-15T15:21:09.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.713+0000 7f863293a640 1 --2- 172.21.15.105:0/2299722026 >> v2:172.21.15.154:3300/0 conn(0x7f862c0d1820 0x7f862c0b7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:21:09.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.714+0000 7f8632139640 1 --2- 172.21.15.105:0/2299722026 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f862c15e8d0 0x7f862c0d2ab0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f861c009d90 tx=0x7f861c02bf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3134483ffa165845 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:09.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.714+0000 7f8631137640 1 -- 172.21.15.105:0/2299722026 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f861c039070 con 0x7f862c15e8d0 2024-04-15T15:21:09.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.714+0000 7f8631137640 1 -- 172.21.15.105:0/2299722026 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f861c02cce0 con 0x7f862c15e8d0 2024-04-15T15:21:09.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.714+0000 7f8631137640 1 -- 172.21.15.105:0/2299722026 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f861c034660 con 0x7f862c15e8d0 2024-04-15T15:21:09.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.715+0000 7f863439d640 1 -- 172.21.15.105:0/2299722026 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f862c15e8d0 msgr2=0x7f862c0d2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:09.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.715+0000 7f863439d640 1 --2- 172.21.15.105:0/2299722026 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f862c15e8d0 0x7f862c0d2ab0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f861c009d90 tx=0x7f861c02bf30 comp rx=0 tx=0).stop 2024-04-15T15:21:09.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.715+0000 7f863439d640 1 -- 172.21.15.105:0/2299722026 shutdown_connections 2024-04-15T15:21:09.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.715+0000 7f863439d640 1 --2- 172.21.15.105:0/2299722026 >> v2:172.21.15.154:3300/0 conn(0x7f862c0d1820 0x7f862c0b7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:09.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.715+0000 7f863439d640 1 --2- 172.21.15.105:0/2299722026 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f862c15e8d0 0x7f862c0d2ab0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:09.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.715+0000 7f863439d640 1 -- 172.21.15.105:0/2299722026 >> 172.21.15.105:0/2299722026 conn(0x7f862c06d460 msgr2=0x7f862c067720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:09.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.715+0000 7f863439d640 1 -- 172.21.15.105:0/2299722026 shutdown_connections 2024-04-15T15:21:09.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.715+0000 7f863439d640 1 -- 172.21.15.105:0/2299722026 wait complete. 2024-04-15T15:21:09.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.716+0000 7f863439d640 1 Processor -- start 2024-04-15T15:21:09.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.716+0000 7f863439d640 1 -- start start 2024-04-15T15:21:09.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.717+0000 7f863439d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f862c0ba480 0x7f862c1f7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:09.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.717+0000 7f863439d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f862c0d1820 0x7f862c0cefa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:09.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.717+0000 7f863439d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f862c15e8d0 0x7f862c0b6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:09.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.717+0000 7f863439d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f862c0d2ff0 con 0x7f862c0ba480 2024-04-15T15:21:09.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.717+0000 7f863439d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f862c0d44a0 con 0x7f862c15e8d0 2024-04-15T15:21:09.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.717+0000 7f863439d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f862c0d3170 con 0x7f862c0d1820 2024-04-15T15:21:09.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.717+0000 7f8632139640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f862c0ba480 0x7f862c1f7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:09.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.717+0000 7f8632139640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f862c0ba480 0x7f862c1f7730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46774/0 (socket says 172.21.15.105:46774) 2024-04-15T15:21:09.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.717+0000 7f8632139640 1 -- 172.21.15.105:0/1844025664 learned_addr learned my addr 172.21.15.105:0/1844025664 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:09.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.717+0000 7f863293a640 1 --2- 172.21.15.105:0/1844025664 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f862c15e8d0 0x7f862c0b6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:09.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.717+0000 7f8631938640 1 --2- 172.21.15.105:0/1844025664 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f862c0d1820 0x7f862c0cefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:09.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.717+0000 7f8632139640 1 -- 172.21.15.105:0/1844025664 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f862c0d1820 msgr2=0x7f862c0cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:09.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.717+0000 7f8632139640 1 --2- 172.21.15.105:0/1844025664 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f862c0d1820 0x7f862c0cefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:09.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.717+0000 7f8632139640 1 -- 172.21.15.105:0/1844025664 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f862c15e8d0 msgr2=0x7f862c0b6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:09.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.717+0000 7f8632139640 1 --2- 172.21.15.105:0/1844025664 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f862c15e8d0 0x7f862c0b6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:09.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.717+0000 7f8632139640 1 -- 172.21.15.105:0/1844025664 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f862c14aec0 con 0x7f862c0ba480 2024-04-15T15:21:09.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.717+0000 7f863293a640 1 --2- 172.21.15.105:0/1844025664 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f862c15e8d0 0x7f862c0b6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:21:09.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.717+0000 7f8631938640 1 --2- 172.21.15.105:0/1844025664 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f862c0d1820 0x7f862c0cefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:21:09.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.717+0000 7f8632139640 1 --2- 172.21.15.105:0/1844025664 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f862c0ba480 0x7f862c1f7730 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f861c009d90 tx=0x7f861c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:09.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.717+0000 7f862affd640 1 -- 172.21.15.105:0/1844025664 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f861c039050 con 0x7f862c0ba480 2024-04-15T15:21:09.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.717+0000 7f862affd640 1 -- 172.21.15.105:0/1844025664 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f861c045070 con 0x7f862c0ba480 2024-04-15T15:21:09.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.717+0000 7f863439d640 1 -- 172.21.15.105:0/1844025664 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f862c14b1a0 con 0x7f862c0ba480 2024-04-15T15:21:09.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.717+0000 7f862affd640 1 -- 172.21.15.105:0/1844025664 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f861c009450 con 0x7f862c0ba480 2024-04-15T15:21:09.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.717+0000 7f863439d640 1 -- 172.21.15.105:0/1844025664 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f862c14b760 con 0x7f862c0ba480 2024-04-15T15:21:09.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.718+0000 7f863439d640 1 -- 172.21.15.105:0/1844025664 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f862c0b7460 con 0x7f862c0ba480 2024-04-15T15:21:09.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.719+0000 7f862affd640 1 -- 172.21.15.105:0/1844025664 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f861c03e850 con 0x7f862c0ba480 2024-04-15T15:21:09.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.719+0000 7f862affd640 1 --2- 172.21.15.105:0/1844025664 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f861806c980 0x7f861806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:09.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.719+0000 7f862affd640 1 -- 172.21.15.105:0/1844025664 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5795+0+0 (secure 0 0 0) 0x7f861c0ae8f0 con 0x7f862c0ba480 2024-04-15T15:21:09.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.719+0000 7f8631938640 1 --2- 172.21.15.105:0/1844025664 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f861806c980 0x7f861806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:09.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.720+0000 7f8631938640 1 --2- 172.21.15.105:0/1844025664 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f861806c980 0x7f861806ee40 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8620007fd0 tx=0x7f8620009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:09.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.724+0000 7f862affd640 1 -- 172.21.15.105:0/1844025664 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f862c0b7460 con 0x7f862c0ba480 2024-04-15T15:21:09.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.903+0000 7f863439d640 1 -- 172.21.15.105:0/1844025664 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f862c0b6160 con 0x7f862c0ba480 2024-04-15T15:21:09.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.904+0000 7f862affd640 1 -- 172.21.15.105:0/1844025664 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) ==== 93+0+10 (secure 0 0 0) 0x7f861c07cf40 con 0x7f862c0ba480 2024-04-15T15:21:09.906 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-04-15T15:21:09.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.906+0000 7f863439d640 1 -- 172.21.15.105:0/1844025664 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f861806c980 msgr2=0x7f861806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:09.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.906+0000 7f863439d640 1 --2- 172.21.15.105:0/1844025664 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f861806c980 0x7f861806ee40 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8620007fd0 tx=0x7f8620009040 comp rx=0 tx=0).stop 2024-04-15T15:21:09.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.906+0000 7f863439d640 1 -- 172.21.15.105:0/1844025664 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f862c0ba480 msgr2=0x7f862c1f7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:09.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.906+0000 7f863439d640 1 --2- 172.21.15.105:0/1844025664 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f862c0ba480 0x7f862c1f7730 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f861c009d90 tx=0x7f861c004a40 comp rx=0 tx=0).stop 2024-04-15T15:21:09.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.906+0000 7f863439d640 1 -- 172.21.15.105:0/1844025664 shutdown_connections 2024-04-15T15:21:09.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.906+0000 7f863439d640 1 --2- 172.21.15.105:0/1844025664 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f862c0d1820 0x7f862c0cefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:09.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.906+0000 7f863439d640 1 --2- 172.21.15.105:0/1844025664 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f861806c980 0x7f861806ee40 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:09.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.906+0000 7f863439d640 1 --2- 172.21.15.105:0/1844025664 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f862c15e8d0 0x7f862c0b6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:09.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.906+0000 7f863439d640 1 --2- 172.21.15.105:0/1844025664 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f862c0ba480 0x7f862c1f7730 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:09.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.906+0000 7f863439d640 1 -- 172.21.15.105:0/1844025664 >> 172.21.15.105:0/1844025664 conn(0x7f862c06d460 msgr2=0x7f862c0b9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:09.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.906+0000 7f863439d640 1 -- 172.21.15.105:0/1844025664 shutdown_connections 2024-04-15T15:21:09.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:09.906+0000 7f863439d640 1 -- 172.21.15.105:0/1844025664 wait complete. 2024-04-15T15:21:09.919 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-15T15:21:10.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.023+0000 7f3410952640 1 Processor -- start 2024-04-15T15:21:10.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.023+0000 7f3410952640 1 -- start start 2024-04-15T15:21:10.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.024+0000 7f3410952640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f340c0b7910 0x7f340c0d2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:10.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.024+0000 7f3410952640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f340c1625c0 0x7f340c162a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:10.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.024+0000 7f3410952640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f340c0b7d70 con 0x7f340c1620c0 2024-04-15T15:21:10.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.024+0000 7f3410952640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f340c0b8c00 con 0x7f340c0b7910 2024-04-15T15:21:10.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.024+0000 7f3410952640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f340c0b7ff0 con 0x7f340c1625c0 2024-04-15T15:21:10.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.024+0000 7f340a59c640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f340c1620c0 0x7f340c0ba190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:50874/0 (socket says 172.21.15.105:50874) 2024-04-15T15:21:10.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.024+0000 7f340a59c640 1 -- 172.21.15.105:0/449412709 learned_addr learned my addr 172.21.15.105:0/449412709 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:21:10.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.025+0000 7f340ad9d640 1 --2- 172.21.15.105:0/449412709 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f340c1625c0 0x7f340c162a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:10.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.025+0000 7f3409d9b640 1 --2- 172.21.15.105:0/449412709 >> v2:172.21.15.154:3300/0 conn(0x7f340c0b7910 0x7f340c0d2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:10.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.025+0000 7f340959a640 1 -- 172.21.15.105:0/449412709 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4155399682 0 0) 0x7f33fc001490 con 0x7f340c1620c0 2024-04-15T15:21:10.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.025+0000 7f340959a640 1 -- 172.21.15.105:0/449412709 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f33f4003520 con 0x7f340c1620c0 2024-04-15T15:21:10.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.025+0000 7f340959a640 1 -- 172.21.15.105:0/449412709 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2712229023 0 0) 0x7f33f4003520 con 0x7f340c1620c0 2024-04-15T15:21:10.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.025+0000 7f340959a640 1 -- 172.21.15.105:0/449412709 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f340c1625c0 msgr2=0x7f340c162a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:10.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.025+0000 7f340959a640 1 --2- 172.21.15.105:0/449412709 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f340c1625c0 0x7f340c162a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:10.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.025+0000 7f340959a640 1 -- 172.21.15.105:0/449412709 >> v2:172.21.15.154:3300/0 conn(0x7f340c0b7910 msgr2=0x7f340c0d2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:10.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.025+0000 7f340959a640 1 --2- 172.21.15.105:0/449412709 >> v2:172.21.15.154:3300/0 conn(0x7f340c0b7910 0x7f340c0d2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:10.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.025+0000 7f340959a640 1 -- 172.21.15.105:0/449412709 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f340c0b82d0 con 0x7f340c1620c0 2024-04-15T15:21:10.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.025+0000 7f340959a640 1 -- 172.21.15.105:0/449412709 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7f33fc001bc0 con 0x7f340c1620c0 2024-04-15T15:21:10.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.025+0000 7f340959a640 1 -- 172.21.15.105:0/449412709 >> v1:172.21.15.105:6789/0 conn(0x7f340c1620c0 legacy=0x7f340c0ba190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:10.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.025+0000 7f340959a640 1 --2- 172.21.15.105:0/449412709 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33f40039b0 0x7f33f4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:10.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.025+0000 7f340959a640 1 --2- 172.21.15.105:0/449412709 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33f4004800 0x7f33f4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:10.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.025+0000 7f340959a640 1 --2- 172.21.15.105:0/449412709 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f340c1625c0 0x7f33f40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:10.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.025+0000 7f340959a640 1 -- 172.21.15.105:0/449412709 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f340c0b8c00 con 0x7f33f4004800 2024-04-15T15:21:10.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.025+0000 7f340959a640 1 -- 172.21.15.105:0/449412709 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f340c0b7ff0 con 0x7f33f40039b0 2024-04-15T15:21:10.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.025+0000 7f340959a640 1 -- 172.21.15.105:0/449412709 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f33f400b9c0 con 0x7f340c1625c0 2024-04-15T15:21:10.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.025+0000 7f3409d9b640 1 --2- 172.21.15.105:0/449412709 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33f4004800 0x7f33f4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:10.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.025+0000 7f340a59c640 1 --2- 172.21.15.105:0/449412709 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33f40039b0 0x7f33f4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:10.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.025+0000 7f340ad9d640 1 --2- 172.21.15.105:0/449412709 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f340c1625c0 0x7f33f40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:10.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.025+0000 7f3409d9b640 1 -- 172.21.15.105:0/449412709 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f340c1625c0 msgr2=0x7f33f40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:10.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.025+0000 7f3409d9b640 1 --2- 172.21.15.105:0/449412709 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f340c1625c0 0x7f33f40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:10.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.025+0000 7f3409d9b640 1 -- 172.21.15.105:0/449412709 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33f40039b0 msgr2=0x7f33f4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:10.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.025+0000 7f3409d9b640 1 --2- 172.21.15.105:0/449412709 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33f40039b0 0x7f33f4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:10.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.025+0000 7f3409d9b640 1 -- 172.21.15.105:0/449412709 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33f4003520 con 0x7f33f4004800 2024-04-15T15:21:10.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.026+0000 7f3409d9b640 1 --2- 172.21.15.105:0/449412709 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33f4004800 0x7f33f4004c80 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f3400009ac0 tx=0x7f33f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ba555c9d8b1eadf server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:10.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.026+0000 7f340959a640 1 -- 172.21.15.105:0/449412709 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3400039070 con 0x7f33f4004800 2024-04-15T15:21:10.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.026+0000 7f340959a640 1 -- 172.21.15.105:0/449412709 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f340002c710 con 0x7f33f4004800 2024-04-15T15:21:10.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.026+0000 7f340959a640 1 -- 172.21.15.105:0/449412709 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3400009690 con 0x7f33f4004800 2024-04-15T15:21:10.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.027+0000 7f3410952640 1 -- 172.21.15.105:0/449412709 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33f4004800 msgr2=0x7f33f4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:10.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.027+0000 7f3410952640 1 --2- 172.21.15.105:0/449412709 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33f4004800 0x7f33f4004c80 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f3400009ac0 tx=0x7f33f4005830 comp rx=0 tx=0).stop 2024-04-15T15:21:10.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.027+0000 7f340ad9d640 1 -- 172.21.15.105:0/449412709 reap_dead start 2024-04-15T15:21:10.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.027+0000 7f3410952640 1 -- 172.21.15.105:0/449412709 shutdown_connections 2024-04-15T15:21:10.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.027+0000 7f3410952640 1 -- 172.21.15.105:0/449412709 >> 172.21.15.105:0/449412709 conn(0x7f340c06ccb0 msgr2=0x7f340c06d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:10.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.027+0000 7f3410952640 1 -- 172.21.15.105:0/449412709 shutdown_connections 2024-04-15T15:21:10.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.027+0000 7f3410952640 1 -- 172.21.15.105:0/449412709 wait complete. 2024-04-15T15:21:10.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.028+0000 7f3410952640 1 Processor -- start 2024-04-15T15:21:10.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.028+0000 7f3410952640 1 -- start start 2024-04-15T15:21:10.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.029+0000 7f3410952640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33f4004800 0x7f340c15ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:10.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.029+0000 7f3410952640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f340c153c60 0x7f340c15e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:10.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.029+0000 7f3410952640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f340c15e930 0x7f340c158230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:10.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.029+0000 7f3410952640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f340c162a50 con 0x7f340c153c60 2024-04-15T15:21:10.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.029+0000 7f3410952640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f340c0b8890 con 0x7f33f4004800 2024-04-15T15:21:10.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.029+0000 7f3410952640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f340c162bd0 con 0x7f340c15e930 2024-04-15T15:21:10.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.029+0000 7f3409d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f340c153c60 0x7f340c15e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:10.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.029+0000 7f3409d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f340c153c60 0x7f340c15e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46794/0 (socket says 172.21.15.105:46794) 2024-04-15T15:21:10.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.029+0000 7f3409d9b640 1 -- 172.21.15.105:0/1086358375 learned_addr learned my addr 172.21.15.105:0/1086358375 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:10.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.029+0000 7f340a59c640 1 --2- 172.21.15.105:0/1086358375 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33f4004800 0x7f340c15ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:10.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.029+0000 7f340ad9d640 1 --2- 172.21.15.105:0/1086358375 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f340c15e930 0x7f340c158230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:10.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.029+0000 7f3409d9b640 1 -- 172.21.15.105:0/1086358375 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f340c15e930 msgr2=0x7f340c158230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:10.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.029+0000 7f3409d9b640 1 --2- 172.21.15.105:0/1086358375 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f340c15e930 0x7f340c158230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:10.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.029+0000 7f3409d9b640 1 -- 172.21.15.105:0/1086358375 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33f4004800 msgr2=0x7f340c15ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:10.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.029+0000 7f3409d9b640 1 --2- 172.21.15.105:0/1086358375 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33f4004800 0x7f340c15ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:10.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.029+0000 7f3409d9b640 1 -- 172.21.15.105:0/1086358375 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f340c15f120 con 0x7f340c153c60 2024-04-15T15:21:10.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.030+0000 7f3409d9b640 1 --2- 172.21.15.105:0/1086358375 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f340c153c60 0x7f340c15e3f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f3400000c00 tx=0x7f3400004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:10.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.030+0000 7f33faffd640 1 -- 172.21.15.105:0/1086358375 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3400039050 con 0x7f340c153c60 2024-04-15T15:21:10.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.030+0000 7f33faffd640 1 -- 172.21.15.105:0/1086358375 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3400045070 con 0x7f340c153c60 2024-04-15T15:21:10.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.030+0000 7f3410952640 1 -- 172.21.15.105:0/1086358375 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f340c15f460 con 0x7f340c153c60 2024-04-15T15:21:10.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.030+0000 7f33faffd640 1 -- 172.21.15.105:0/1086358375 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34000044e0 con 0x7f340c153c60 2024-04-15T15:21:10.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.030+0000 7f3410952640 1 -- 172.21.15.105:0/1086358375 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f340c15fca0 con 0x7f340c153c60 2024-04-15T15:21:10.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.031+0000 7f3410952640 1 -- 172.21.15.105:0/1086358375 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f340c068a40 con 0x7f340c153c60 2024-04-15T15:21:10.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.032+0000 7f33faffd640 1 -- 172.21.15.105:0/1086358375 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f3400003a50 con 0x7f340c153c60 2024-04-15T15:21:10.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.032+0000 7f33faffd640 1 --2- 172.21.15.105:0/1086358375 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f33ec06c980 0x7f33ec06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:10.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.032+0000 7f33faffd640 1 -- 172.21.15.105:0/1086358375 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5795+0+0 (secure 0 0 0) 0x7f34000adbb0 con 0x7f340c153c60 2024-04-15T15:21:10.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.032+0000 7f340a59c640 1 --2- 172.21.15.105:0/1086358375 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f33ec06c980 0x7f33ec06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:10.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.032+0000 7f340a59c640 1 --2- 172.21.15.105:0/1086358375 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f33ec06c980 0x7f33ec06ee40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f33fc001be0 tx=0x7f33fc006040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:10.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.036+0000 7f33faffd640 1 -- 172.21.15.105:0/1086358375 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f34000b4080 con 0x7f340c153c60 2024-04-15T15:21:10.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.211+0000 7f3410952640 1 -- 172.21.15.105:0/1086358375 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f340c15a8e0 con 0x7f340c153c60 2024-04-15T15:21:10.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.211+0000 7f33faffd640 1 -- 172.21.15.105:0/1086358375 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v35) ==== 104+0+11 (secure 0 0 0) 0x7f34000ad020 con 0x7f340c153c60 2024-04-15T15:21:10.213 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-04-15T15:21:10.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.213+0000 7f3410952640 1 -- 172.21.15.105:0/1086358375 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f33ec06c980 msgr2=0x7f33ec06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:10.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.213+0000 7f3410952640 1 --2- 172.21.15.105:0/1086358375 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f33ec06c980 0x7f33ec06ee40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f33fc001be0 tx=0x7f33fc006040 comp rx=0 tx=0).stop 2024-04-15T15:21:10.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.213+0000 7f3410952640 1 -- 172.21.15.105:0/1086358375 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f340c153c60 msgr2=0x7f340c15e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:10.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.213+0000 7f3410952640 1 --2- 172.21.15.105:0/1086358375 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f340c153c60 0x7f340c15e3f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f3400000c00 tx=0x7f3400004730 comp rx=0 tx=0).stop 2024-04-15T15:21:10.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.213+0000 7f3410952640 1 -- 172.21.15.105:0/1086358375 shutdown_connections 2024-04-15T15:21:10.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.213+0000 7f3410952640 1 --2- 172.21.15.105:0/1086358375 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f340c15e930 0x7f340c158230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:10.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.214+0000 7f3410952640 1 --2- 172.21.15.105:0/1086358375 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f33ec06c980 0x7f33ec06ee40 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:10.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.214+0000 7f3410952640 1 --2- 172.21.15.105:0/1086358375 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f340c153c60 0x7f340c15e3f0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:10.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.214+0000 7f3410952640 1 --2- 172.21.15.105:0/1086358375 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33f4004800 0x7f340c15ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:10.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.214+0000 7f3410952640 1 -- 172.21.15.105:0/1086358375 >> 172.21.15.105:0/1086358375 conn(0x7f340c06ccb0 msgr2=0x7f340c0cf3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:10.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.214+0000 7f3410952640 1 -- 172.21.15.105:0/1086358375 shutdown_connections 2024-04-15T15:21:10.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.214+0000 7f3410952640 1 -- 172.21.15.105:0/1086358375 wait complete. 2024-04-15T15:21:10.226 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-15T15:21:10.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.330+0000 7fe95b7a8640 1 Processor -- start 2024-04-15T15:21:10.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.331+0000 7fe95b7a8640 1 -- start start 2024-04-15T15:21:10.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.331+0000 7fe95b7a8640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe95415c970 0x7fe9540cdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:10.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.331+0000 7fe95b7a8640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9540ce3b0 con 0x7fe95409b150 2024-04-15T15:21:10.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.331+0000 7fe95b7a8640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9540b5450 con 0x7fe9540b8220 2024-04-15T15:21:10.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.331+0000 7fe95b7a8640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fe95415cd50 con 0x7fe95415c970 2024-04-15T15:21:10.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.332+0000 7fe959544640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe95415c970 0x7fe9540cdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:10.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.332+0000 7fe959d45640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fe95409b150 0x7fe954166410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:50888/0 (socket says 172.21.15.105:50888) 2024-04-15T15:21:10.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.332+0000 7fe959d45640 1 -- 172.21.15.105:0/293550171 learned_addr learned my addr 172.21.15.105:0/293550171 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:21:10.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.332+0000 7fe959544640 1 -- 172.21.15.105:0/293550171 >> v1:172.21.15.154:6789/0 conn(0x7fe9540b8220 legacy=0x7fe95409aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:10.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.332+0000 7fe94bfff640 1 -- 172.21.15.105:0/293550171 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 880533962 0 0) 0x7fe9540ce3b0 con 0x7fe95409b150 2024-04-15T15:21:10.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.332+0000 7fe959544640 1 -- 172.21.15.105:0/293550171 >> v1:172.21.15.105:6789/0 conn(0x7fe95409b150 legacy=0x7fe954166410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:10.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.332+0000 7fe959544640 1 -- 172.21.15.105:0/293550171 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9540b5730 con 0x7fe95415c970 2024-04-15T15:21:10.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.333+0000 7fe959544640 1 --2- 172.21.15.105:0/293550171 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe95415c970 0x7fe9540cdb80 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fe9500097a0 tx=0x7fe95002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26b18ea28b510d76 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:10.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.333+0000 7fe94bfff640 1 -- 172.21.15.105:0/293550171 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe950034070 con 0x7fe95415c970 2024-04-15T15:21:10.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.333+0000 7fe94bfff640 1 -- 172.21.15.105:0/293550171 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe950030610 con 0x7fe95415c970 2024-04-15T15:21:10.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.333+0000 7fe94bfff640 1 -- 172.21.15.105:0/293550171 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9500309e0 con 0x7fe95415c970 2024-04-15T15:21:10.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.333+0000 7fe95b7a8640 1 -- 172.21.15.105:0/293550171 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe95415c970 msgr2=0x7fe9540cdb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:10.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.333+0000 7fe95b7a8640 1 --2- 172.21.15.105:0/293550171 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe95415c970 0x7fe9540cdb80 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fe9500097a0 tx=0x7fe95002b960 comp rx=0 tx=0).stop 2024-04-15T15:21:10.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.333+0000 7fe95b7a8640 1 -- 172.21.15.105:0/293550171 shutdown_connections 2024-04-15T15:21:10.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.333+0000 7fe95b7a8640 1 --2- 172.21.15.105:0/293550171 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe95415c970 0x7fe9540cdb80 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:10.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.333+0000 7fe95b7a8640 1 -- 172.21.15.105:0/293550171 >> 172.21.15.105:0/293550171 conn(0x7fe9540b4050 msgr2=0x7fe9540b4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:10.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.333+0000 7fe95b7a8640 1 -- 172.21.15.105:0/293550171 shutdown_connections 2024-04-15T15:21:10.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.334+0000 7fe95b7a8640 1 -- 172.21.15.105:0/293550171 wait complete. 2024-04-15T15:21:10.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.335+0000 7fe95b7a8640 1 Processor -- start 2024-04-15T15:21:10.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.335+0000 7fe95b7a8640 1 -- start start 2024-04-15T15:21:10.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.335+0000 7fe95b7a8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe95409b150 0x7fe9540cc250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:10.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.335+0000 7fe95b7a8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9540b8220 0x7fe9541e5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:10.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.335+0000 7fe95b7a8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe95415c970 0x7fe9540ca6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:10.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.335+0000 7fe95b7a8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe95409b730 con 0x7fe95415c970 2024-04-15T15:21:10.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.335+0000 7fe95b7a8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe9540ce0c0 con 0x7fe95409b150 2024-04-15T15:21:10.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.335+0000 7fe95b7a8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fe95409b8b0 con 0x7fe9540b8220 2024-04-15T15:21:10.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.335+0000 7fe959d45640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe95415c970 0x7fe9540ca6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:10.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.336+0000 7fe959d45640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe95415c970 0x7fe9540ca6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46816/0 (socket says 172.21.15.105:46816) 2024-04-15T15:21:10.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.336+0000 7fe959d45640 1 -- 172.21.15.105:0/2206864701 learned_addr learned my addr 172.21.15.105:0/2206864701 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:10.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.336+0000 7fe959544640 1 --2- 172.21.15.105:0/2206864701 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe95409b150 0x7fe9540cc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:10.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.336+0000 7fe959d45640 1 -- 172.21.15.105:0/2206864701 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9540b8220 msgr2=0x7fe9541e5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:10.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.336+0000 7fe958d43640 1 --2- 172.21.15.105:0/2206864701 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9540b8220 0x7fe9541e5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:10.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.336+0000 7fe959d45640 1 --2- 172.21.15.105:0/2206864701 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9540b8220 0x7fe9541e5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:10.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.336+0000 7fe959d45640 1 -- 172.21.15.105:0/2206864701 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe95409b150 msgr2=0x7fe9540cc250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:10.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.336+0000 7fe959d45640 1 --2- 172.21.15.105:0/2206864701 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe95409b150 0x7fe9540cc250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:10.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.336+0000 7fe959d45640 1 -- 172.21.15.105:0/2206864701 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9540cb1b0 con 0x7fe95415c970 2024-04-15T15:21:10.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.336+0000 7fe959544640 1 --2- 172.21.15.105:0/2206864701 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe95409b150 0x7fe9540cc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:21:10.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.336+0000 7fe959d45640 1 --2- 172.21.15.105:0/2206864701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe95415c970 0x7fe9540ca6f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fe94c00c730 tx=0x7fe94c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:10.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.336+0000 7fe949ffb640 1 -- 172.21.15.105:0/2206864701 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe94c018070 con 0x7fe95415c970 2024-04-15T15:21:10.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.336+0000 7fe949ffb640 1 -- 172.21.15.105:0/2206864701 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe94c00ce40 con 0x7fe95415c970 2024-04-15T15:21:10.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.336+0000 7fe95b7a8640 1 -- 172.21.15.105:0/2206864701 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9540cb490 con 0x7fe95415c970 2024-04-15T15:21:10.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.336+0000 7fe949ffb640 1 -- 172.21.15.105:0/2206864701 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe94c00a880 con 0x7fe95415c970 2024-04-15T15:21:10.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.336+0000 7fe95b7a8640 1 -- 172.21.15.105:0/2206864701 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9540cba50 con 0x7fe95415c970 2024-04-15T15:21:10.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.337+0000 7fe95b7a8640 1 -- 172.21.15.105:0/2206864701 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe954068ab0 con 0x7fe95415c970 2024-04-15T15:21:10.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.338+0000 7fe949ffb640 1 -- 172.21.15.105:0/2206864701 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fe94c01b070 con 0x7fe95415c970 2024-04-15T15:21:10.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.338+0000 7fe949ffb640 1 --2- 172.21.15.105:0/2206864701 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fe93806c9b0 0x7fe93806ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:10.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.338+0000 7fe949ffb640 1 -- 172.21.15.105:0/2206864701 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5795+0+0 (secure 0 0 0) 0x7fe94c08b160 con 0x7fe95415c970 2024-04-15T15:21:10.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.338+0000 7fe959544640 1 --2- 172.21.15.105:0/2206864701 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fe93806c9b0 0x7fe93806ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:10.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.339+0000 7fe959544640 1 --2- 172.21.15.105:0/2206864701 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fe93806c9b0 0x7fe93806ee70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe950036000 tx=0x7fe95003b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:10.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.342+0000 7fe949ffb640 1 -- 172.21.15.105:0/2206864701 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe94c059540 con 0x7fe95415c970 2024-04-15T15:21:10.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.517+0000 7fe95b7a8640 1 -- 172.21.15.105:0/2206864701 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe9540b6f00 con 0x7fe95415c970 2024-04-15T15:21:10.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.517+0000 7fe949ffb640 1 -- 172.21.15.105:0/2206864701 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v35) ==== 100+0+11 (secure 0 0 0) 0x7fe94c058e40 con 0x7fe95415c970 2024-04-15T15:21:10.519 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-04-15T15:21:10.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.519+0000 7fe95b7a8640 1 -- 172.21.15.105:0/2206864701 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fe93806c9b0 msgr2=0x7fe93806ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:10.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.519+0000 7fe95b7a8640 1 --2- 172.21.15.105:0/2206864701 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fe93806c9b0 0x7fe93806ee70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe950036000 tx=0x7fe95003b040 comp rx=0 tx=0).stop 2024-04-15T15:21:10.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.519+0000 7fe95b7a8640 1 -- 172.21.15.105:0/2206864701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe95415c970 msgr2=0x7fe9540ca6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:10.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.519+0000 7fe95b7a8640 1 --2- 172.21.15.105:0/2206864701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe95415c970 0x7fe9540ca6f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fe94c00c730 tx=0x7fe94c00cbf0 comp rx=0 tx=0).stop 2024-04-15T15:21:10.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.519+0000 7fe95b7a8640 1 -- 172.21.15.105:0/2206864701 shutdown_connections 2024-04-15T15:21:10.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.519+0000 7fe95b7a8640 1 --2- 172.21.15.105:0/2206864701 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9540b8220 0x7fe9541e5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:10.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.519+0000 7fe95b7a8640 1 --2- 172.21.15.105:0/2206864701 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fe93806c9b0 0x7fe93806ee70 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:10.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.519+0000 7fe95b7a8640 1 --2- 172.21.15.105:0/2206864701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe95415c970 0x7fe9540ca6f0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:10.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.519+0000 7fe95b7a8640 1 --2- 172.21.15.105:0/2206864701 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe95409b150 0x7fe9540cc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:10.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.519+0000 7fe95b7a8640 1 -- 172.21.15.105:0/2206864701 >> 172.21.15.105:0/2206864701 conn(0x7fe9540b4050 msgr2=0x7fe9540b7700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:10.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.519+0000 7fe95b7a8640 1 -- 172.21.15.105:0/2206864701 shutdown_connections 2024-04-15T15:21:10.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.519+0000 7fe95b7a8640 1 -- 172.21.15.105:0/2206864701 wait complete. 2024-04-15T15:21:10.532 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-15T15:21:10.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.634+0000 7f5de584a640 1 Processor -- start 2024-04-15T15:21:10.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.634+0000 7f5de584a640 1 -- start start 2024-04-15T15:21:10.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.634+0000 7f5de584a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5de015ee60 0x7f5de00ba110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:10.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.635+0000 7f5de584a640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5de00cf550 0x7f5de00cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:10.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.635+0000 7f5de584a640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5de00d2d00 con 0x7f5de00d2870 2024-04-15T15:21:10.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.635+0000 7f5de584a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f5de00cff00 con 0x7f5de015ee60 2024-04-15T15:21:10.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.635+0000 7f5de584a640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f5de00d44b0 con 0x7f5de00cf550 2024-04-15T15:21:10.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.635+0000 7f5ddeffd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5de015ee60 0x7f5de00ba110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:10.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.635+0000 7f5ddeffd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5de015ee60 0x7f5de00ba110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33094/0 (socket says 172.21.15.105:33094) 2024-04-15T15:21:10.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.635+0000 7f5ddeffd640 1 -- 172.21.15.105:0/2002201931 learned_addr learned my addr 172.21.15.105:0/2002201931 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:10.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.635+0000 7f5ddf7fe640 1 --2- 172.21.15.105:0/2002201931 >> v2:172.21.15.154:3300/0 conn(0x7f5de00cf550 0x7f5de00cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:10.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.635+0000 7f5ddeffd640 1 -- 172.21.15.105:0/2002201931 >> v2:172.21.15.154:3300/0 conn(0x7f5de00cf550 msgr2=0x7f5de00cf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:10.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.635+0000 7f5ddeffd640 1 --2- 172.21.15.105:0/2002201931 >> v2:172.21.15.154:3300/0 conn(0x7f5de00cf550 0x7f5de00cf9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:10.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.635+0000 7f5ddeffd640 1 -- 172.21.15.105:0/2002201931 >> v1:172.21.15.154:6789/0 conn(0x7f5de00d2870 legacy=0x7f5de00d3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:10.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.635+0000 7f5ddeffd640 1 -- 172.21.15.105:0/2002201931 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5de00d2fe0 con 0x7f5de015ee60 2024-04-15T15:21:10.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.636+0000 7f5ddeffd640 1 --2- 172.21.15.105:0/2002201931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5de015ee60 0x7f5de00ba110 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f5de00d2110 tx=0x7f5dd002ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2225f0b54924d9a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:10.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.636+0000 7f5dde7fc640 1 -- 172.21.15.105:0/2002201931 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd002f850 con 0x7f5de015ee60 2024-04-15T15:21:10.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.636+0000 7f5dde7fc640 1 -- 172.21.15.105:0/2002201931 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dd002f9f0 con 0x7f5de015ee60 2024-04-15T15:21:10.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.636+0000 7f5dde7fc640 1 -- 172.21.15.105:0/2002201931 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd002fdc0 con 0x7f5de015ee60 2024-04-15T15:21:10.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.636+0000 7f5de584a640 1 -- 172.21.15.105:0/2002201931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5de015ee60 msgr2=0x7f5de00ba110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:10.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.636+0000 7f5de584a640 1 --2- 172.21.15.105:0/2002201931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5de015ee60 0x7f5de00ba110 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f5de00d2110 tx=0x7f5dd002ba30 comp rx=0 tx=0).stop 2024-04-15T15:21:10.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.636+0000 7f5de584a640 1 -- 172.21.15.105:0/2002201931 shutdown_connections 2024-04-15T15:21:10.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.636+0000 7f5de584a640 1 --2- 172.21.15.105:0/2002201931 >> v2:172.21.15.154:3300/0 conn(0x7f5de00cf550 0x7f5de00cf9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:10.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.636+0000 7f5de584a640 1 --2- 172.21.15.105:0/2002201931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5de015ee60 0x7f5de00ba110 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:10.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.636+0000 7f5de584a640 1 -- 172.21.15.105:0/2002201931 >> 172.21.15.105:0/2002201931 conn(0x7f5de0067b40 msgr2=0x7f5de0067f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:10.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.637+0000 7f5de584a640 1 -- 172.21.15.105:0/2002201931 shutdown_connections 2024-04-15T15:21:10.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.637+0000 7f5de584a640 1 -- 172.21.15.105:0/2002201931 wait complete. 2024-04-15T15:21:10.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.638+0000 7f5de584a640 1 Processor -- start 2024-04-15T15:21:10.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.638+0000 7f5de584a640 1 -- start start 2024-04-15T15:21:10.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.639+0000 7f5de584a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5de00cf550 0x7f5de00af050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:10.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.639+0000 7f5de584a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5de015ee60 0x7f5de00cce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:10.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.639+0000 7f5de584a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5de00cd380 0x7f5de014e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:10.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.639+0000 7f5de584a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5de00d3880 con 0x7f5de00cd380 2024-04-15T15:21:10.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.639+0000 7f5de584a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5de00d35a0 con 0x7f5de00cf550 2024-04-15T15:21:10.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.639+0000 7f5de584a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f5de00ba650 con 0x7f5de015ee60 2024-04-15T15:21:10.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.639+0000 7f5ddf7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5de00cd380 0x7f5de014e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:10.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.639+0000 7f5ddf7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5de00cd380 0x7f5de014e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33104/0 (socket says 172.21.15.105:33104) 2024-04-15T15:21:10.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.639+0000 7f5ddf7fe640 1 -- 172.21.15.105:0/1945527254 learned_addr learned my addr 172.21.15.105:0/1945527254 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:10.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.639+0000 7f5ddeffd640 1 --2- 172.21.15.105:0/1945527254 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5de00cf550 0x7f5de00af050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:10.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.639+0000 7f5dd67fc640 1 --2- 172.21.15.105:0/1945527254 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5de015ee60 0x7f5de00cce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:10.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.639+0000 7f5ddf7fe640 1 -- 172.21.15.105:0/1945527254 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5de015ee60 msgr2=0x7f5de00cce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:10.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.639+0000 7f5ddf7fe640 1 --2- 172.21.15.105:0/1945527254 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5de015ee60 0x7f5de00cce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:10.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.639+0000 7f5ddf7fe640 1 -- 172.21.15.105:0/1945527254 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5de00cf550 msgr2=0x7f5de00af050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:10.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.639+0000 7f5ddf7fe640 1 --2- 172.21.15.105:0/1945527254 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5de00cf550 0x7f5de00af050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:10.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.639+0000 7f5ddf7fe640 1 -- 172.21.15.105:0/1945527254 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5de014f260 con 0x7f5de00cd380 2024-04-15T15:21:10.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.639+0000 7f5ddf7fe640 1 --2- 172.21.15.105:0/1945527254 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5de00cd380 0x7f5de014e5e0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8007e40 tx=0x7f5dd800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:10.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.639+0000 7f5ddcff9640 1 -- 172.21.15.105:0/1945527254 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd8014620 con 0x7f5de00cd380 2024-04-15T15:21:10.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.639+0000 7f5ddcff9640 1 -- 172.21.15.105:0/1945527254 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dd8019070 con 0x7f5de00cd380 2024-04-15T15:21:10.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.639+0000 7f5ddcff9640 1 -- 172.21.15.105:0/1945527254 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd8014a20 con 0x7f5de00cd380 2024-04-15T15:21:10.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.639+0000 7f5de584a640 1 -- 172.21.15.105:0/1945527254 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5de014f540 con 0x7f5de00cd380 2024-04-15T15:21:10.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.639+0000 7f5de584a640 1 -- 172.21.15.105:0/1945527254 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5de014fb00 con 0x7f5de00cd380 2024-04-15T15:21:10.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.640+0000 7f5de584a640 1 -- 172.21.15.105:0/1945527254 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5de00cfc60 con 0x7f5de00cd380 2024-04-15T15:21:10.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.641+0000 7f5ddcff9640 1 -- 172.21.15.105:0/1945527254 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f5dd8014bc0 con 0x7f5de00cd380 2024-04-15T15:21:10.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.641+0000 7f5ddcff9640 1 --2- 172.21.15.105:0/1945527254 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f5dc006c9b0 0x7f5dc006ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:10.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.641+0000 7f5ddcff9640 1 -- 172.21.15.105:0/1945527254 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5795+0+0 (secure 0 0 0) 0x7f5dd808c2f0 con 0x7f5de00cd380 2024-04-15T15:21:10.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.641+0000 7f5ddeffd640 1 --2- 172.21.15.105:0/1945527254 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f5dc006c9b0 0x7f5dc006ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:10.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.642+0000 7f5ddeffd640 1 --2- 172.21.15.105:0/1945527254 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f5dc006c9b0 0x7f5dc006ee70 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5dd0036000 tx=0x7f5dd0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:10.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.646+0000 7f5ddcff9640 1 -- 172.21.15.105:0/1945527254 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5de00cfc60 con 0x7f5de00cd380 2024-04-15T15:21:10.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.833+0000 7f5de584a640 1 -- 172.21.15.105:0/1945527254 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5de0068a40 con 0x7f5de00cd380 2024-04-15T15:21:10.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.833+0000 7f5ddcff9640 1 -- 172.21.15.105:0/1945527254 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v35) ==== 80+0+122 (secure 0 0 0) 0x7f5dd805a110 con 0x7f5de00cd380 2024-04-15T15:21:10.836 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.105:6837/958066041 2024-04-16T15:21:03.660841+0000 2024-04-15T15:21:10.836 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.105:6835/958066041 2024-04-16T15:21:03.660841+0000 2024-04-15T15:21:10.836 INFO:teuthology.orchestra.run.smithi105.stderr:listed 2 entries 2024-04-15T15:21:10.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.835+0000 7f5de584a640 1 -- 172.21.15.105:0/1945527254 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f5dc006c9b0 msgr2=0x7f5dc006ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:10.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.835+0000 7f5de584a640 1 --2- 172.21.15.105:0/1945527254 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f5dc006c9b0 0x7f5dc006ee70 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5dd0036000 tx=0x7f5dd0008030 comp rx=0 tx=0).stop 2024-04-15T15:21:10.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.835+0000 7f5de584a640 1 -- 172.21.15.105:0/1945527254 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5de00cd380 msgr2=0x7f5de014e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:10.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.835+0000 7f5de584a640 1 --2- 172.21.15.105:0/1945527254 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5de00cd380 0x7f5de014e5e0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8007e40 tx=0x7f5dd800e4b0 comp rx=0 tx=0).stop 2024-04-15T15:21:10.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.835+0000 7f5de584a640 1 -- 172.21.15.105:0/1945527254 shutdown_connections 2024-04-15T15:21:10.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.835+0000 7f5de584a640 1 --2- 172.21.15.105:0/1945527254 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5de015ee60 0x7f5de00cce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:10.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.835+0000 7f5de584a640 1 --2- 172.21.15.105:0/1945527254 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f5dc006c9b0 0x7f5dc006ee70 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:10.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.835+0000 7f5de584a640 1 --2- 172.21.15.105:0/1945527254 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5de00cd380 0x7f5de014e5e0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:10.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.835+0000 7f5de584a640 1 --2- 172.21.15.105:0/1945527254 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5de00cf550 0x7f5de00af050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:10.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.835+0000 7f5de584a640 1 -- 172.21.15.105:0/1945527254 >> 172.21.15.105:0/1945527254 conn(0x7f5de0067b40 msgr2=0x7f5de0160560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:10.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.836+0000 7f5de584a640 1 -- 172.21.15.105:0/1945527254 shutdown_connections 2024-04-15T15:21:10.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:10.836+0000 7f5de584a640 1 -- 172.21.15.105:0/1945527254 wait complete. 2024-04-15T15:21:10.848 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi105.front.sepia.ceph.com... 2024-04-15T15:21:10.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:21:10.848 DEBUG:teuthology.orchestra.run.smithi105:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-15T15:21:10.875 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi105.stderr:ceph-fuse[53766]: fuse finished with error 0 and tester_r 0 2024-04-15T15:21:10.877 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi105.front.sepia.ceph.com 2024-04-15T15:21:10.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:21:10.877 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-15T15:21:11.044 INFO:tasks.cephfs.mount:Terminating background process 2024-04-15T15:21:11.044 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-15T15:21:11.044 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:21:11.064 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:21:11.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.186+0000 7ff0b76b1640 1 Processor -- start 2024-04-15T15:21:11.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.186+0000 7ff0b76b1640 1 -- start start 2024-04-15T15:21:11.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.187+0000 7ff0b76b1640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff0b0145e00 0x7ff0b00ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:11.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.187+0000 7ff0b76b1640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0b0072570 0x7ff0b006df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:11.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.187+0000 7ff0b76b1640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0b00ccfb0 con 0x7ff0b00ced30 2024-04-15T15:21:11.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.187+0000 7ff0b76b1640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7ff0b00cdbc0 con 0x7ff0b0145e00 2024-04-15T15:21:11.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.187+0000 7ff0b76b1640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7ff0b00cd290 con 0x7ff0b0072570 2024-04-15T15:21:11.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.187+0000 7ff0b4c4c640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7ff0b00ced30 0x7ff0b0071e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.105:36224/0 (socket says 172.21.15.105:36224) 2024-04-15T15:21:11.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.187+0000 7ff0b5c4e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0b0072570 0x7ff0b006df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:11.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.187+0000 7ff0b544d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff0b0145e00 0x7ff0b00ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:11.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.187+0000 7ff0b4c4c640 1 -- 172.21.15.105:0/3503605743 learned_addr learned my addr 172.21.15.105:0/3503605743 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:21:11.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.188+0000 7ff0a7fff640 1 -- 172.21.15.105:0/3503605743 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2878559074 0 0) 0x7ff0b00ccfb0 con 0x7ff0b00ced30 2024-04-15T15:21:11.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.188+0000 7ff0a7fff640 1 -- 172.21.15.105:0/3503605743 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff09c003520 con 0x7ff0b00ced30 2024-04-15T15:21:11.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.188+0000 7ff0b5c4e640 1 -- 172.21.15.105:0/3503605743 >> v2:172.21.15.154:3300/0 conn(0x7ff0b0145e00 msgr2=0x7ff0b00ce7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:11.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.188+0000 7ff0b5c4e640 1 --2- 172.21.15.105:0/3503605743 >> v2:172.21.15.154:3300/0 conn(0x7ff0b0145e00 0x7ff0b00ce7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:11.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.188+0000 7ff0b5c4e640 1 -- 172.21.15.105:0/3503605743 >> v1:172.21.15.154:6789/0 conn(0x7ff0b00ced30 legacy=0x7ff0b0071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:11.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.188+0000 7ff0b5c4e640 1 -- 172.21.15.105:0/3503605743 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0b00cd570 con 0x7ff0b0072570 2024-04-15T15:21:11.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.188+0000 7ff0b544d640 1 --2- 172.21.15.105:0/3503605743 >> v2:172.21.15.154:3300/0 conn(0x7ff0b0145e00 0x7ff0b00ce7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:21:11.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.188+0000 7ff0b5c4e640 1 --2- 172.21.15.105:0/3503605743 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0b0072570 0x7ff0b006df80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff0a8009d90 tx=0x7ff0a802bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76fa76aa0f210d6d server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:11.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.189+0000 7ff0a7fff640 1 -- 172.21.15.105:0/3503605743 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a8039070 con 0x7ff0b0072570 2024-04-15T15:21:11.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.189+0000 7ff0a7fff640 1 -- 172.21.15.105:0/3503605743 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0a802cc50 con 0x7ff0b0072570 2024-04-15T15:21:11.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.189+0000 7ff0a7fff640 1 -- 172.21.15.105:0/3503605743 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a8034440 con 0x7ff0b0072570 2024-04-15T15:21:11.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.189+0000 7ff0b76b1640 1 -- 172.21.15.105:0/3503605743 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0b0072570 msgr2=0x7ff0b006df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:11.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.189+0000 7ff0b76b1640 1 --2- 172.21.15.105:0/3503605743 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0b0072570 0x7ff0b006df80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff0a8009d90 tx=0x7ff0a802bea0 comp rx=0 tx=0).stop 2024-04-15T15:21:11.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.189+0000 7ff0b76b1640 1 -- 172.21.15.105:0/3503605743 shutdown_connections 2024-04-15T15:21:11.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.189+0000 7ff0b76b1640 1 --2- 172.21.15.105:0/3503605743 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0b0072570 0x7ff0b006df80 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:11.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.189+0000 7ff0b76b1640 1 --2- 172.21.15.105:0/3503605743 >> v2:172.21.15.154:3300/0 conn(0x7ff0b0145e00 0x7ff0b00ce7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:11.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.189+0000 7ff0b76b1640 1 -- 172.21.15.105:0/3503605743 >> 172.21.15.105:0/3503605743 conn(0x7ff0b0067d50 msgr2=0x7ff0b006b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:11.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.189+0000 7ff0b76b1640 1 -- 172.21.15.105:0/3503605743 shutdown_connections 2024-04-15T15:21:11.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.189+0000 7ff0b76b1640 1 -- 172.21.15.105:0/3503605743 wait complete. 2024-04-15T15:21:11.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.190+0000 7ff0b76b1640 1 Processor -- start 2024-04-15T15:21:11.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.190+0000 7ff0b76b1640 1 -- start start 2024-04-15T15:21:11.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.191+0000 7ff0b76b1640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0b0072570 0x7ff0b00aeee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:11.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.191+0000 7ff0b76b1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0b00ced30 0x7ff0b00b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:11.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.191+0000 7ff0b76b1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0b0145e00 0x7ff0b01598a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:11.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.191+0000 7ff0b76b1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b01461e0 con 0x7ff0b00ced30 2024-04-15T15:21:11.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.191+0000 7ff0b76b1640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b00ce370 con 0x7ff0b0145e00 2024-04-15T15:21:11.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.191+0000 7ff0b76b1640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7ff0b0146360 con 0x7ff0b0072570 2024-04-15T15:21:11.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.191+0000 7ff0b4c4c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0b00ced30 0x7ff0b00b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:11.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.191+0000 7ff0b4c4c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0b00ced30 0x7ff0b00b28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33120/0 (socket says 172.21.15.105:33120) 2024-04-15T15:21:11.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.191+0000 7ff0b4c4c640 1 -- 172.21.15.105:0/4187172105 learned_addr learned my addr 172.21.15.105:0/4187172105 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:11.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.191+0000 7ff0b544d640 1 --2- 172.21.15.105:0/4187172105 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0b0072570 0x7ff0b00aeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:11.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.191+0000 7ff0b5c4e640 1 --2- 172.21.15.105:0/4187172105 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0b0145e00 0x7ff0b01598a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:11.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.192+0000 7ff0b4c4c640 1 -- 172.21.15.105:0/4187172105 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0b0072570 msgr2=0x7ff0b00aeee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:11.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.192+0000 7ff0b4c4c640 1 --2- 172.21.15.105:0/4187172105 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0b0072570 0x7ff0b00aeee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:11.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.192+0000 7ff0b4c4c640 1 -- 172.21.15.105:0/4187172105 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0b0145e00 msgr2=0x7ff0b01598a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:11.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.192+0000 7ff0b4c4c640 1 --2- 172.21.15.105:0/4187172105 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0b0145e00 0x7ff0b01598a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:11.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.192+0000 7ff0b4c4c640 1 -- 172.21.15.105:0/4187172105 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0b00ae440 con 0x7ff0b00ced30 2024-04-15T15:21:11.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.192+0000 7ff0b5c4e640 1 --2- 172.21.15.105:0/4187172105 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0b0145e00 0x7ff0b01598a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:21:11.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.192+0000 7ff0b4c4c640 1 --2- 172.21.15.105:0/4187172105 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0b00ced30 0x7ff0b00b28c0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7ff0a000c500 tx=0x7ff0a000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:11.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.192+0000 7ff0a5ffb640 1 -- 172.21.15.105:0/4187172105 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a0017070 con 0x7ff0b00ced30 2024-04-15T15:21:11.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.192+0000 7ff0a5ffb640 1 -- 172.21.15.105:0/4187172105 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0a0005ce0 con 0x7ff0b00ced30 2024-04-15T15:21:11.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.192+0000 7ff0a5ffb640 1 -- 172.21.15.105:0/4187172105 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a0007330 con 0x7ff0b00ced30 2024-04-15T15:21:11.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.192+0000 7ff0b76b1640 1 -- 172.21.15.105:0/4187172105 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0b0157ca0 con 0x7ff0b00ced30 2024-04-15T15:21:11.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.192+0000 7ff0b76b1640 1 -- 172.21.15.105:0/4187172105 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0b0158260 con 0x7ff0b00ced30 2024-04-15T15:21:11.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.193+0000 7ff0b76b1640 1 -- 172.21.15.105:0/4187172105 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0b0068a40 con 0x7ff0b00ced30 2024-04-15T15:21:11.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.194+0000 7ff0a5ffb640 1 -- 172.21.15.105:0/4187172105 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7ff0a001a070 con 0x7ff0b00ced30 2024-04-15T15:21:11.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.194+0000 7ff0a5ffb640 1 --2- 172.21.15.105:0/4187172105 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff09406c980 0x7ff09406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:11.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.194+0000 7ff0a5ffb640 1 -- 172.21.15.105:0/4187172105 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5795+0+0 (secure 0 0 0) 0x7ff0a008afd0 con 0x7ff0b00ced30 2024-04-15T15:21:11.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.194+0000 7ff0b544d640 1 --2- 172.21.15.105:0/4187172105 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff09406c980 0x7ff09406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:11.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.194+0000 7ff0b544d640 1 --2- 172.21.15.105:0/4187172105 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff09406c980 0x7ff09406ee40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac009a30 tx=0x7ff0ac008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:11.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.198+0000 7ff0a5ffb640 1 -- 172.21.15.105:0/4187172105 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff0a0059500 con 0x7ff0b00ced30 2024-04-15T15:21:11.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.373+0000 7ff0b76b1640 1 -- 172.21.15.105:0/4187172105 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff0b00af420 con 0x7ff0b00ced30 2024-04-15T15:21:11.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.374+0000 7ff0a5ffb640 1 -- 172.21.15.105:0/4187172105 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16846 (secure 0 0 0) 0x7ff0a0058e00 con 0x7ff0b00ced30 2024-04-15T15:21:11.376 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:21:11.376 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":35,"fsid":"09e4d2a4-a324-4793-af77-644b25e520e1","created":"2024-04-15T15:19:01.524543+0000","modified":"2024-04-15T15:21:03.660870+0000","last_up_change":"2024-04-15T15:19:12.825815+0000","last_in_change":"2024-04-15T15:19:03.788126+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:19:13.748585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:20:26.983353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-15T15:20:27.961802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6f6292b-2d7e-4aa0-9d1c-6ddd0f97287c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6801","nonce":2873040054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6803","nonce":2873040054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6807","nonce":2873040054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6805","nonce":2873040054}]},"public_addr":"172.21.15.105:6801/2873040054","cluster_addr":"172.21.15.105:6803/2873040054","heartbeat_back_addr":"172.21.15.105:6807/2873040054","heartbeat_front_addr":"172.21.15.105:6805/2873040054","state":["exists","up"]},{"osd":1,"uuid":"8900966a-b393-41e3-8f93-e5a43bd23622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6818","nonce":2494994479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6821","nonce":2494994479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6828","nonce":2494994479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6823","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6825","nonce":2494994479}]},"public_addr":"172.21.15.105:6818/2494994479","cluster_addr":"172.21.15.105:6821/2494994479","heartbeat_back_addr":"172.21.15.105:6828/2494994479","heartbeat_front_addr":"172.21.15.105:6825/2494994479","state":["exists","up"]},{"osd":2,"uuid":"81a5dc7a-9325-4ff5-941d-a2d5fba21f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6809","nonce":3798408699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6811","nonce":3798408699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6815","nonce":3798408699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6813","nonce":3798408699}]},"public_addr":"172.21.15.105:6809/3798408699","cluster_addr":"172.21.15.105:6811/3798408699","heartbeat_back_addr":"172.21.15.105:6815/3798408699","heartbeat_front_addr":"172.21.15.105:6813/3798408699","state":["exists","up"]},{"osd":3,"uuid":"df766ed2-5c71-4805-96f4-3d4970edc960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6819","nonce":1920720186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6824","nonce":1920720186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6831","nonce":1920720186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6827","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6829","nonce":1920720186}]},"public_addr":"172.21.15.105:6819/1920720186","cluster_addr":"172.21.15.105:6824/1920720186","heartbeat_back_addr":"172.21.15.105:6831/1920720186","heartbeat_front_addr":"172.21.15.105:6829/1920720186","state":["exists","up"]},{"osd":4,"uuid":"e1039e03-28b8-49df-b951-c91930b93c9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6810","nonce":3516494456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6813","nonce":3516494456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6820","nonce":3516494456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6817","nonce":3516494456}]},"public_addr":"172.21.15.154:6810/3516494456","cluster_addr":"172.21.15.154:6813/3516494456","heartbeat_back_addr":"172.21.15.154:6820/3516494456","heartbeat_front_addr":"172.21.15.154:6817/3516494456","state":["exists","up"]},{"osd":5,"uuid":"a9f5eaf1-66ff-405a-a9e4-ec3340812a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6801","nonce":540367751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6803","nonce":540367751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6807","nonce":540367751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6805","nonce":540367751}]},"public_addr":"172.21.15.154:6801/540367751","cluster_addr":"172.21.15.154:6803/540367751","heartbeat_back_addr":"172.21.15.154:6807/540367751","heartbeat_front_addr":"172.21.15.154:6805/540367751","state":["exists","up"]},{"osd":6,"uuid":"267d02ea-675f-428f-aae8-e0375e4d4feb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6825","nonce":1550829005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6827","nonce":1550829005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6831","nonce":1550829005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6829","nonce":1550829005}]},"public_addr":"172.21.15.154:6825/1550829005","cluster_addr":"172.21.15.154:6827/1550829005","heartbeat_back_addr":"172.21.15.154:6831/1550829005","heartbeat_front_addr":"172.21.15.154:6829/1550829005","state":["exists","up"]},{"osd":7,"uuid":"759ac6cc-896a-423c-885f-9fe90d8dcb14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6809","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6811","nonce":3920089903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6816","nonce":3920089903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6823","nonce":3920089903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6819","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6821","nonce":3920089903}]},"public_addr":"172.21.15.154:6811/3920089903","cluster_addr":"172.21.15.154:6816/3920089903","heartbeat_back_addr":"172.21.15.154:6823/3920089903","heartbeat_front_addr":"172.21.15.154:6821/3920089903","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.949509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.984072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.907278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.947391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.975447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.998896+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:09.028739+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.894119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.105:6837/958066041":"2024-04-16T15:21:03.660841+0000","172.21.15.105:6835/958066041":"2024-04-16T15:21:03.660841+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:21:11.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.376+0000 7ff0b76b1640 1 -- 172.21.15.105:0/4187172105 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff09406c980 msgr2=0x7ff09406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:11.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.376+0000 7ff0b76b1640 1 --2- 172.21.15.105:0/4187172105 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff09406c980 0x7ff09406ee40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac009a30 tx=0x7ff0ac008040 comp rx=0 tx=0).stop 2024-04-15T15:21:11.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.376+0000 7ff0b76b1640 1 -- 172.21.15.105:0/4187172105 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0b00ced30 msgr2=0x7ff0b00b28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:11.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.376+0000 7ff0b76b1640 1 --2- 172.21.15.105:0/4187172105 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0b00ced30 0x7ff0b00b28c0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7ff0a000c500 tx=0x7ff0a000c9c0 comp rx=0 tx=0).stop 2024-04-15T15:21:11.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.376+0000 7ff0b76b1640 1 -- 172.21.15.105:0/4187172105 shutdown_connections 2024-04-15T15:21:11.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.376+0000 7ff0b76b1640 1 --2- 172.21.15.105:0/4187172105 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff09406c980 0x7ff09406ee40 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:11.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.376+0000 7ff0b76b1640 1 --2- 172.21.15.105:0/4187172105 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0b0145e00 0x7ff0b01598a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:11.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.376+0000 7ff0b76b1640 1 --2- 172.21.15.105:0/4187172105 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0b00ced30 0x7ff0b00b28c0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:11.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.376+0000 7ff0b76b1640 1 --2- 172.21.15.105:0/4187172105 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0b0072570 0x7ff0b00aeee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:11.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.376+0000 7ff0b76b1640 1 -- 172.21.15.105:0/4187172105 >> 172.21.15.105:0/4187172105 conn(0x7ff0b0067d50 msgr2=0x7ff0b006b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:11.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.376+0000 7ff0b76b1640 1 -- 172.21.15.105:0/4187172105 shutdown_connections 2024-04-15T15:21:11.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.376+0000 7ff0b76b1640 1 -- 172.21.15.105:0/4187172105 wait complete. 2024-04-15T15:21:11.389 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:19:13.748585+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-15T15:20:26.983353+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-15T15:20:27.961802+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:21:11.390 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:21:11.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.494+0000 7f8e23ca5640 1 Processor -- start 2024-04-15T15:21:11.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.494+0000 7f8e23ca5640 1 -- start start 2024-04-15T15:21:11.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.495+0000 7f8e23ca5640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8e1c162040 0x7f8e1c06d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:11.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.495+0000 7f8e23ca5640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e1c0ccc40 con 0x7f8e1c0ce430 2024-04-15T15:21:11.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.495+0000 7f8e23ca5640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e1c09af40 con 0x7f8e1c09aa40 2024-04-15T15:21:11.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.495+0000 7f8e23ca5640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f8e1c09b220 con 0x7f8e1c162040 2024-04-15T15:21:11.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.495+0000 7f8e21a41640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8e1c162040 0x7f8e1c06d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:11.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.495+0000 7f8e21a41640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8e1c162040 0x7f8e1c06d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33136/0 (socket says 172.21.15.105:33136) 2024-04-15T15:21:11.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.495+0000 7f8e21a41640 1 -- 172.21.15.105:0/2544027824 learned_addr learned my addr 172.21.15.105:0/2544027824 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:11.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.495+0000 7f8e21a41640 1 -- 172.21.15.105:0/2544027824 >> v1:172.21.15.154:6789/0 conn(0x7f8e1c09aa40 legacy=0x7f8e1c0d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:11.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.495+0000 7f8e21a41640 1 -- 172.21.15.105:0/2544027824 >> v1:172.21.15.105:6789/0 conn(0x7f8e1c0ce430 legacy=0x7f8e1c0d0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:11.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.495+0000 7f8e21a41640 1 -- 172.21.15.105:0/2544027824 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e1c09b500 con 0x7f8e1c162040 2024-04-15T15:21:11.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.496+0000 7f8e21a41640 1 --2- 172.21.15.105:0/2544027824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e1c162040 0x7f8e1c06d2c0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f8e14009870 tx=0x7f8e1402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdc0472d3e04b213 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:11.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.496+0000 7f8e20a3f640 1 -- 172.21.15.105:0/2544027824 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e14034070 con 0x7f8e1c162040 2024-04-15T15:21:11.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.496+0000 7f8e20a3f640 1 -- 172.21.15.105:0/2544027824 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e14030860 con 0x7f8e1c162040 2024-04-15T15:21:11.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.496+0000 7f8e20a3f640 1 -- 172.21.15.105:0/2544027824 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e14030c30 con 0x7f8e1c162040 2024-04-15T15:21:11.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.497+0000 7f8e23ca5640 1 -- 172.21.15.105:0/2544027824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e1c162040 msgr2=0x7f8e1c06d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:11.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.497+0000 7f8e23ca5640 1 --2- 172.21.15.105:0/2544027824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e1c162040 0x7f8e1c06d2c0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f8e14009870 tx=0x7f8e1402b9a0 comp rx=0 tx=0).stop 2024-04-15T15:21:11.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.497+0000 7f8e23ca5640 1 -- 172.21.15.105:0/2544027824 shutdown_connections 2024-04-15T15:21:11.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.497+0000 7f8e23ca5640 1 --2- 172.21.15.105:0/2544027824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e1c162040 0x7f8e1c06d2c0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:11.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.497+0000 7f8e23ca5640 1 -- 172.21.15.105:0/2544027824 >> 172.21.15.105:0/2544027824 conn(0x7f8e1c0d25f0 msgr2=0x7f8e1c0d2a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:11.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.497+0000 7f8e23ca5640 1 -- 172.21.15.105:0/2544027824 shutdown_connections 2024-04-15T15:21:11.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.497+0000 7f8e23ca5640 1 -- 172.21.15.105:0/2544027824 wait complete. 2024-04-15T15:21:11.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.498+0000 7f8e23ca5640 1 Processor -- start 2024-04-15T15:21:11.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.498+0000 7f8e23ca5640 1 -- start start 2024-04-15T15:21:11.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.499+0000 7f8e23ca5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e1c09aa40 0x7f8e1c152a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:11.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.499+0000 7f8e23ca5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e1c0ce430 0x7f8e1c0c6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:11.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.499+0000 7f8e23ca5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e1c162040 0x7f8e1c15db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:11.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.499+0000 7f8e23ca5640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8e1c0d1510 con 0x7f8e1c0ce430 2024-04-15T15:21:11.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.499+0000 7f8e23ca5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8e1c162420 con 0x7f8e1c162040 2024-04-15T15:21:11.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.499+0000 7f8e23ca5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f8e1c0d3390 con 0x7f8e1c09aa40 2024-04-15T15:21:11.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.499+0000 7f8e21240640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e1c0ce430 0x7f8e1c0c6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:11.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.499+0000 7f8e21240640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e1c0ce430 0x7f8e1c0c6b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33142/0 (socket says 172.21.15.105:33142) 2024-04-15T15:21:11.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.499+0000 7f8e21240640 1 -- 172.21.15.105:0/3352443832 learned_addr learned my addr 172.21.15.105:0/3352443832 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:11.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.499+0000 7f8e22242640 1 --2- 172.21.15.105:0/3352443832 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e1c162040 0x7f8e1c15db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:11.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.499+0000 7f8e21a41640 1 --2- 172.21.15.105:0/3352443832 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e1c09aa40 0x7f8e1c152a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:11.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.499+0000 7f8e21240640 1 -- 172.21.15.105:0/3352443832 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e1c09aa40 msgr2=0x7f8e1c152a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:11.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.499+0000 7f8e21240640 1 --2- 172.21.15.105:0/3352443832 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e1c09aa40 0x7f8e1c152a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:11.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.499+0000 7f8e21240640 1 -- 172.21.15.105:0/3352443832 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e1c162040 msgr2=0x7f8e1c15db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:11.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.499+0000 7f8e21240640 1 --2- 172.21.15.105:0/3352443832 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e1c162040 0x7f8e1c15db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:11.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.499+0000 7f8e21240640 1 -- 172.21.15.105:0/3352443832 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e1c0c74d0 con 0x7f8e1c0ce430 2024-04-15T15:21:11.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.499+0000 7f8e21a41640 1 --2- 172.21.15.105:0/3352443832 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e1c09aa40 0x7f8e1c152a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:21:11.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.499+0000 7f8e21240640 1 --2- 172.21.15.105:0/3352443832 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e1c0ce430 0x7f8e1c0c6b30 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c00c430 tx=0x7f8e0c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:11.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.499+0000 7f8e127fc640 1 -- 172.21.15.105:0/3352443832 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c011790 con 0x7f8e1c0ce430 2024-04-15T15:21:11.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.499+0000 7f8e127fc640 1 -- 172.21.15.105:0/3352443832 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e0c017070 con 0x7f8e1c0ce430 2024-04-15T15:21:11.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.499+0000 7f8e23ca5640 1 -- 172.21.15.105:0/3352443832 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e1c0c77b0 con 0x7f8e1c0ce430 2024-04-15T15:21:11.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.499+0000 7f8e127fc640 1 -- 172.21.15.105:0/3352443832 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c011bb0 con 0x7f8e1c0ce430 2024-04-15T15:21:11.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.500+0000 7f8e23ca5640 1 -- 172.21.15.105:0/3352443832 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e1c0c7d70 con 0x7f8e1c0ce430 2024-04-15T15:21:11.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.500+0000 7f8e23ca5640 1 -- 172.21.15.105:0/3352443832 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e1c068ab0 con 0x7f8e1c0ce430 2024-04-15T15:21:11.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.501+0000 7f8e127fc640 1 -- 172.21.15.105:0/3352443832 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f8e0c01c460 con 0x7f8e1c0ce430 2024-04-15T15:21:11.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.501+0000 7f8e127fc640 1 --2- 172.21.15.105:0/3352443832 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f8e0006c980 0x7f8e0006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:11.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.501+0000 7f8e127fc640 1 -- 172.21.15.105:0/3352443832 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5795+0+0 (secure 0 0 0) 0x7f8e0c08ba70 con 0x7f8e1c0ce430 2024-04-15T15:21:11.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.501+0000 7f8e21a41640 1 --2- 172.21.15.105:0/3352443832 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f8e0006c980 0x7f8e0006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:11.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.502+0000 7f8e21a41640 1 --2- 172.21.15.105:0/3352443832 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f8e0006c980 0x7f8e0006ee40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8e14036000 tx=0x7f8e1403b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:11.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.506+0000 7f8e127fc640 1 -- 172.21.15.105:0/3352443832 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8e0c059ee0 con 0x7f8e1c0ce430 2024-04-15T15:21:11.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.679+0000 7f8e23ca5640 1 -- 172.21.15.105:0/3352443832 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8e1c06ce90 con 0x7f8e1c0ce430 2024-04-15T15:21:11.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.680+0000 7f8e127fc640 1 -- 172.21.15.105:0/3352443832 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) ==== 93+0+10 (secure 0 0 0) 0x7f8e0c0597e0 con 0x7f8e1c0ce430 2024-04-15T15:21:11.682 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-04-15T15:21:11.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.682+0000 7f8e23ca5640 1 -- 172.21.15.105:0/3352443832 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f8e0006c980 msgr2=0x7f8e0006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:11.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.682+0000 7f8e23ca5640 1 --2- 172.21.15.105:0/3352443832 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f8e0006c980 0x7f8e0006ee40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8e14036000 tx=0x7f8e1403b040 comp rx=0 tx=0).stop 2024-04-15T15:21:11.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.682+0000 7f8e23ca5640 1 -- 172.21.15.105:0/3352443832 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e1c0ce430 msgr2=0x7f8e1c0c6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:11.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.682+0000 7f8e23ca5640 1 --2- 172.21.15.105:0/3352443832 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e1c0ce430 0x7f8e1c0c6b30 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c00c430 tx=0x7f8e0c00c8f0 comp rx=0 tx=0).stop 2024-04-15T15:21:11.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.682+0000 7f8e23ca5640 1 -- 172.21.15.105:0/3352443832 shutdown_connections 2024-04-15T15:21:11.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.682+0000 7f8e23ca5640 1 --2- 172.21.15.105:0/3352443832 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f8e0006c980 0x7f8e0006ee40 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:11.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.682+0000 7f8e23ca5640 1 --2- 172.21.15.105:0/3352443832 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e1c162040 0x7f8e1c15db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:11.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.682+0000 7f8e23ca5640 1 --2- 172.21.15.105:0/3352443832 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e1c0ce430 0x7f8e1c0c6b30 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:11.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.682+0000 7f8e23ca5640 1 --2- 172.21.15.105:0/3352443832 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e1c09aa40 0x7f8e1c152a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:11.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.682+0000 7f8e23ca5640 1 -- 172.21.15.105:0/3352443832 >> 172.21.15.105:0/3352443832 conn(0x7f8e1c0d25f0 msgr2=0x7f8e1c0cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:11.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.682+0000 7f8e23ca5640 1 -- 172.21.15.105:0/3352443832 shutdown_connections 2024-04-15T15:21:11.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.682+0000 7f8e23ca5640 1 -- 172.21.15.105:0/3352443832 wait complete. 2024-04-15T15:21:11.695 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-15T15:21:11.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.800+0000 7f8372fa8640 1 Processor -- start 2024-04-15T15:21:11.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.800+0000 7f8372fa8640 1 -- start start 2024-04-15T15:21:11.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.800+0000 7f8372fa8640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f836c144e80 0x7f836c06dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:11.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.800+0000 7f8372fa8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f836c0b4910 0x7f836c06fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:11.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.800+0000 7f8372fa8640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f836c06c790 con 0x7f836c06f860 2024-04-15T15:21:11.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.800+0000 7f8372fa8640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f836c0b3470 con 0x7f836c144e80 2024-04-15T15:21:11.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.800+0000 7f8372fa8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f836c0cf7b0 con 0x7f836c0b4910 2024-04-15T15:21:11.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.801+0000 7f8371545640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f836c0b4910 0x7f836c06fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:11.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.801+0000 7f8370d44640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f836c144e80 0x7f836c06dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:11.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.801+0000 7f8371545640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f836c0b4910 0x7f836c06fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.105:45844/0 (socket says 172.21.15.105:45844) 2024-04-15T15:21:11.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.801+0000 7f8371545640 1 -- 172.21.15.105:0/1681096363 learned_addr learned my addr 172.21.15.105:0/1681096363 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:11.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.801+0000 7f8370d44640 1 -- 172.21.15.105:0/1681096363 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f836c0b4910 msgr2=0x7f836c06fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:11.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.801+0000 7f8370d44640 1 --2- 172.21.15.105:0/1681096363 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f836c0b4910 0x7f836c06fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:11.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.801+0000 7f8370d44640 1 -- 172.21.15.105:0/1681096363 >> v1:172.21.15.154:6789/0 conn(0x7f836c06f860 legacy=0x7f836c15f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:11.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.801+0000 7f8370d44640 1 -- 172.21.15.105:0/1681096363 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f836c0cfa90 con 0x7f836c144e80 2024-04-15T15:21:11.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.801+0000 7f8371545640 1 --2- 172.21.15.105:0/1681096363 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f836c0b4910 0x7f836c06fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:21:11.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.802+0000 7f8370d44640 1 --2- 172.21.15.105:0/1681096363 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f836c144e80 0x7f836c06dec0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f83600097a0 tx=0x7f836002b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c521a199316e629 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:11.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.802+0000 7f836b7fe640 1 -- 172.21.15.105:0/1681096363 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8360039070 con 0x7f836c144e80 2024-04-15T15:21:11.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.802+0000 7f836b7fe640 1 -- 172.21.15.105:0/1681096363 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83600316f0 con 0x7f836c144e80 2024-04-15T15:21:11.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.802+0000 7f836b7fe640 1 -- 172.21.15.105:0/1681096363 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8360031ac0 con 0x7f836c144e80 2024-04-15T15:21:11.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.803+0000 7f8372fa8640 1 -- 172.21.15.105:0/1681096363 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f836c144e80 msgr2=0x7f836c06dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:11.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.803+0000 7f8372fa8640 1 --2- 172.21.15.105:0/1681096363 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f836c144e80 0x7f836c06dec0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f83600097a0 tx=0x7f836002b790 comp rx=0 tx=0).stop 2024-04-15T15:21:11.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.803+0000 7f8372fa8640 1 -- 172.21.15.105:0/1681096363 shutdown_connections 2024-04-15T15:21:11.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.803+0000 7f8372fa8640 1 --2- 172.21.15.105:0/1681096363 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f836c0b4910 0x7f836c06fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:11.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.803+0000 7f8372fa8640 1 --2- 172.21.15.105:0/1681096363 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f836c144e80 0x7f836c06dec0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:11.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.803+0000 7f8372fa8640 1 -- 172.21.15.105:0/1681096363 >> 172.21.15.105:0/1681096363 conn(0x7f836c06b9c0 msgr2=0x7f836c06bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:11.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.803+0000 7f8372fa8640 1 -- 172.21.15.105:0/1681096363 shutdown_connections 2024-04-15T15:21:11.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.803+0000 7f8372fa8640 1 -- 172.21.15.105:0/1681096363 wait complete. 2024-04-15T15:21:11.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.804+0000 7f8372fa8640 1 Processor -- start 2024-04-15T15:21:11.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.804+0000 7f8372fa8640 1 -- start start 2024-04-15T15:21:11.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.805+0000 7f8372fa8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f836c06f860 0x7f836c136280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:11.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.805+0000 7f8372fa8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f836c0b4910 0x7f836c15af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:11.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.805+0000 7f8372fa8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f836c144e80 0x7f836c138700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:11.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.805+0000 7f8372fa8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f836c06e400 con 0x7f836c0b4910 2024-04-15T15:21:11.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.805+0000 7f8372fa8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f836c0d0050 con 0x7f836c06f860 2024-04-15T15:21:11.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.805+0000 7f8372fa8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f836c145260 con 0x7f836c144e80 2024-04-15T15:21:11.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.805+0000 7f836bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f836c0b4910 0x7f836c15af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:11.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.805+0000 7f836bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f836c0b4910 0x7f836c15af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33154/0 (socket says 172.21.15.105:33154) 2024-04-15T15:21:11.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.805+0000 7f836bfff640 1 -- 172.21.15.105:0/1441892905 learned_addr learned my addr 172.21.15.105:0/1441892905 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:11.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.805+0000 7f8370d44640 1 --2- 172.21.15.105:0/1441892905 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f836c06f860 0x7f836c136280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:11.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.805+0000 7f8371545640 1 --2- 172.21.15.105:0/1441892905 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f836c144e80 0x7f836c138700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:11.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.805+0000 7f836bfff640 1 -- 172.21.15.105:0/1441892905 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f836c144e80 msgr2=0x7f836c138700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:11.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.805+0000 7f836bfff640 1 --2- 172.21.15.105:0/1441892905 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f836c144e80 0x7f836c138700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:11.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.805+0000 7f836bfff640 1 -- 172.21.15.105:0/1441892905 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f836c06f860 msgr2=0x7f836c136280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:11.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.805+0000 7f836bfff640 1 --2- 172.21.15.105:0/1441892905 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f836c06f860 0x7f836c136280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:11.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.805+0000 7f836bfff640 1 -- 172.21.15.105:0/1441892905 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f836c138fc0 con 0x7f836c0b4910 2024-04-15T15:21:11.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.805+0000 7f8371545640 1 --2- 172.21.15.105:0/1441892905 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f836c144e80 0x7f836c138700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:21:11.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.805+0000 7f836bfff640 1 --2- 172.21.15.105:0/1441892905 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f836c0b4910 0x7f836c15af40 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f835800c450 tx=0x7f835800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:11.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.806+0000 7f83697fa640 1 -- 172.21.15.105:0/1441892905 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83580052d0 con 0x7f836c0b4910 2024-04-15T15:21:11.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.806+0000 7f83697fa640 1 -- 172.21.15.105:0/1441892905 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8358005470 con 0x7f836c0b4910 2024-04-15T15:21:11.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.806+0000 7f8372fa8640 1 -- 172.21.15.105:0/1441892905 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f836c133d60 con 0x7f836c0b4910 2024-04-15T15:21:11.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.806+0000 7f83697fa640 1 -- 172.21.15.105:0/1441892905 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8358012870 con 0x7f836c0b4910 2024-04-15T15:21:11.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.806+0000 7f8372fa8640 1 -- 172.21.15.105:0/1441892905 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f836c138c40 con 0x7f836c0b4910 2024-04-15T15:21:11.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.807+0000 7f8372fa8640 1 -- 172.21.15.105:0/1441892905 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f836c068a40 con 0x7f836c0b4910 2024-04-15T15:21:11.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.807+0000 7f83697fa640 1 -- 172.21.15.105:0/1441892905 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f8358012a10 con 0x7f836c0b4910 2024-04-15T15:21:11.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.808+0000 7f83697fa640 1 --2- 172.21.15.105:0/1441892905 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f835006c980 0x7f835006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:11.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.808+0000 7f83697fa640 1 -- 172.21.15.105:0/1441892905 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5795+0+0 (secure 0 0 0) 0x7f835808b810 con 0x7f836c0b4910 2024-04-15T15:21:11.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.808+0000 7f8370d44640 1 --2- 172.21.15.105:0/1441892905 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f835006c980 0x7f835006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:11.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.808+0000 7f8370d44640 1 --2- 172.21.15.105:0/1441892905 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f835006c980 0x7f835006ee40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8360036000 tx=0x7f8360008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:11.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.812+0000 7f83697fa640 1 -- 172.21.15.105:0/1441892905 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8358059bf0 con 0x7f836c0b4910 2024-04-15T15:21:11.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.986+0000 7f8372fa8640 1 -- 172.21.15.105:0/1441892905 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f836c1327a0 con 0x7f836c0b4910 2024-04-15T15:21:11.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.987+0000 7f83697fa640 1 -- 172.21.15.105:0/1441892905 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v35) ==== 104+0+11 (secure 0 0 0) 0x7f83580594f0 con 0x7f836c0b4910 2024-04-15T15:21:11.989 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-04-15T15:21:11.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.989+0000 7f8372fa8640 1 -- 172.21.15.105:0/1441892905 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f835006c980 msgr2=0x7f835006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:11.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.989+0000 7f8372fa8640 1 --2- 172.21.15.105:0/1441892905 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f835006c980 0x7f835006ee40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8360036000 tx=0x7f8360008030 comp rx=0 tx=0).stop 2024-04-15T15:21:11.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.989+0000 7f8372fa8640 1 -- 172.21.15.105:0/1441892905 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f836c0b4910 msgr2=0x7f836c15af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:11.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.989+0000 7f8372fa8640 1 --2- 172.21.15.105:0/1441892905 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f836c0b4910 0x7f836c15af40 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f835800c450 tx=0x7f835800c910 comp rx=0 tx=0).stop 2024-04-15T15:21:11.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.989+0000 7f8372fa8640 1 -- 172.21.15.105:0/1441892905 shutdown_connections 2024-04-15T15:21:11.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.989+0000 7f8372fa8640 1 --2- 172.21.15.105:0/1441892905 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f836c144e80 0x7f836c138700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:11.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.989+0000 7f8372fa8640 1 --2- 172.21.15.105:0/1441892905 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f835006c980 0x7f835006ee40 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:11.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.989+0000 7f8372fa8640 1 --2- 172.21.15.105:0/1441892905 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f836c0b4910 0x7f836c15af40 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:11.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.989+0000 7f8372fa8640 1 --2- 172.21.15.105:0/1441892905 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f836c06f860 0x7f836c136280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:11.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.989+0000 7f8372fa8640 1 -- 172.21.15.105:0/1441892905 >> 172.21.15.105:0/1441892905 conn(0x7f836c06b9c0 msgr2=0x7f836c06eda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:11.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.989+0000 7f8372fa8640 1 -- 172.21.15.105:0/1441892905 shutdown_connections 2024-04-15T15:21:11.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:11.989+0000 7f8372fa8640 1 -- 172.21.15.105:0/1441892905 wait complete. 2024-04-15T15:21:12.002 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-15T15:21:12.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.107+0000 7f0401047640 1 Processor -- start 2024-04-15T15:21:12.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.107+0000 7f0401047640 1 -- start start 2024-04-15T15:21:12.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.108+0000 7f0401047640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f03fc0d1070 0x7f03fc0cd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:12.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.108+0000 7f0401047640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03fc0d0640 con 0x7f03fc0cec00 2024-04-15T15:21:12.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.108+0000 7f0401047640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03fc0d0360 con 0x7f03fc15ff80 2024-04-15T15:21:12.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.108+0000 7f0401047640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f03fc160360 con 0x7f03fc0d1070 2024-04-15T15:21:12.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.108+0000 7f03fb59e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f03fc0d1070 0x7f03fc0cd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:12.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.108+0000 7f03fb59e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f03fc0d1070 0x7f03fc0cd6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33158/0 (socket says 172.21.15.105:33158) 2024-04-15T15:21:12.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.108+0000 7f03fb59e640 1 -- 172.21.15.105:0/160289373 learned_addr learned my addr 172.21.15.105:0/160289373 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:12.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.108+0000 7f03fb59e640 1 -- 172.21.15.105:0/160289373 >> v1:172.21.15.154:6789/0 conn(0x7f03fc15ff80 legacy=0x7f03fc0d0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:12.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.108+0000 7f03fb59e640 1 -- 172.21.15.105:0/160289373 >> v1:172.21.15.105:6789/0 conn(0x7f03fc0cec00 legacy=0x7f03fc0ccfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:12.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.108+0000 7f03fb59e640 1 -- 172.21.15.105:0/160289373 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03fc0c61e0 con 0x7f03fc0d1070 2024-04-15T15:21:12.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.109+0000 7f03fb59e640 1 --2- 172.21.15.105:0/160289373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03fc0d1070 0x7f03fc0cd6e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f03f4009e50 tx=0x7f03f402c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=580d72f1d5b670df server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:12.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.109+0000 7f03fa59c640 1 -- 172.21.15.105:0/160289373 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f402cbb0 con 0x7f03fc0d1070 2024-04-15T15:21:12.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.109+0000 7f03fa59c640 1 -- 172.21.15.105:0/160289373 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03f402cd50 con 0x7f03fc0d1070 2024-04-15T15:21:12.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.109+0000 7f03fa59c640 1 -- 172.21.15.105:0/160289373 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f4034610 con 0x7f03fc0d1070 2024-04-15T15:21:12.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.110+0000 7f0401047640 1 -- 172.21.15.105:0/160289373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03fc0d1070 msgr2=0x7f03fc0cd6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:12.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.110+0000 7f0401047640 1 --2- 172.21.15.105:0/160289373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03fc0d1070 0x7f03fc0cd6e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f03f4009e50 tx=0x7f03f402c010 comp rx=0 tx=0).stop 2024-04-15T15:21:12.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.110+0000 7f0401047640 1 -- 172.21.15.105:0/160289373 shutdown_connections 2024-04-15T15:21:12.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.110+0000 7f0401047640 1 --2- 172.21.15.105:0/160289373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03fc0d1070 0x7f03fc0cd6e0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:12.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.110+0000 7f0401047640 1 -- 172.21.15.105:0/160289373 >> 172.21.15.105:0/160289373 conn(0x7f03fc0d25f0 msgr2=0x7f03fc0d2a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:12.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.110+0000 7f0401047640 1 -- 172.21.15.105:0/160289373 shutdown_connections 2024-04-15T15:21:12.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.110+0000 7f0401047640 1 -- 172.21.15.105:0/160289373 wait complete. 2024-04-15T15:21:12.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.111+0000 7f0401047640 1 Processor -- start 2024-04-15T15:21:12.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.111+0000 7f0401047640 1 -- start start 2024-04-15T15:21:12.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.112+0000 7f0401047640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f03fc0cec00 0x7f03fc09e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:12.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.112+0000 7f0401047640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03fc0d1070 0x7f03fc09fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:12.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.112+0000 7f0401047640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03fc15ff80 0x7f03fc1f2c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:12.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.112+0000 7f0401047640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc0c79f0 con 0x7f03fc0d1070 2024-04-15T15:21:12.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.112+0000 7f0401047640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc0c6550 con 0x7f03fc15ff80 2024-04-15T15:21:12.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.112+0000 7f0401047640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f03fc0c7b70 con 0x7f03fc0cec00 2024-04-15T15:21:12.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.112+0000 7f03f3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03fc0d1070 0x7f03fc09fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:12.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.112+0000 7f03f3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03fc0d1070 0x7f03fc09fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33174/0 (socket says 172.21.15.105:33174) 2024-04-15T15:21:12.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.112+0000 7f03f3fff640 1 -- 172.21.15.105:0/1947706491 learned_addr learned my addr 172.21.15.105:0/1947706491 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:12.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.112+0000 7f03fad9d640 1 --2- 172.21.15.105:0/1947706491 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f03fc0cec00 0x7f03fc09e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:12.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.112+0000 7f03f3fff640 1 -- 172.21.15.105:0/1947706491 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f03fc0cec00 msgr2=0x7f03fc09e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:12.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.112+0000 7f03f3fff640 1 --2- 172.21.15.105:0/1947706491 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f03fc0cec00 0x7f03fc09e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:12.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.112+0000 7f03fb59e640 1 --2- 172.21.15.105:0/1947706491 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03fc15ff80 0x7f03fc1f2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:12.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.112+0000 7f03f3fff640 1 -- 172.21.15.105:0/1947706491 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03fc15ff80 msgr2=0x7f03fc1f2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:12.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.112+0000 7f03f3fff640 1 --2- 172.21.15.105:0/1947706491 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03fc15ff80 0x7f03fc1f2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:12.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.112+0000 7f03f3fff640 1 -- 172.21.15.105:0/1947706491 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03fc09eb90 con 0x7f03fc0d1070 2024-04-15T15:21:12.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.112+0000 7f03fad9d640 1 --2- 172.21.15.105:0/1947706491 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f03fc0cec00 0x7f03fc09e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:21:12.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.112+0000 7f03f3fff640 1 --2- 172.21.15.105:0/1947706491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03fc0d1070 0x7f03fc09fc10 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f03e400c3c0 tx=0x7f03e400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:12.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.112+0000 7f03f37fe640 1 -- 172.21.15.105:0/1947706491 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e4017070 con 0x7f03fc0d1070 2024-04-15T15:21:12.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.112+0000 7f03f37fe640 1 -- 172.21.15.105:0/1947706491 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03e40052d0 con 0x7f03fc0d1070 2024-04-15T15:21:12.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.112+0000 7f0401047640 1 -- 172.21.15.105:0/1947706491 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03fc09ee50 con 0x7f03fc0d1070 2024-04-15T15:21:12.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.113+0000 7f03f37fe640 1 -- 172.21.15.105:0/1947706491 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e4003850 con 0x7f03fc0d1070 2024-04-15T15:21:12.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.113+0000 7f0401047640 1 -- 172.21.15.105:0/1947706491 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03fc09f410 con 0x7f03fc0d1070 2024-04-15T15:21:12.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.114+0000 7f0401047640 1 -- 172.21.15.105:0/1947706491 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03fc068ab0 con 0x7f03fc0d1070 2024-04-15T15:21:12.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.114+0000 7f03f37fe640 1 -- 172.21.15.105:0/1947706491 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f03e401a070 con 0x7f03fc0d1070 2024-04-15T15:21:12.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.114+0000 7f03f37fe640 1 --2- 172.21.15.105:0/1947706491 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f03dc06c980 0x7f03dc06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:12.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.115+0000 7f03f37fe640 1 -- 172.21.15.105:0/1947706491 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5795+0+0 (secure 0 0 0) 0x7f03e408af10 con 0x7f03fc0d1070 2024-04-15T15:21:12.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.115+0000 7f03fad9d640 1 --2- 172.21.15.105:0/1947706491 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f03dc06c980 0x7f03dc06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:12.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.115+0000 7f03fad9d640 1 --2- 172.21.15.105:0/1947706491 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f03dc06c980 0x7f03dc06ee40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f03fc0c5da0 tx=0x7f03ec009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:12.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.119+0000 7f03f37fe640 1 -- 172.21.15.105:0/1947706491 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f03e40593b0 con 0x7f03fc0d1070 2024-04-15T15:21:12.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.292+0000 7f0401047640 1 -- 172.21.15.105:0/1947706491 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f03fc09b450 con 0x7f03fc0d1070 2024-04-15T15:21:12.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.292+0000 7f03f37fe640 1 -- 172.21.15.105:0/1947706491 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v35) ==== 100+0+11 (secure 0 0 0) 0x7f03e4058cb0 con 0x7f03fc0d1070 2024-04-15T15:21:12.294 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-04-15T15:21:12.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.294+0000 7f0401047640 1 -- 172.21.15.105:0/1947706491 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f03dc06c980 msgr2=0x7f03dc06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:12.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.294+0000 7f0401047640 1 --2- 172.21.15.105:0/1947706491 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f03dc06c980 0x7f03dc06ee40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f03fc0c5da0 tx=0x7f03ec009040 comp rx=0 tx=0).stop 2024-04-15T15:21:12.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.294+0000 7f0401047640 1 -- 172.21.15.105:0/1947706491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03fc0d1070 msgr2=0x7f03fc09fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:12.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.294+0000 7f0401047640 1 --2- 172.21.15.105:0/1947706491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03fc0d1070 0x7f03fc09fc10 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f03e400c3c0 tx=0x7f03e400c880 comp rx=0 tx=0).stop 2024-04-15T15:21:12.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.294+0000 7f0401047640 1 -- 172.21.15.105:0/1947706491 shutdown_connections 2024-04-15T15:21:12.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.294+0000 7f0401047640 1 --2- 172.21.15.105:0/1947706491 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f03dc06c980 0x7f03dc06ee40 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:12.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.294+0000 7f0401047640 1 --2- 172.21.15.105:0/1947706491 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03fc15ff80 0x7f03fc1f2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:12.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.294+0000 7f0401047640 1 --2- 172.21.15.105:0/1947706491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03fc0d1070 0x7f03fc09fc10 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:12.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.294+0000 7f0401047640 1 --2- 172.21.15.105:0/1947706491 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f03fc0cec00 0x7f03fc09e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:12.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.294+0000 7f0401047640 1 -- 172.21.15.105:0/1947706491 >> 172.21.15.105:0/1947706491 conn(0x7f03fc0d25f0 msgr2=0x7f03fc0cd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:12.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.294+0000 7f0401047640 1 -- 172.21.15.105:0/1947706491 shutdown_connections 2024-04-15T15:21:12.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.295+0000 7f0401047640 1 -- 172.21.15.105:0/1947706491 wait complete. 2024-04-15T15:21:12.307 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-15T15:21:12.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.410+0000 7f67f5b32640 1 Processor -- start 2024-04-15T15:21:12.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.410+0000 7f67f5b32640 1 -- start start 2024-04-15T15:21:12.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.410+0000 7f67f5b32640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f67f00b03f0 0x7f67f00cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:12.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.411+0000 7f67f5b32640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f67f0072d00 0x7f67f015ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:12.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.411+0000 7f67f5b32640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67f00b07d0 con 0x7f67f006c590 2024-04-15T15:21:12.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.411+0000 7f67f5b32640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f67f00b54a0 con 0x7f67f00b03f0 2024-04-15T15:21:12.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.411+0000 7f67f5b32640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f67f00d0700 con 0x7f67f0072d00 2024-04-15T15:21:12.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.411+0000 7f67eeffd640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f67f006c590 0x7f67f00725f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:37534/0 (socket says 172.21.15.105:37534) 2024-04-15T15:21:12.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.411+0000 7f67eeffd640 1 -- 172.21.15.105:0/2616078392 learned_addr learned my addr 172.21.15.105:0/2616078392 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:21:12.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.411+0000 7f67effff640 1 --2- 172.21.15.105:0/2616078392 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f67f0072d00 0x7f67f015ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:12.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.411+0000 7f67ef7fe640 1 --2- 172.21.15.105:0/2616078392 >> v2:172.21.15.154:3300/0 conn(0x7f67f00b03f0 0x7f67f00cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:12.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.411+0000 7f67ee7fc640 1 -- 172.21.15.105:0/2616078392 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1309613877 0 0) 0x7f67f00b07d0 con 0x7f67f006c590 2024-04-15T15:21:12.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.411+0000 7f67ee7fc640 1 -- 172.21.15.105:0/2616078392 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f67d8003520 con 0x7f67f006c590 2024-04-15T15:21:12.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.411+0000 7f67ee7fc640 1 -- 172.21.15.105:0/2616078392 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2801614353 0 0) 0x7f67d8003520 con 0x7f67f006c590 2024-04-15T15:21:12.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.411+0000 7f67ee7fc640 1 -- 172.21.15.105:0/2616078392 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f67f0072d00 msgr2=0x7f67f015ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:12.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.411+0000 7f67ee7fc640 1 --2- 172.21.15.105:0/2616078392 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f67f0072d00 0x7f67f015ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:12.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.411+0000 7f67ee7fc640 1 -- 172.21.15.105:0/2616078392 >> v2:172.21.15.154:3300/0 conn(0x7f67f00b03f0 msgr2=0x7f67f00cfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:12.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.411+0000 7f67ee7fc640 1 --2- 172.21.15.105:0/2616078392 >> v2:172.21.15.154:3300/0 conn(0x7f67f00b03f0 0x7f67f00cfe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:12.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.411+0000 7f67ee7fc640 1 -- 172.21.15.105:0/2616078392 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67f00b5740 con 0x7f67f006c590 2024-04-15T15:21:12.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.411+0000 7f67ee7fc640 1 -- 172.21.15.105:0/2616078392 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7f67e4001670 con 0x7f67f006c590 2024-04-15T15:21:12.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.411+0000 7f67ee7fc640 1 -- 172.21.15.105:0/2616078392 >> v1:172.21.15.105:6789/0 conn(0x7f67f006c590 legacy=0x7f67f00725f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:12.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.411+0000 7f67ee7fc640 1 --2- 172.21.15.105:0/2616078392 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67d80039b0 0x7f67d8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:12.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.411+0000 7f67ee7fc640 1 --2- 172.21.15.105:0/2616078392 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67d8004800 0x7f67d8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:12.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.411+0000 7f67ee7fc640 1 --2- 172.21.15.105:0/2616078392 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f67f0072d00 0x7f67d80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:12.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.411+0000 7f67ee7fc640 1 -- 172.21.15.105:0/2616078392 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f67f00b54a0 con 0x7f67d80039b0 2024-04-15T15:21:12.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.411+0000 7f67ee7fc640 1 -- 172.21.15.105:0/2616078392 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f67f00d0700 con 0x7f67d8004800 2024-04-15T15:21:12.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.411+0000 7f67ee7fc640 1 -- 172.21.15.105:0/2616078392 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f67d800b9f0 con 0x7f67f0072d00 2024-04-15T15:21:12.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.411+0000 7f67ef7fe640 1 --2- 172.21.15.105:0/2616078392 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67d80039b0 0x7f67d8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:12.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.411+0000 7f67eeffd640 1 --2- 172.21.15.105:0/2616078392 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67d8004800 0x7f67d8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:12.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.411+0000 7f67effff640 1 --2- 172.21.15.105:0/2616078392 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f67f0072d00 0x7f67d80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:12.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.412+0000 7f67ef7fe640 1 -- 172.21.15.105:0/2616078392 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f67f0072d00 msgr2=0x7f67d80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:12.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.412+0000 7f67ef7fe640 1 --2- 172.21.15.105:0/2616078392 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f67f0072d00 0x7f67d80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:12.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.412+0000 7f67ef7fe640 1 -- 172.21.15.105:0/2616078392 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67d8004800 msgr2=0x7f67d8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:12.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.412+0000 7f67ef7fe640 1 --2- 172.21.15.105:0/2616078392 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67d8004800 0x7f67d8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:12.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.412+0000 7f67ef7fe640 1 -- 172.21.15.105:0/2616078392 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67d8003520 con 0x7f67d80039b0 2024-04-15T15:21:12.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.412+0000 7f67ef7fe640 1 --2- 172.21.15.105:0/2616078392 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67d80039b0 0x7f67d8003dd0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f67e0009ac0 tx=0x7f67d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa4d692af7353a50 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:12.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.412+0000 7f67ee7fc640 1 -- 172.21.15.105:0/2616078392 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e0039070 con 0x7f67d80039b0 2024-04-15T15:21:12.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.412+0000 7f67ee7fc640 1 -- 172.21.15.105:0/2616078392 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67e002c6d0 con 0x7f67d80039b0 2024-04-15T15:21:12.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.412+0000 7f67ee7fc640 1 -- 172.21.15.105:0/2616078392 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e0009690 con 0x7f67d80039b0 2024-04-15T15:21:12.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.413+0000 7f67f5b32640 1 -- 172.21.15.105:0/2616078392 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67d80039b0 msgr2=0x7f67d8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:12.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.413+0000 7f67f5b32640 1 --2- 172.21.15.105:0/2616078392 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67d80039b0 0x7f67d8003dd0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f67e0009ac0 tx=0x7f67d8005830 comp rx=0 tx=0).stop 2024-04-15T15:21:12.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.413+0000 7f67effff640 1 -- 172.21.15.105:0/2616078392 reap_dead start 2024-04-15T15:21:12.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.413+0000 7f67f5b32640 1 -- 172.21.15.105:0/2616078392 shutdown_connections 2024-04-15T15:21:12.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.413+0000 7f67f5b32640 1 -- 172.21.15.105:0/2616078392 >> 172.21.15.105:0/2616078392 conn(0x7f67f00d0d00 msgr2=0x7f67f00d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:12.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.413+0000 7f67f5b32640 1 -- 172.21.15.105:0/2616078392 shutdown_connections 2024-04-15T15:21:12.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.413+0000 7f67f5b32640 1 -- 172.21.15.105:0/2616078392 wait complete. 2024-04-15T15:21:12.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.414+0000 7f67f5b32640 1 Processor -- start 2024-04-15T15:21:12.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.414+0000 7f67f5b32640 1 -- start start 2024-04-15T15:21:12.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.415+0000 7f67f5b32640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67d80039b0 0x7f67f01f5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:12.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.415+0000 7f67f5b32640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f67f01e6360 0x7f67f01e67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:12.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.415+0000 7f67f5b32640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67f01e4100 0x7f67f01e4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:12.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.415+0000 7f67f5b32640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f67f00d0340 con 0x7f67f01e4100 2024-04-15T15:21:12.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.415+0000 7f67f5b32640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f67f00d09e0 con 0x7f67d80039b0 2024-04-15T15:21:12.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.415+0000 7f67f5b32640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f67f00d04c0 con 0x7f67f01e6360 2024-04-15T15:21:12.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.415+0000 7f67effff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67f01e4100 0x7f67f01e4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:12.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.415+0000 7f67effff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67f01e4100 0x7f67f01e4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33188/0 (socket says 172.21.15.105:33188) 2024-04-15T15:21:12.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.415+0000 7f67effff640 1 -- 172.21.15.105:0/2019804155 learned_addr learned my addr 172.21.15.105:0/2019804155 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:12.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.415+0000 7f67ef7fe640 1 --2- 172.21.15.105:0/2019804155 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67d80039b0 0x7f67f01f5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:12.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.415+0000 7f67eeffd640 1 --2- 172.21.15.105:0/2019804155 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f67f01e6360 0x7f67f01e67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:12.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.415+0000 7f67effff640 1 -- 172.21.15.105:0/2019804155 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f67f01e6360 msgr2=0x7f67f01e67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:12.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.415+0000 7f67effff640 1 --2- 172.21.15.105:0/2019804155 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f67f01e6360 0x7f67f01e67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:12.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.415+0000 7f67effff640 1 -- 172.21.15.105:0/2019804155 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67d80039b0 msgr2=0x7f67f01f5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:12.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.415+0000 7f67effff640 1 --2- 172.21.15.105:0/2019804155 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67d80039b0 0x7f67f01f5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:12.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.415+0000 7f67effff640 1 -- 172.21.15.105:0/2019804155 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67f01e4e70 con 0x7f67f01e4100 2024-04-15T15:21:12.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.416+0000 7f67eeffd640 1 --2- 172.21.15.105:0/2019804155 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f67f01e6360 0x7f67f01e67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:21:12.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.416+0000 7f67effff640 1 --2- 172.21.15.105:0/2019804155 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67f01e4100 0x7f67f01e4580 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f67f00cea90 tx=0x7f67e80115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:12.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.416+0000 7f67d7fff640 1 -- 172.21.15.105:0/2019804155 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e8011e10 con 0x7f67f01e4100 2024-04-15T15:21:12.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.416+0000 7f67d7fff640 1 -- 172.21.15.105:0/2019804155 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67e801b070 con 0x7f67f01e4100 2024-04-15T15:21:12.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.416+0000 7f67f5b32640 1 -- 172.21.15.105:0/2019804155 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67f01e5150 con 0x7f67f01e4100 2024-04-15T15:21:12.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.416+0000 7f67d7fff640 1 -- 172.21.15.105:0/2019804155 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e8003990 con 0x7f67f01e4100 2024-04-15T15:21:12.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.416+0000 7f67f5b32640 1 -- 172.21.15.105:0/2019804155 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67f01e5710 con 0x7f67f01e4100 2024-04-15T15:21:12.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.417+0000 7f67f5b32640 1 -- 172.21.15.105:0/2019804155 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67f0068a40 con 0x7f67f01e4100 2024-04-15T15:21:12.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.417+0000 7f67d7fff640 1 -- 172.21.15.105:0/2019804155 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f67e8003b50 con 0x7f67f01e4100 2024-04-15T15:21:12.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.418+0000 7f67d7fff640 1 --2- 172.21.15.105:0/2019804155 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f67dc06c980 0x7f67dc06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:12.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.418+0000 7f67d7fff640 1 -- 172.21.15.105:0/2019804155 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5795+0+0 (secure 0 0 0) 0x7f67e808c4d0 con 0x7f67f01e4100 2024-04-15T15:21:12.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.418+0000 7f67ef7fe640 1 --2- 172.21.15.105:0/2019804155 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f67dc06c980 0x7f67dc06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:12.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.418+0000 7f67ef7fe640 1 --2- 172.21.15.105:0/2019804155 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f67dc06c980 0x7f67dc06ee40 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f67e0036000 tx=0x7f67e0030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:12.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.422+0000 7f67d7fff640 1 -- 172.21.15.105:0/2019804155 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f67e805a830 con 0x7f67f01e4100 2024-04-15T15:21:12.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.612+0000 7f67f5b32640 1 -- 172.21.15.105:0/2019804155 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f67f0067820 con 0x7f67f01e4100 2024-04-15T15:21:12.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.612+0000 7f67d7fff640 1 -- 172.21.15.105:0/2019804155 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v35) ==== 80+0+122 (secure 0 0 0) 0x7f67e805a130 con 0x7f67f01e4100 2024-04-15T15:21:12.614 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.105:6837/958066041 2024-04-16T15:21:03.660841+0000 2024-04-15T15:21:12.615 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.105:6835/958066041 2024-04-16T15:21:03.660841+0000 2024-04-15T15:21:12.615 INFO:teuthology.orchestra.run.smithi105.stderr:listed 2 entries 2024-04-15T15:21:12.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.614+0000 7f67f5b32640 1 -- 172.21.15.105:0/2019804155 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f67dc06c980 msgr2=0x7f67dc06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:12.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.614+0000 7f67f5b32640 1 --2- 172.21.15.105:0/2019804155 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f67dc06c980 0x7f67dc06ee40 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f67e0036000 tx=0x7f67e0030000 comp rx=0 tx=0).stop 2024-04-15T15:21:12.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.614+0000 7f67f5b32640 1 -- 172.21.15.105:0/2019804155 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67f01e4100 msgr2=0x7f67f01e4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:12.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.614+0000 7f67f5b32640 1 --2- 172.21.15.105:0/2019804155 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67f01e4100 0x7f67f01e4580 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f67f00cea90 tx=0x7f67e80115a0 comp rx=0 tx=0).stop 2024-04-15T15:21:12.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.614+0000 7f67f5b32640 1 -- 172.21.15.105:0/2019804155 shutdown_connections 2024-04-15T15:21:12.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.614+0000 7f67f5b32640 1 --2- 172.21.15.105:0/2019804155 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f67f01e6360 0x7f67f01e67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:12.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.614+0000 7f67f5b32640 1 --2- 172.21.15.105:0/2019804155 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f67dc06c980 0x7f67dc06ee40 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:12.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.614+0000 7f67f5b32640 1 --2- 172.21.15.105:0/2019804155 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67f01e4100 0x7f67f01e4580 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:12.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.614+0000 7f67f5b32640 1 --2- 172.21.15.105:0/2019804155 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67d80039b0 0x7f67f01f5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:12.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.614+0000 7f67f5b32640 1 -- 172.21.15.105:0/2019804155 >> 172.21.15.105:0/2019804155 conn(0x7f67f00d0d00 msgr2=0x7f67f006b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:12.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.614+0000 7f67f5b32640 1 -- 172.21.15.105:0/2019804155 shutdown_connections 2024-04-15T15:21:12.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.614+0000 7f67f5b32640 1 -- 172.21.15.105:0/2019804155 wait complete. 2024-04-15T15:21:12.627 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi154.front.sepia.ceph.com... 2024-04-15T15:21:12.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:21:12.628 DEBUG:teuthology.orchestra.run.smithi154:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-15T15:21:12.660 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi154.stderr:ceph-fuse[45615]: fuse finished with error 0 and tester_r 0 2024-04-15T15:21:12.662 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi154.front.sepia.ceph.com 2024-04-15T15:21:12.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:21:12.663 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-15T15:21:12.834 INFO:tasks.cephfs.mount:Terminating background process 2024-04-15T15:21:12.834 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-04-15T15:21:12.834 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:21:12.851 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:21:12.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.992+0000 7f09a32c5640 1 Processor -- start 2024-04-15T15:21:12.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.993+0000 7f09a32c5640 1 -- start start 2024-04-15T15:21:12.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.993+0000 7f09a32c5640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f099c15eb60 0x7f099c0d11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:12.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.993+0000 7f09a32c5640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f099c0d1710 0x7f099c162630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:12.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.993+0000 7f09a32c5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f099c162b70 0x7f099c0b6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:12.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.993+0000 7f09a32c5640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f099c0d1b60 con 0x7f099c15eb60 2024-04-15T15:21:12.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.993+0000 7f09a32c5640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f099c163540 con 0x7f099c0d1710 2024-04-15T15:21:12.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.993+0000 7f09a32c5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f099c0d2fc0 con 0x7f099c162b70 2024-04-15T15:21:12.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.993+0000 7f09a1061640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f099c15eb60 0x7f099c0d11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:12.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.994+0000 7f09a1061640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f099c15eb60 0x7f099c0d11d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33204/0 (socket says 172.21.15.105:33204) 2024-04-15T15:21:12.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.994+0000 7f09a1061640 1 -- 172.21.15.105:0/3298477396 learned_addr learned my addr 172.21.15.105:0/3298477396 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:12.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.994+0000 7f09a1862640 1 --2- 172.21.15.105:0/3298477396 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f099c162b70 0x7f099c0b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:12.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.994+0000 7f09a0860640 1 --2- 172.21.15.105:0/3298477396 >> v2:172.21.15.154:3300/0 conn(0x7f099c0d1710 0x7f099c162630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:12.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.994+0000 7f09a1061640 1 -- 172.21.15.105:0/3298477396 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f099c162b70 msgr2=0x7f099c0b6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:12.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.994+0000 7f09a1061640 1 --2- 172.21.15.105:0/3298477396 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f099c162b70 0x7f099c0b6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:12.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.994+0000 7f09a1061640 1 -- 172.21.15.105:0/3298477396 >> v2:172.21.15.154:3300/0 conn(0x7f099c0d1710 msgr2=0x7f099c162630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:12.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.994+0000 7f09a1061640 1 --2- 172.21.15.105:0/3298477396 >> v2:172.21.15.154:3300/0 conn(0x7f099c0d1710 0x7f099c162630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:12.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.994+0000 7f09a1061640 1 -- 172.21.15.105:0/3298477396 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f099c15ef40 con 0x7f099c15eb60 2024-04-15T15:21:12.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.994+0000 7f09a1862640 1 --2- 172.21.15.105:0/3298477396 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f099c162b70 0x7f099c0b6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:21:12.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.994+0000 7f09a0860640 1 --2- 172.21.15.105:0/3298477396 >> v2:172.21.15.154:3300/0 conn(0x7f099c0d1710 0x7f099c162630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:21:12.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.995+0000 7f09a1061640 1 --2- 172.21.15.105:0/3298477396 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f099c15eb60 0x7f099c0d11d0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f099c0b7680 tx=0x7f099802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d038e4d14dae572 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:12.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.995+0000 7f0993fff640 1 -- 172.21.15.105:0/3298477396 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099802ca70 con 0x7f099c15eb60 2024-04-15T15:21:12.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.995+0000 7f0993fff640 1 -- 172.21.15.105:0/3298477396 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f099802cc10 con 0x7f099c15eb60 2024-04-15T15:21:12.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.995+0000 7f0993fff640 1 -- 172.21.15.105:0/3298477396 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09980343f0 con 0x7f099c15eb60 2024-04-15T15:21:13.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.995+0000 7f09a32c5640 1 -- 172.21.15.105:0/3298477396 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f099c15eb60 msgr2=0x7f099c0d11d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:13.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.995+0000 7f09a32c5640 1 --2- 172.21.15.105:0/3298477396 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f099c15eb60 0x7f099c0d11d0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f099c0b7680 tx=0x7f099802bed0 comp rx=0 tx=0).stop 2024-04-15T15:21:13.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.995+0000 7f09a32c5640 1 -- 172.21.15.105:0/3298477396 shutdown_connections 2024-04-15T15:21:13.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.995+0000 7f09a32c5640 1 --2- 172.21.15.105:0/3298477396 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f099c162b70 0x7f099c0b6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:13.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.995+0000 7f09a32c5640 1 --2- 172.21.15.105:0/3298477396 >> v2:172.21.15.154:3300/0 conn(0x7f099c0d1710 0x7f099c162630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:13.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.995+0000 7f09a32c5640 1 --2- 172.21.15.105:0/3298477396 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f099c15eb60 0x7f099c0d11d0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:13.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.995+0000 7f09a32c5640 1 -- 172.21.15.105:0/3298477396 >> 172.21.15.105:0/3298477396 conn(0x7f099c067720 msgr2=0x7f099c067b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:13.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.995+0000 7f09a32c5640 1 -- 172.21.15.105:0/3298477396 shutdown_connections 2024-04-15T15:21:13.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.995+0000 7f09a32c5640 1 -- 172.21.15.105:0/3298477396 wait complete. 2024-04-15T15:21:13.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.997+0000 7f09a32c5640 1 Processor -- start 2024-04-15T15:21:13.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.997+0000 7f09a32c5640 1 -- start start 2024-04-15T15:21:13.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.997+0000 7f09a32c5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f099c0d1710 0x7f099c0b5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:13.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.997+0000 7f09a32c5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f099c15eb60 0x7f099c1f3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:13.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.997+0000 7f09a32c5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f099c162b70 0x7f099c0c9d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:13.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.997+0000 7f09a32c5640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f099c0d2970 con 0x7f099c0d1710 2024-04-15T15:21:13.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.997+0000 7f09a32c5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f099c163260 con 0x7f099c162b70 2024-04-15T15:21:13.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.997+0000 7f09a32c5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f099c0d2c50 con 0x7f099c15eb60 2024-04-15T15:21:13.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.997+0000 7f09a1061640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f099c0d1710 0x7f099c0b5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:13.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.997+0000 7f09a1061640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f099c0d1710 0x7f099c0b5df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33210/0 (socket says 172.21.15.105:33210) 2024-04-15T15:21:13.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.997+0000 7f09a1061640 1 -- 172.21.15.105:0/1134629327 learned_addr learned my addr 172.21.15.105:0/1134629327 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:13.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.998+0000 7f09a1862640 1 --2- 172.21.15.105:0/1134629327 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f099c162b70 0x7f099c0c9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:13.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.998+0000 7f09a0860640 1 --2- 172.21.15.105:0/1134629327 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f099c15eb60 0x7f099c1f3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:13.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.998+0000 7f09a1061640 1 -- 172.21.15.105:0/1134629327 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f099c15eb60 msgr2=0x7f099c1f3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:13.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.998+0000 7f09a1061640 1 --2- 172.21.15.105:0/1134629327 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f099c15eb60 0x7f099c1f3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:13.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.998+0000 7f09a1061640 1 -- 172.21.15.105:0/1134629327 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f099c162b70 msgr2=0x7f099c0c9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:13.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.998+0000 7f09a1061640 1 --2- 172.21.15.105:0/1134629327 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f099c162b70 0x7f099c0c9d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:13.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.998+0000 7f09a1061640 1 -- 172.21.15.105:0/1134629327 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f099c0b71c0 con 0x7f099c0d1710 2024-04-15T15:21:13.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.998+0000 7f09a1061640 1 --2- 172.21.15.105:0/1134629327 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f099c0d1710 0x7f099c0b5df0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f0998000f80 tx=0x7f0998004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:13.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.998+0000 7f0991ffb640 1 -- 172.21.15.105:0/1134629327 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0998043020 con 0x7f099c0d1710 2024-04-15T15:21:13.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.998+0000 7f0991ffb640 1 -- 172.21.15.105:0/1134629327 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f099803e450 con 0x7f099c0d1710 2024-04-15T15:21:13.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.998+0000 7f09a32c5640 1 -- 172.21.15.105:0/1134629327 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f099c0b6640 con 0x7f099c0d1710 2024-04-15T15:21:13.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.998+0000 7f0991ffb640 1 -- 172.21.15.105:0/1134629327 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099803e840 con 0x7f099c0d1710 2024-04-15T15:21:13.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.998+0000 7f09a32c5640 1 -- 172.21.15.105:0/1134629327 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f099c0b6c00 con 0x7f099c0d1710 2024-04-15T15:21:13.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:12.999+0000 7f09a32c5640 1 -- 172.21.15.105:0/1134629327 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f099c068a40 con 0x7f099c0d1710 2024-04-15T15:21:13.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.000+0000 7f0991ffb640 1 -- 172.21.15.105:0/1134629327 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f0998046020 con 0x7f099c0d1710 2024-04-15T15:21:13.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.000+0000 7f0991ffb640 1 --2- 172.21.15.105:0/1134629327 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f098006c980 0x7f098006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:13.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.000+0000 7f0991ffb640 1 -- 172.21.15.105:0/1134629327 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5795+0+0 (secure 0 0 0) 0x7f09980ae4f0 con 0x7f099c0d1710 2024-04-15T15:21:13.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.000+0000 7f09a0860640 1 --2- 172.21.15.105:0/1134629327 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f098006c980 0x7f098006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:13.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.001+0000 7f09a0860640 1 --2- 172.21.15.105:0/1134629327 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f098006c980 0x7f098006ee40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f099c14d6b0 tx=0x7f098c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:13.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.004+0000 7f0991ffb640 1 -- 172.21.15.105:0/1134629327 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f099804d030 con 0x7f099c0d1710 2024-04-15T15:21:13.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.182+0000 7f09a32c5640 1 -- 172.21.15.105:0/1134629327 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f099c0d23f0 con 0x7f099c0d1710 2024-04-15T15:21:13.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.183+0000 7f0991ffb640 1 -- 172.21.15.105:0/1134629327 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16846 (secure 0 0 0) 0x7f099807c960 con 0x7f099c0d1710 2024-04-15T15:21:13.185 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:21:13.185 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":35,"fsid":"09e4d2a4-a324-4793-af77-644b25e520e1","created":"2024-04-15T15:19:01.524543+0000","modified":"2024-04-15T15:21:03.660870+0000","last_up_change":"2024-04-15T15:19:12.825815+0000","last_in_change":"2024-04-15T15:19:03.788126+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:19:13.748585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:20:26.983353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-15T15:20:27.961802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6f6292b-2d7e-4aa0-9d1c-6ddd0f97287c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6801","nonce":2873040054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6803","nonce":2873040054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6807","nonce":2873040054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6805","nonce":2873040054}]},"public_addr":"172.21.15.105:6801/2873040054","cluster_addr":"172.21.15.105:6803/2873040054","heartbeat_back_addr":"172.21.15.105:6807/2873040054","heartbeat_front_addr":"172.21.15.105:6805/2873040054","state":["exists","up"]},{"osd":1,"uuid":"8900966a-b393-41e3-8f93-e5a43bd23622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6818","nonce":2494994479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6821","nonce":2494994479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6828","nonce":2494994479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6823","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6825","nonce":2494994479}]},"public_addr":"172.21.15.105:6818/2494994479","cluster_addr":"172.21.15.105:6821/2494994479","heartbeat_back_addr":"172.21.15.105:6828/2494994479","heartbeat_front_addr":"172.21.15.105:6825/2494994479","state":["exists","up"]},{"osd":2,"uuid":"81a5dc7a-9325-4ff5-941d-a2d5fba21f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6809","nonce":3798408699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6811","nonce":3798408699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6815","nonce":3798408699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6813","nonce":3798408699}]},"public_addr":"172.21.15.105:6809/3798408699","cluster_addr":"172.21.15.105:6811/3798408699","heartbeat_back_addr":"172.21.15.105:6815/3798408699","heartbeat_front_addr":"172.21.15.105:6813/3798408699","state":["exists","up"]},{"osd":3,"uuid":"df766ed2-5c71-4805-96f4-3d4970edc960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6819","nonce":1920720186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6824","nonce":1920720186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6831","nonce":1920720186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6827","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6829","nonce":1920720186}]},"public_addr":"172.21.15.105:6819/1920720186","cluster_addr":"172.21.15.105:6824/1920720186","heartbeat_back_addr":"172.21.15.105:6831/1920720186","heartbeat_front_addr":"172.21.15.105:6829/1920720186","state":["exists","up"]},{"osd":4,"uuid":"e1039e03-28b8-49df-b951-c91930b93c9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6810","nonce":3516494456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6813","nonce":3516494456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6820","nonce":3516494456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6817","nonce":3516494456}]},"public_addr":"172.21.15.154:6810/3516494456","cluster_addr":"172.21.15.154:6813/3516494456","heartbeat_back_addr":"172.21.15.154:6820/3516494456","heartbeat_front_addr":"172.21.15.154:6817/3516494456","state":["exists","up"]},{"osd":5,"uuid":"a9f5eaf1-66ff-405a-a9e4-ec3340812a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6801","nonce":540367751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6803","nonce":540367751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6807","nonce":540367751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6805","nonce":540367751}]},"public_addr":"172.21.15.154:6801/540367751","cluster_addr":"172.21.15.154:6803/540367751","heartbeat_back_addr":"172.21.15.154:6807/540367751","heartbeat_front_addr":"172.21.15.154:6805/540367751","state":["exists","up"]},{"osd":6,"uuid":"267d02ea-675f-428f-aae8-e0375e4d4feb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6825","nonce":1550829005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6827","nonce":1550829005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6831","nonce":1550829005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6829","nonce":1550829005}]},"public_addr":"172.21.15.154:6825/1550829005","cluster_addr":"172.21.15.154:6827/1550829005","heartbeat_back_addr":"172.21.15.154:6831/1550829005","heartbeat_front_addr":"172.21.15.154:6829/1550829005","state":["exists","up"]},{"osd":7,"uuid":"759ac6cc-896a-423c-885f-9fe90d8dcb14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6809","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6811","nonce":3920089903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6816","nonce":3920089903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6823","nonce":3920089903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6819","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6821","nonce":3920089903}]},"public_addr":"172.21.15.154:6811/3920089903","cluster_addr":"172.21.15.154:6816/3920089903","heartbeat_back_addr":"172.21.15.154:6823/3920089903","heartbeat_front_addr":"172.21.15.154:6821/3920089903","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.949509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.984072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.907278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.947391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.975447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.998896+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:09.028739+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.894119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.105:6837/958066041":"2024-04-16T15:21:03.660841+0000","172.21.15.105:6835/958066041":"2024-04-16T15:21:03.660841+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:21:13.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.185+0000 7f09a32c5640 1 -- 172.21.15.105:0/1134629327 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f098006c980 msgr2=0x7f098006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:13.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.185+0000 7f09a32c5640 1 --2- 172.21.15.105:0/1134629327 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f098006c980 0x7f098006ee40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f099c14d6b0 tx=0x7f098c008040 comp rx=0 tx=0).stop 2024-04-15T15:21:13.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.185+0000 7f09a32c5640 1 -- 172.21.15.105:0/1134629327 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f099c0d1710 msgr2=0x7f099c0b5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:13.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.185+0000 7f09a32c5640 1 --2- 172.21.15.105:0/1134629327 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f099c0d1710 0x7f099c0b5df0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f0998000f80 tx=0x7f0998004a40 comp rx=0 tx=0).stop 2024-04-15T15:21:13.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.185+0000 7f09a32c5640 1 -- 172.21.15.105:0/1134629327 shutdown_connections 2024-04-15T15:21:13.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.185+0000 7f09a32c5640 1 --2- 172.21.15.105:0/1134629327 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f099c15eb60 0x7f099c1f3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:13.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.185+0000 7f09a32c5640 1 --2- 172.21.15.105:0/1134629327 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f098006c980 0x7f098006ee40 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:13.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.185+0000 7f09a32c5640 1 --2- 172.21.15.105:0/1134629327 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f099c162b70 0x7f099c0c9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:13.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.185+0000 7f09a32c5640 1 --2- 172.21.15.105:0/1134629327 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f099c0d1710 0x7f099c0b5df0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:13.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.185+0000 7f09a32c5640 1 -- 172.21.15.105:0/1134629327 >> 172.21.15.105:0/1134629327 conn(0x7f099c067720 msgr2=0x7f099c0cfa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:13.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.185+0000 7f09a32c5640 1 -- 172.21.15.105:0/1134629327 shutdown_connections 2024-04-15T15:21:13.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.185+0000 7f09a32c5640 1 -- 172.21.15.105:0/1134629327 wait complete. 2024-04-15T15:21:13.199 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:19:13.748585+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-15T15:20:26.983353+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-15T15:20:27.961802+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:21:13.199 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:21:13.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.304+0000 7f60ace4a640 1 Processor -- start 2024-04-15T15:21:13.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.304+0000 7f60ace4a640 1 -- start start 2024-04-15T15:21:13.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.305+0000 7f60ace4a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f60a806fc90 0x7f60a815e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:13.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.305+0000 7f60ace4a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60a80700d0 con 0x7f60a815ee90 2024-04-15T15:21:13.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.305+0000 7f60ace4a640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60a815f3b0 con 0x7f60a815e570 2024-04-15T15:21:13.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.305+0000 7f60ace4a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f60a806f860 con 0x7f60a806fc90 2024-04-15T15:21:13.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.305+0000 7f60a6d9d640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f60a815ee90 0x7f60a8072270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:37546/0 (socket says 172.21.15.105:37546) 2024-04-15T15:21:13.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.305+0000 7f60a6d9d640 1 -- 172.21.15.105:0/3959943682 learned_addr learned my addr 172.21.15.105:0/3959943682 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:21:13.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.305+0000 7f609dd9b640 1 --2- 172.21.15.105:0/3959943682 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f60a806fc90 0x7f60a815e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:13.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.305+0000 7f60a5d9b640 1 -- 172.21.15.105:0/3959943682 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2291087043 0 0) 0x7f60a80700d0 con 0x7f60a815ee90 2024-04-15T15:21:13.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.305+0000 7f60a5d9b640 1 -- 172.21.15.105:0/3959943682 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6094003520 con 0x7f60a815ee90 2024-04-15T15:21:13.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.305+0000 7f60a5d9b640 1 -- 172.21.15.105:0/3959943682 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1867399664 0 0) 0x7f60a815f3b0 con 0x7f60a815e570 2024-04-15T15:21:13.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.305+0000 7f60a5d9b640 1 -- 172.21.15.105:0/3959943682 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f60a80700d0 con 0x7f60a815e570 2024-04-15T15:21:13.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.305+0000 7f60a5d9b640 1 -- 172.21.15.105:0/3959943682 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 995646592 0 0) 0x7f6094003520 con 0x7f60a815ee90 2024-04-15T15:21:13.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.305+0000 7f609dd9b640 1 -- 172.21.15.105:0/3959943682 >> v1:172.21.15.154:6789/0 conn(0x7f60a815e570 legacy=0x7f60a80aebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:13.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.305+0000 7f609dd9b640 1 -- 172.21.15.105:0/3959943682 >> v1:172.21.15.105:6789/0 conn(0x7f60a815ee90 legacy=0x7f60a8072270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:13.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.305+0000 7f609dd9b640 1 -- 172.21.15.105:0/3959943682 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60a815f690 con 0x7f60a806fc90 2024-04-15T15:21:13.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.306+0000 7f609dd9b640 1 --2- 172.21.15.105:0/3959943682 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f60a806fc90 0x7f60a815e950 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f60900097d0 tx=0x7f609002b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ad7d607f4ee8ec server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:13.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.306+0000 7f60a5d9b640 1 -- 172.21.15.105:0/3959943682 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6090037070 con 0x7f60a806fc90 2024-04-15T15:21:13.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.306+0000 7f60a5d9b640 1 -- 172.21.15.105:0/3959943682 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6090030510 con 0x7f60a806fc90 2024-04-15T15:21:13.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.306+0000 7f60a5d9b640 1 -- 172.21.15.105:0/3959943682 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a00019f0 con 0x7f60a806fc90 2024-04-15T15:21:13.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.307+0000 7f60ace4a640 1 -- 172.21.15.105:0/3959943682 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f60a806fc90 msgr2=0x7f60a815e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:13.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.307+0000 7f60ace4a640 1 --2- 172.21.15.105:0/3959943682 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f60a806fc90 0x7f60a815e950 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f60900097d0 tx=0x7f609002b800 comp rx=0 tx=0).stop 2024-04-15T15:21:13.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.307+0000 7f60ace4a640 1 -- 172.21.15.105:0/3959943682 shutdown_connections 2024-04-15T15:21:13.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.307+0000 7f60ace4a640 1 --2- 172.21.15.105:0/3959943682 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f60a806fc90 0x7f60a815e950 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:13.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.307+0000 7f60ace4a640 1 -- 172.21.15.105:0/3959943682 >> 172.21.15.105:0/3959943682 conn(0x7f60a80b2640 msgr2=0x7f60a80b2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:13.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.307+0000 7f60ace4a640 1 -- 172.21.15.105:0/3959943682 shutdown_connections 2024-04-15T15:21:13.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.307+0000 7f60ace4a640 1 -- 172.21.15.105:0/3959943682 wait complete. 2024-04-15T15:21:13.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.308+0000 7f60ace4a640 1 Processor -- start 2024-04-15T15:21:13.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.309+0000 7f60ace4a640 1 -- start start 2024-04-15T15:21:13.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.309+0000 7f60ace4a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f60a806fc90 0x7f60a80b1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:13.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.309+0000 7f60ace4a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f60a815e570 0x7f60a81e9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:13.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.309+0000 7f60ace4a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60a815ee90 0x7f60a81e8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:13.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.309+0000 7f60ace4a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f60a80af2e0 con 0x7f60a806fc90 2024-04-15T15:21:13.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.309+0000 7f60ace4a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f60a815fc50 con 0x7f60a815ee90 2024-04-15T15:21:13.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.309+0000 7f60ace4a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f60a8072980 con 0x7f60a815e570 2024-04-15T15:21:13.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.309+0000 7f60a659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f60a806fc90 0x7f60a80b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:13.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.309+0000 7f60a659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f60a806fc90 0x7f60a80b1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33224/0 (socket says 172.21.15.105:33224) 2024-04-15T15:21:13.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.309+0000 7f60a659c640 1 -- 172.21.15.105:0/3103071437 learned_addr learned my addr 172.21.15.105:0/3103071437 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:13.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.310+0000 7f60a6d9d640 1 --2- 172.21.15.105:0/3103071437 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60a815ee90 0x7f60a81e8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:13.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.310+0000 7f609dd9b640 1 --2- 172.21.15.105:0/3103071437 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f60a815e570 0x7f60a81e9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:13.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.310+0000 7f60a659c640 1 -- 172.21.15.105:0/3103071437 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f60a815e570 msgr2=0x7f60a81e9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:13.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.310+0000 7f60a659c640 1 --2- 172.21.15.105:0/3103071437 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f60a815e570 0x7f60a81e9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:13.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.310+0000 7f60a659c640 1 -- 172.21.15.105:0/3103071437 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60a815ee90 msgr2=0x7f60a81e8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:13.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.310+0000 7f60a659c640 1 --2- 172.21.15.105:0/3103071437 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60a815ee90 0x7f60a81e8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:13.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.310+0000 7f60a659c640 1 -- 172.21.15.105:0/3103071437 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60a80b0890 con 0x7f60a806fc90 2024-04-15T15:21:13.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.310+0000 7f60a659c640 1 --2- 172.21.15.105:0/3103071437 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f60a806fc90 0x7f60a80b1930 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f609800c500 tx=0x7f609800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:13.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.310+0000 7f609f7fe640 1 -- 172.21.15.105:0/3103071437 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60980052d0 con 0x7f60a806fc90 2024-04-15T15:21:13.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.310+0000 7f609f7fe640 1 -- 172.21.15.105:0/3103071437 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6098005490 con 0x7f60a806fc90 2024-04-15T15:21:13.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.310+0000 7f60ace4a640 1 -- 172.21.15.105:0/3103071437 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60a80b0b70 con 0x7f60a806fc90 2024-04-15T15:21:13.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.310+0000 7f609f7fe640 1 -- 172.21.15.105:0/3103071437 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6098012870 con 0x7f60a806fc90 2024-04-15T15:21:13.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.310+0000 7f60ace4a640 1 -- 172.21.15.105:0/3103071437 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60a80b1130 con 0x7f60a806fc90 2024-04-15T15:21:13.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.311+0000 7f60ace4a640 1 -- 172.21.15.105:0/3103071437 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60a8068ab0 con 0x7f60a806fc90 2024-04-15T15:21:13.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.312+0000 7f609f7fe640 1 -- 172.21.15.105:0/3103071437 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f6098012a10 con 0x7f60a806fc90 2024-04-15T15:21:13.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.312+0000 7f609f7fe640 1 --2- 172.21.15.105:0/3103071437 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f608806c980 0x7f608806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:13.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.312+0000 7f609f7fe640 1 -- 172.21.15.105:0/3103071437 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5795+0+0 (secure 0 0 0) 0x7f609808b8d0 con 0x7f60a806fc90 2024-04-15T15:21:13.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.312+0000 7f609dd9b640 1 --2- 172.21.15.105:0/3103071437 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f608806c980 0x7f608806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:13.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.312+0000 7f609dd9b640 1 --2- 172.21.15.105:0/3103071437 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f608806c980 0x7f608806ee40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6090034000 tx=0x7f6090008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:13.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.316+0000 7f609f7fe640 1 -- 172.21.15.105:0/3103071437 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6098059cc0 con 0x7f60a806fc90 2024-04-15T15:21:13.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.492+0000 7f60ace4a640 1 -- 172.21.15.105:0/3103071437 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f60a80cd100 con 0x7f60a806fc90 2024-04-15T15:21:13.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.492+0000 7f609f7fe640 1 -- 172.21.15.105:0/3103071437 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) ==== 93+0+10 (secure 0 0 0) 0x7f60980595c0 con 0x7f60a806fc90 2024-04-15T15:21:13.494 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-04-15T15:21:13.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.494+0000 7f60ace4a640 1 -- 172.21.15.105:0/3103071437 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f608806c980 msgr2=0x7f608806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:13.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.494+0000 7f60ace4a640 1 --2- 172.21.15.105:0/3103071437 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f608806c980 0x7f608806ee40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6090034000 tx=0x7f6090008030 comp rx=0 tx=0).stop 2024-04-15T15:21:13.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.494+0000 7f60ace4a640 1 -- 172.21.15.105:0/3103071437 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f60a806fc90 msgr2=0x7f60a80b1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:13.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.494+0000 7f60ace4a640 1 --2- 172.21.15.105:0/3103071437 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f60a806fc90 0x7f60a80b1930 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f609800c500 tx=0x7f609800c9c0 comp rx=0 tx=0).stop 2024-04-15T15:21:13.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.494+0000 7f60ace4a640 1 -- 172.21.15.105:0/3103071437 shutdown_connections 2024-04-15T15:21:13.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.494+0000 7f60ace4a640 1 --2- 172.21.15.105:0/3103071437 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f60a815e570 0x7f60a81e9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:13.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.494+0000 7f60ace4a640 1 --2- 172.21.15.105:0/3103071437 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f608806c980 0x7f608806ee40 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:13.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.494+0000 7f60ace4a640 1 --2- 172.21.15.105:0/3103071437 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60a815ee90 0x7f60a81e8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:13.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.494+0000 7f60ace4a640 1 --2- 172.21.15.105:0/3103071437 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f60a806fc90 0x7f60a80b1930 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:13.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.494+0000 7f60ace4a640 1 -- 172.21.15.105:0/3103071437 >> 172.21.15.105:0/3103071437 conn(0x7f60a80b2640 msgr2=0x7f60a806ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:13.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.494+0000 7f60ace4a640 1 -- 172.21.15.105:0/3103071437 shutdown_connections 2024-04-15T15:21:13.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.494+0000 7f60ace4a640 1 -- 172.21.15.105:0/3103071437 wait complete. 2024-04-15T15:21:13.507 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-15T15:21:13.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.611+0000 7fdf7c44e640 1 Processor -- start 2024-04-15T15:21:13.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.611+0000 7fdf7c44e640 1 -- start start 2024-04-15T15:21:13.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.612+0000 7fdf7c44e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdf741620c0 0x7fdf740b7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:13.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.612+0000 7fdf7c44e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fdf740d2b40 0x7fdf740ba190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:13.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.612+0000 7fdf7c44e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdf741625c0 0x7fdf74162a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:13.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.612+0000 7fdf7c44e640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fdf740b8c00 con 0x7fdf741625c0 2024-04-15T15:21:13.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.612+0000 7fdf7c44e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fdf740d2fc0 con 0x7fdf740d2b40 2024-04-15T15:21:13.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.612+0000 7fdf7c44e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fdf740b7ff0 con 0x7fdf741620c0 2024-04-15T15:21:13.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.612+0000 7fdf7a9eb640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdf741625c0 0x7fdf74162a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:13.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.613+0000 7fdf7a9eb640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdf741625c0 0x7fdf74162a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33230/0 (socket says 172.21.15.105:33230) 2024-04-15T15:21:13.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.613+0000 7fdf7a9eb640 1 -- 172.21.15.105:0/2047943137 learned_addr learned my addr 172.21.15.105:0/2047943137 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:13.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.613+0000 7fdf7a1ea640 1 --2- 172.21.15.105:0/2047943137 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdf741620c0 0x7fdf740b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:13.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.613+0000 7fdf7a9eb640 1 -- 172.21.15.105:0/2047943137 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdf741620c0 msgr2=0x7fdf740b7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:13.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.613+0000 7fdf7a9eb640 1 --2- 172.21.15.105:0/2047943137 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdf741620c0 0x7fdf740b7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:13.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.613+0000 7fdf7a9eb640 1 -- 172.21.15.105:0/2047943137 >> v2:172.21.15.154:3300/0 conn(0x7fdf740d2b40 msgr2=0x7fdf740ba190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:13.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.613+0000 7fdf799e9640 1 --2- 172.21.15.105:0/2047943137 >> v2:172.21.15.154:3300/0 conn(0x7fdf740d2b40 0x7fdf740ba190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:13.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.613+0000 7fdf7a9eb640 1 --2- 172.21.15.105:0/2047943137 >> v2:172.21.15.154:3300/0 conn(0x7fdf740d2b40 0x7fdf740ba190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:13.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.613+0000 7fdf7a9eb640 1 -- 172.21.15.105:0/2047943137 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf740b82d0 con 0x7fdf741625c0 2024-04-15T15:21:13.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.614+0000 7fdf7a9eb640 1 --2- 172.21.15.105:0/2047943137 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf741625c0 0x7fdf74162a50 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fdf740d2360 tx=0x7fdf6c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68c0e05b7a9160a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:13.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.614+0000 7fdf791e8640 1 -- 172.21.15.105:0/2047943137 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c035020 con 0x7fdf741625c0 2024-04-15T15:21:13.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.614+0000 7fdf791e8640 1 -- 172.21.15.105:0/2047943137 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf6c031870 con 0x7fdf741625c0 2024-04-15T15:21:13.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.614+0000 7fdf791e8640 1 -- 172.21.15.105:0/2047943137 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c031c40 con 0x7fdf741625c0 2024-04-15T15:21:13.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.614+0000 7fdf7c44e640 1 -- 172.21.15.105:0/2047943137 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf741625c0 msgr2=0x7fdf74162a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:13.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.614+0000 7fdf7c44e640 1 --2- 172.21.15.105:0/2047943137 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf741625c0 0x7fdf74162a50 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fdf740d2360 tx=0x7fdf6c02b910 comp rx=0 tx=0).stop 2024-04-15T15:21:13.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.614+0000 7fdf7c44e640 1 -- 172.21.15.105:0/2047943137 shutdown_connections 2024-04-15T15:21:13.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.614+0000 7fdf7c44e640 1 --2- 172.21.15.105:0/2047943137 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf741625c0 0x7fdf74162a50 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:13.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.614+0000 7fdf7c44e640 1 --2- 172.21.15.105:0/2047943137 >> v2:172.21.15.154:3300/0 conn(0x7fdf740d2b40 0x7fdf740ba190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:13.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.614+0000 7fdf7c44e640 1 --2- 172.21.15.105:0/2047943137 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdf741620c0 0x7fdf740b7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:13.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.614+0000 7fdf7c44e640 1 -- 172.21.15.105:0/2047943137 >> 172.21.15.105:0/2047943137 conn(0x7fdf7406ccb0 msgr2=0x7fdf7406d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:13.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.614+0000 7fdf7c44e640 1 -- 172.21.15.105:0/2047943137 shutdown_connections 2024-04-15T15:21:13.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.614+0000 7fdf7c44e640 1 -- 172.21.15.105:0/2047943137 wait complete. 2024-04-15T15:21:13.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.616+0000 7fdf7c44e640 1 Processor -- start 2024-04-15T15:21:13.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.616+0000 7fdf7c44e640 1 -- start start 2024-04-15T15:21:13.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.616+0000 7fdf7c44e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf740d2b40 0x7fdf74152e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:13.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.616+0000 7fdf7c44e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdf741620c0 0x7fdf74153ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:13.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.616+0000 7fdf7c44e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf741625c0 0x7fdf740cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:13.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.616+0000 7fdf7c44e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdf740ba6d0 con 0x7fdf741625c0 2024-04-15T15:21:13.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.616+0000 7fdf7c44e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdf740b8890 con 0x7fdf740d2b40 2024-04-15T15:21:13.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.616+0000 7fdf7c44e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fdf740b9090 con 0x7fdf741620c0 2024-04-15T15:21:13.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.616+0000 7fdf7a9eb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf741625c0 0x7fdf740cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:13.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.616+0000 7fdf7a9eb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf741625c0 0x7fdf740cccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33244/0 (socket says 172.21.15.105:33244) 2024-04-15T15:21:13.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.616+0000 7fdf7a9eb640 1 -- 172.21.15.105:0/3394645124 learned_addr learned my addr 172.21.15.105:0/3394645124 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:13.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.617+0000 7fdf7a1ea640 1 --2- 172.21.15.105:0/3394645124 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf740d2b40 0x7fdf74152e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:13.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.617+0000 7fdf799e9640 1 --2- 172.21.15.105:0/3394645124 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdf741620c0 0x7fdf74153ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:13.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.617+0000 7fdf7a9eb640 1 -- 172.21.15.105:0/3394645124 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdf741620c0 msgr2=0x7fdf74153ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:13.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.617+0000 7fdf7a9eb640 1 --2- 172.21.15.105:0/3394645124 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdf741620c0 0x7fdf74153ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:13.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.617+0000 7fdf7a9eb640 1 -- 172.21.15.105:0/3394645124 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf740d2b40 msgr2=0x7fdf74152e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:13.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.617+0000 7fdf7a9eb640 1 --2- 172.21.15.105:0/3394645124 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf740d2b40 0x7fdf74152e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:13.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.617+0000 7fdf7a9eb640 1 -- 172.21.15.105:0/3394645124 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf741533a0 con 0x7fdf741625c0 2024-04-15T15:21:13.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.617+0000 7fdf799e9640 1 --2- 172.21.15.105:0/3394645124 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdf741620c0 0x7fdf74153ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:21:13.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.617+0000 7fdf7a9eb640 1 --2- 172.21.15.105:0/3394645124 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf741625c0 0x7fdf740cccc0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c0090b0 tx=0x7fdf6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:13.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.617+0000 7fdf6affd640 1 -- 172.21.15.105:0/3394645124 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c040070 con 0x7fdf741625c0 2024-04-15T15:21:13.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.617+0000 7fdf6affd640 1 -- 172.21.15.105:0/3394645124 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf6c03a420 con 0x7fdf741625c0 2024-04-15T15:21:13.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.617+0000 7fdf7c44e640 1 -- 172.21.15.105:0/3394645124 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf741570f0 con 0x7fdf741625c0 2024-04-15T15:21:13.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.617+0000 7fdf6affd640 1 -- 172.21.15.105:0/3394645124 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c035040 con 0x7fdf741625c0 2024-04-15T15:21:13.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.617+0000 7fdf7c44e640 1 -- 172.21.15.105:0/3394645124 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf74157670 con 0x7fdf741625c0 2024-04-15T15:21:13.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.618+0000 7fdf7c44e640 1 -- 172.21.15.105:0/3394645124 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf74068a40 con 0x7fdf741625c0 2024-04-15T15:21:13.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.618+0000 7fdf6affd640 1 -- 172.21.15.105:0/3394645124 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fdf6c04b060 con 0x7fdf741625c0 2024-04-15T15:21:13.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.619+0000 7fdf6affd640 1 --2- 172.21.15.105:0/3394645124 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fdf5806c980 0x7fdf5806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:13.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.619+0000 7fdf6affd640 1 -- 172.21.15.105:0/3394645124 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5795+0+0 (secure 0 0 0) 0x7fdf6c0ad9c0 con 0x7fdf741625c0 2024-04-15T15:21:13.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.619+0000 7fdf7a1ea640 1 --2- 172.21.15.105:0/3394645124 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fdf5806c980 0x7fdf5806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:13.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.619+0000 7fdf7a1ea640 1 --2- 172.21.15.105:0/3394645124 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fdf5806c980 0x7fdf5806ee40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fdf7000f510 tx=0x7fdf70007790 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:13.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.623+0000 7fdf6affd640 1 -- 172.21.15.105:0/3394645124 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdf741533a0 con 0x7fdf741625c0 2024-04-15T15:21:13.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.804+0000 7fdf7c44e640 1 -- 172.21.15.105:0/3394645124 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdf740d2410 con 0x7fdf741625c0 2024-04-15T15:21:13.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.804+0000 7fdf6affd640 1 -- 172.21.15.105:0/3394645124 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v35) ==== 104+0+11 (secure 0 0 0) 0x7fdf6c07be30 con 0x7fdf741625c0 2024-04-15T15:21:13.806 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-04-15T15:21:13.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.806+0000 7fdf7c44e640 1 -- 172.21.15.105:0/3394645124 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fdf5806c980 msgr2=0x7fdf5806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:13.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.806+0000 7fdf7c44e640 1 --2- 172.21.15.105:0/3394645124 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fdf5806c980 0x7fdf5806ee40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fdf7000f510 tx=0x7fdf70007790 comp rx=0 tx=0).stop 2024-04-15T15:21:13.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.806+0000 7fdf7c44e640 1 -- 172.21.15.105:0/3394645124 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf741625c0 msgr2=0x7fdf740cccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:13.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.806+0000 7fdf7c44e640 1 --2- 172.21.15.105:0/3394645124 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf741625c0 0x7fdf740cccc0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c0090b0 tx=0x7fdf6c0046c0 comp rx=0 tx=0).stop 2024-04-15T15:21:13.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.806+0000 7fdf7c44e640 1 -- 172.21.15.105:0/3394645124 shutdown_connections 2024-04-15T15:21:13.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.806+0000 7fdf7c44e640 1 --2- 172.21.15.105:0/3394645124 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdf741620c0 0x7fdf74153ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:13.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.806+0000 7fdf7c44e640 1 --2- 172.21.15.105:0/3394645124 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fdf5806c980 0x7fdf5806ee40 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:13.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.806+0000 7fdf7c44e640 1 --2- 172.21.15.105:0/3394645124 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf741625c0 0x7fdf740cccc0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:13.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.806+0000 7fdf7c44e640 1 --2- 172.21.15.105:0/3394645124 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf740d2b40 0x7fdf74152e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:13.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.806+0000 7fdf7c44e640 1 -- 172.21.15.105:0/3394645124 >> 172.21.15.105:0/3394645124 conn(0x7fdf7406ccb0 msgr2=0x7fdf740d0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:13.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.806+0000 7fdf7c44e640 1 -- 172.21.15.105:0/3394645124 shutdown_connections 2024-04-15T15:21:13.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.806+0000 7fdf7c44e640 1 -- 172.21.15.105:0/3394645124 wait complete. 2024-04-15T15:21:13.819 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-15T15:21:13.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.922+0000 7f0d63731640 1 Processor -- start 2024-04-15T15:21:13.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.923+0000 7f0d63731640 1 -- start start 2024-04-15T15:21:13.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.923+0000 7f0d63731640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d5c0b8220 0x7f0d5c09aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:13.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.923+0000 7f0d63731640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0d5c09b730 0x7f0d5c09af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:13.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.923+0000 7f0d63731640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d5c0ce3b0 con 0x7f0d5c15c970 2024-04-15T15:21:13.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.923+0000 7f0d63731640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f0d5c15cd50 con 0x7f0d5c09b730 2024-04-15T15:21:13.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.923+0000 7f0d63731640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f0d5c0b5450 con 0x7f0d5c0b8220 2024-04-15T15:21:13.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.924+0000 7f0d61cce640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0d5c09b730 0x7f0d5c09af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:13.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.924+0000 7f0d61cce640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0d5c09b730 0x7f0d5c09af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33260/0 (socket says 172.21.15.105:33260) 2024-04-15T15:21:13.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.924+0000 7f0d61cce640 1 -- 172.21.15.105:0/3268039097 learned_addr learned my addr 172.21.15.105:0/3268039097 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:13.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.924+0000 7f0d614cd640 1 --1- 172.21.15.105:0/3268039097 >> v1:172.21.15.105:6789/0 conn(0x7f0d5c15c970 0x7f0d5c0cdb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:37556/0 (socket says 172.21.15.105:37556) 2024-04-15T15:21:13.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.924+0000 7f0d61cce640 1 -- 172.21.15.105:0/3268039097 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d5c0b8220 msgr2=0x7f0d5c09aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:13.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.924+0000 7f0d60ccc640 1 --2- 172.21.15.105:0/3268039097 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d5c0b8220 0x7f0d5c09aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:13.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.924+0000 7f0d61cce640 1 --2- 172.21.15.105:0/3268039097 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d5c0b8220 0x7f0d5c09aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:13.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.924+0000 7f0d61cce640 1 -- 172.21.15.105:0/3268039097 >> v1:172.21.15.105:6789/0 conn(0x7f0d5c15c970 legacy=0x7f0d5c0cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:13.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.924+0000 7f0d61cce640 1 -- 172.21.15.105:0/3268039097 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d5c0b5730 con 0x7f0d5c09b730 2024-04-15T15:21:13.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.924+0000 7f0d60ccc640 1 --2- 172.21.15.105:0/3268039097 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d5c0b8220 0x7f0d5c09aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:21:13.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.925+0000 7f0d61cce640 1 --2- 172.21.15.105:0/3268039097 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d5c09b730 0x7f0d5c09af80 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f0d54009870 tx=0x7f0d5402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1a295ee4bbbc9f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:13.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.925+0000 7f0d5bfff640 1 -- 172.21.15.105:0/3268039097 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d54034070 con 0x7f0d5c09b730 2024-04-15T15:21:13.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.925+0000 7f0d5bfff640 1 -- 172.21.15.105:0/3268039097 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d54030840 con 0x7f0d5c09b730 2024-04-15T15:21:13.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.925+0000 7f0d5bfff640 1 -- 172.21.15.105:0/3268039097 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d54030c10 con 0x7f0d5c09b730 2024-04-15T15:21:13.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.925+0000 7f0d63731640 1 -- 172.21.15.105:0/3268039097 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d5c09b730 msgr2=0x7f0d5c09af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:13.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.925+0000 7f0d63731640 1 --2- 172.21.15.105:0/3268039097 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d5c09b730 0x7f0d5c09af80 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f0d54009870 tx=0x7f0d5402b980 comp rx=0 tx=0).stop 2024-04-15T15:21:13.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.925+0000 7f0d63731640 1 -- 172.21.15.105:0/3268039097 shutdown_connections 2024-04-15T15:21:13.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.925+0000 7f0d63731640 1 --2- 172.21.15.105:0/3268039097 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d5c09b730 0x7f0d5c09af80 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:13.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.925+0000 7f0d63731640 1 --2- 172.21.15.105:0/3268039097 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d5c0b8220 0x7f0d5c09aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:13.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.925+0000 7f0d63731640 1 -- 172.21.15.105:0/3268039097 >> 172.21.15.105:0/3268039097 conn(0x7f0d5c0b4050 msgr2=0x7f0d5c0b4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:13.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.925+0000 7f0d63731640 1 -- 172.21.15.105:0/3268039097 shutdown_connections 2024-04-15T15:21:13.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.925+0000 7f0d63731640 1 -- 172.21.15.105:0/3268039097 wait complete. 2024-04-15T15:21:13.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.927+0000 7f0d63731640 1 Processor -- start 2024-04-15T15:21:13.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.927+0000 7f0d63731640 1 -- start start 2024-04-15T15:21:13.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.927+0000 7f0d63731640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d5c09b730 0x7f0d5c153800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:13.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.927+0000 7f0d63731640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d5c0b8220 0x7f0d5c15f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:13.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.927+0000 7f0d63731640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d5c15c970 0x7f0d5c0c9630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:13.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.927+0000 7f0d63731640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5c166590 con 0x7f0d5c09b730 2024-04-15T15:21:13.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.927+0000 7f0d63731640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5c166410 con 0x7f0d5c0b8220 2024-04-15T15:21:13.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.927+0000 7f0d63731640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f0d5c166710 con 0x7f0d5c15c970 2024-04-15T15:21:13.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.927+0000 7f0d614cd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d5c09b730 0x7f0d5c153800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:13.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.927+0000 7f0d614cd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d5c09b730 0x7f0d5c153800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33274/0 (socket says 172.21.15.105:33274) 2024-04-15T15:21:13.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.927+0000 7f0d614cd640 1 -- 172.21.15.105:0/1074967220 learned_addr learned my addr 172.21.15.105:0/1074967220 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:13.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.928+0000 7f0d61cce640 1 --2- 172.21.15.105:0/1074967220 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d5c15c970 0x7f0d5c0c9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:13.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.928+0000 7f0d614cd640 1 -- 172.21.15.105:0/1074967220 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d5c15c970 msgr2=0x7f0d5c0c9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:13.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.928+0000 7f0d614cd640 1 --2- 172.21.15.105:0/1074967220 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d5c15c970 0x7f0d5c0c9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:13.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.928+0000 7f0d614cd640 1 -- 172.21.15.105:0/1074967220 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d5c0b8220 msgr2=0x7f0d5c15f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:13.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.928+0000 7f0d614cd640 1 --2- 172.21.15.105:0/1074967220 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d5c0b8220 0x7f0d5c15f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:13.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.928+0000 7f0d614cd640 1 -- 172.21.15.105:0/1074967220 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d5c153d40 con 0x7f0d5c09b730 2024-04-15T15:21:13.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.928+0000 7f0d614cd640 1 --2- 172.21.15.105:0/1074967220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d5c09b730 0x7f0d5c153800 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c00c730 tx=0x7f0d4c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:13.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.928+0000 7f0d59ffb640 1 -- 172.21.15.105:0/1074967220 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c017070 con 0x7f0d5c09b730 2024-04-15T15:21:13.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.928+0000 7f0d59ffb640 1 -- 172.21.15.105:0/1074967220 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d4c00ce40 con 0x7f0d5c09b730 2024-04-15T15:21:13.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.928+0000 7f0d59ffb640 1 -- 172.21.15.105:0/1074967220 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c00a860 con 0x7f0d5c09b730 2024-04-15T15:21:13.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.928+0000 7f0d63731640 1 -- 172.21.15.105:0/1074967220 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d5c154010 con 0x7f0d5c09b730 2024-04-15T15:21:13.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.928+0000 7f0d63731640 1 -- 172.21.15.105:0/1074967220 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d5c1545d0 con 0x7f0d5c09b730 2024-04-15T15:21:13.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.929+0000 7f0d63731640 1 -- 172.21.15.105:0/1074967220 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d5c068ab0 con 0x7f0d5c09b730 2024-04-15T15:21:13.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.930+0000 7f0d59ffb640 1 -- 172.21.15.105:0/1074967220 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f0d4c01a070 con 0x7f0d5c09b730 2024-04-15T15:21:13.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.930+0000 7f0d59ffb640 1 --2- 172.21.15.105:0/1074967220 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f0d4806c980 0x7f0d4806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:13.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.930+0000 7f0d59ffb640 1 -- 172.21.15.105:0/1074967220 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5795+0+0 (secure 0 0 0) 0x7f0d4c08b230 con 0x7f0d5c09b730 2024-04-15T15:21:13.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.930+0000 7f0d60ccc640 1 --2- 172.21.15.105:0/1074967220 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f0d4806c980 0x7f0d4806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:13.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.931+0000 7f0d60ccc640 1 --2- 172.21.15.105:0/1074967220 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f0d4806c980 0x7f0d4806ee40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0d50006fd0 tx=0x7f0d50008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:13.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:13.934+0000 7f0d59ffb640 1 -- 172.21.15.105:0/1074967220 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0d4c059610 con 0x7f0d5c09b730 2024-04-15T15:21:14.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.109+0000 7f0d63731640 1 -- 172.21.15.105:0/1074967220 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0d5c1558a0 con 0x7f0d5c09b730 2024-04-15T15:21:14.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.110+0000 7f0d59ffb640 1 -- 172.21.15.105:0/1074967220 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v35) ==== 100+0+11 (secure 0 0 0) 0x7f0d4c058f10 con 0x7f0d5c09b730 2024-04-15T15:21:14.112 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-04-15T15:21:14.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.112+0000 7f0d63731640 1 -- 172.21.15.105:0/1074967220 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f0d4806c980 msgr2=0x7f0d4806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:14.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.112+0000 7f0d63731640 1 --2- 172.21.15.105:0/1074967220 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f0d4806c980 0x7f0d4806ee40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0d50006fd0 tx=0x7f0d50008040 comp rx=0 tx=0).stop 2024-04-15T15:21:14.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.112+0000 7f0d63731640 1 -- 172.21.15.105:0/1074967220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d5c09b730 msgr2=0x7f0d5c153800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:14.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.112+0000 7f0d63731640 1 --2- 172.21.15.105:0/1074967220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d5c09b730 0x7f0d5c153800 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c00c730 tx=0x7f0d4c00cbf0 comp rx=0 tx=0).stop 2024-04-15T15:21:14.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.112+0000 7f0d63731640 1 -- 172.21.15.105:0/1074967220 shutdown_connections 2024-04-15T15:21:14.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.112+0000 7f0d63731640 1 --2- 172.21.15.105:0/1074967220 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d5c15c970 0x7f0d5c0c9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:14.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.112+0000 7f0d63731640 1 --2- 172.21.15.105:0/1074967220 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f0d4806c980 0x7f0d4806ee40 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:14.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.112+0000 7f0d63731640 1 --2- 172.21.15.105:0/1074967220 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d5c0b8220 0x7f0d5c15f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:14.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.112+0000 7f0d63731640 1 --2- 172.21.15.105:0/1074967220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d5c09b730 0x7f0d5c153800 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:14.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.112+0000 7f0d63731640 1 -- 172.21.15.105:0/1074967220 >> 172.21.15.105:0/1074967220 conn(0x7f0d5c0b4050 msgr2=0x7f0d5c0b36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:14.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.112+0000 7f0d63731640 1 -- 172.21.15.105:0/1074967220 shutdown_connections 2024-04-15T15:21:14.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.112+0000 7f0d63731640 1 -- 172.21.15.105:0/1074967220 wait complete. 2024-04-15T15:21:14.125 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-15T15:21:14.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.225+0000 7f6b1f4f2640 1 Processor -- start 2024-04-15T15:21:14.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.225+0000 7f6b1f4f2640 1 -- start start 2024-04-15T15:21:14.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.226+0000 7f6b1f4f2640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6b18071e60 0x7f6b180722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:14.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.226+0000 7f6b1f4f2640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6b1806df80 0x7f6b1806e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:14.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.226+0000 7f6b1f4f2640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b180ccfb0 con 0x7f6b18145e00 2024-04-15T15:21:14.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.226+0000 7f6b1f4f2640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f6b18072800 con 0x7f6b18071e60 2024-04-15T15:21:14.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.226+0000 7f6b1f4f2640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f6b180cdbc0 con 0x7f6b1806df80 2024-04-15T15:21:14.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.226+0000 7f6b1ca8d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6b18071e60 0x7f6b180722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:14.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.226+0000 7f6b1ca8d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6b18071e60 0x7f6b180722c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33288/0 (socket says 172.21.15.105:33288) 2024-04-15T15:21:14.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.226+0000 7f6b1ca8d640 1 -- 172.21.15.105:0/2620138616 learned_addr learned my addr 172.21.15.105:0/2620138616 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:14.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.226+0000 7f6b1da8f640 1 --2- 172.21.15.105:0/2620138616 >> v2:172.21.15.154:3300/0 conn(0x7f6b1806df80 0x7f6b1806e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:14.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.227+0000 7f6b1ca8d640 1 -- 172.21.15.105:0/2620138616 >> v2:172.21.15.154:3300/0 conn(0x7f6b1806df80 msgr2=0x7f6b1806e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:14.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.227+0000 7f6b1ca8d640 1 --2- 172.21.15.105:0/2620138616 >> v2:172.21.15.154:3300/0 conn(0x7f6b1806df80 0x7f6b1806e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:14.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.227+0000 7f6b1ca8d640 1 -- 172.21.15.105:0/2620138616 >> v1:172.21.15.154:6789/0 conn(0x7f6b18145e00 legacy=0x7f6b180ce7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:14.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.227+0000 7f6b1ca8d640 1 -- 172.21.15.105:0/2620138616 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b180cd290 con 0x7f6b18071e60 2024-04-15T15:21:14.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.227+0000 7f6b1da8f640 1 --2- 172.21.15.105:0/2620138616 >> v2:172.21.15.154:3300/0 conn(0x7f6b1806df80 0x7f6b1806e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:21:14.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.227+0000 7f6b1ca8d640 1 --2- 172.21.15.105:0/2620138616 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b18071e60 0x7f6b180722c0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f6b18157220 tx=0x7f6b0802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf0952209dbbc3c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:14.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.227+0000 7f6b0ffff640 1 -- 172.21.15.105:0/2620138616 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b08035020 con 0x7f6b18071e60 2024-04-15T15:21:14.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.227+0000 7f6b0ffff640 1 -- 172.21.15.105:0/2620138616 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b08031870 con 0x7f6b18071e60 2024-04-15T15:21:14.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.228+0000 7f6b0ffff640 1 -- 172.21.15.105:0/2620138616 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b08031c40 con 0x7f6b18071e60 2024-04-15T15:21:14.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.228+0000 7f6b1f4f2640 1 -- 172.21.15.105:0/2620138616 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b18071e60 msgr2=0x7f6b180722c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:14.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.228+0000 7f6b1f4f2640 1 --2- 172.21.15.105:0/2620138616 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b18071e60 0x7f6b180722c0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f6b18157220 tx=0x7f6b0802b910 comp rx=0 tx=0).stop 2024-04-15T15:21:14.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.228+0000 7f6b1f4f2640 1 -- 172.21.15.105:0/2620138616 shutdown_connections 2024-04-15T15:21:14.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.228+0000 7f6b1f4f2640 1 --2- 172.21.15.105:0/2620138616 >> v2:172.21.15.154:3300/0 conn(0x7f6b1806df80 0x7f6b1806e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:14.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.228+0000 7f6b1f4f2640 1 --2- 172.21.15.105:0/2620138616 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b18071e60 0x7f6b180722c0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:14.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.228+0000 7f6b1f4f2640 1 -- 172.21.15.105:0/2620138616 >> 172.21.15.105:0/2620138616 conn(0x7f6b18067d50 msgr2=0x7f6b1806b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:14.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.228+0000 7f6b1f4f2640 1 -- 172.21.15.105:0/2620138616 shutdown_connections 2024-04-15T15:21:14.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.228+0000 7f6b1f4f2640 1 -- 172.21.15.105:0/2620138616 wait complete. 2024-04-15T15:21:14.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.229+0000 7f6b1f4f2640 1 Processor -- start 2024-04-15T15:21:14.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.229+0000 7f6b1f4f2640 1 -- start start 2024-04-15T15:21:14.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.230+0000 7f6b1f4f2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b1806df80 0x7f6b181f5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:14.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.230+0000 7f6b1f4f2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b18071e60 0x7f6b18141860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:14.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.230+0000 7f6b1f4f2640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6b18145e00 0x7f6b18141da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:14.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.230+0000 7f6b1f4f2640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6b180cef00 con 0x7f6b18071e60 2024-04-15T15:21:14.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.230+0000 7f6b1f4f2640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6b180cd850 con 0x7f6b1806df80 2024-04-15T15:21:14.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.230+0000 7f6b1f4f2640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f6b180ce370 con 0x7f6b18145e00 2024-04-15T15:21:14.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.230+0000 7f6b1ca8d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b18071e60 0x7f6b18141860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:14.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.230+0000 7f6b1ca8d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b18071e60 0x7f6b18141860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33298/0 (socket says 172.21.15.105:33298) 2024-04-15T15:21:14.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.230+0000 7f6b1ca8d640 1 -- 172.21.15.105:0/1823409453 learned_addr learned my addr 172.21.15.105:0/1823409453 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:14.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.230+0000 7f6b1d28e640 1 --2- 172.21.15.105:0/1823409453 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b1806df80 0x7f6b181f5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:14.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.230+0000 7f6b1da8f640 1 --2- 172.21.15.105:0/1823409453 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6b18145e00 0x7f6b18141da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:14.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.230+0000 7f6b1ca8d640 1 -- 172.21.15.105:0/1823409453 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6b18145e00 msgr2=0x7f6b18141da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:14.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.230+0000 7f6b1ca8d640 1 --2- 172.21.15.105:0/1823409453 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6b18145e00 0x7f6b18141da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:14.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.230+0000 7f6b1ca8d640 1 -- 172.21.15.105:0/1823409453 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b1806df80 msgr2=0x7f6b181f5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:14.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.230+0000 7f6b1ca8d640 1 --2- 172.21.15.105:0/1823409453 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b1806df80 0x7f6b181f5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:14.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.230+0000 7f6b1ca8d640 1 -- 172.21.15.105:0/1823409453 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b181e6c00 con 0x7f6b18071e60 2024-04-15T15:21:14.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.230+0000 7f6b1da8f640 1 --2- 172.21.15.105:0/1823409453 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6b18145e00 0x7f6b18141da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:21:14.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.231+0000 7f6b1ca8d640 1 --2- 172.21.15.105:0/1823409453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b18071e60 0x7f6b18141860 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f6b08000c00 tx=0x7f6b080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:14.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.231+0000 7f6b0dffb640 1 -- 172.21.15.105:0/1823409453 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b08040070 con 0x7f6b18071e60 2024-04-15T15:21:14.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.231+0000 7f6b1f4f2640 1 -- 172.21.15.105:0/1823409453 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b181e6ee0 con 0x7f6b18071e60 2024-04-15T15:21:14.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.231+0000 7f6b0dffb640 1 -- 172.21.15.105:0/1823409453 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b0803a420 con 0x7f6b18071e60 2024-04-15T15:21:14.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.231+0000 7f6b0dffb640 1 -- 172.21.15.105:0/1823409453 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b08035040 con 0x7f6b18071e60 2024-04-15T15:21:14.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.231+0000 7f6b1f4f2640 1 -- 172.21.15.105:0/1823409453 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b181e74a0 con 0x7f6b18071e60 2024-04-15T15:21:14.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.232+0000 7f6b1f4f2640 1 -- 172.21.15.105:0/1823409453 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b18068a40 con 0x7f6b18071e60 2024-04-15T15:21:14.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.232+0000 7f6b0dffb640 1 -- 172.21.15.105:0/1823409453 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f6b0804b060 con 0x7f6b18071e60 2024-04-15T15:21:14.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.233+0000 7f6b0dffb640 1 --2- 172.21.15.105:0/1823409453 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f6afc06c980 0x7f6afc06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:14.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.233+0000 7f6b0dffb640 1 -- 172.21.15.105:0/1823409453 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5795+0+0 (secure 0 0 0) 0x7f6b080adb90 con 0x7f6b18071e60 2024-04-15T15:21:14.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.233+0000 7f6b1d28e640 1 --2- 172.21.15.105:0/1823409453 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f6afc06c980 0x7f6afc06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:14.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.233+0000 7f6b1d28e640 1 --2- 172.21.15.105:0/1823409453 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f6afc06c980 0x7f6afc06ee40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6b14004f60 tx=0x7f6b14007e30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:14.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.237+0000 7f6b0dffb640 1 -- 172.21.15.105:0/1823409453 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6b0807bef0 con 0x7f6b18071e60 2024-04-15T15:21:14.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.422+0000 7f6b1f4f2640 1 -- 172.21.15.105:0/1823409453 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6b180aff30 con 0x7f6b18071e60 2024-04-15T15:21:14.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.423+0000 7f6b0dffb640 1 -- 172.21.15.105:0/1823409453 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v35) ==== 80+0+122 (secure 0 0 0) 0x7f6b0807b7f0 con 0x7f6b18071e60 2024-04-15T15:21:14.425 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.105:6837/958066041 2024-04-16T15:21:03.660841+0000 2024-04-15T15:21:14.425 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.105:6835/958066041 2024-04-16T15:21:03.660841+0000 2024-04-15T15:21:14.425 INFO:teuthology.orchestra.run.smithi105.stderr:listed 2 entries 2024-04-15T15:21:14.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.425+0000 7f6b1f4f2640 1 -- 172.21.15.105:0/1823409453 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f6afc06c980 msgr2=0x7f6afc06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:14.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.425+0000 7f6b1f4f2640 1 --2- 172.21.15.105:0/1823409453 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f6afc06c980 0x7f6afc06ee40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6b14004f60 tx=0x7f6b14007e30 comp rx=0 tx=0).stop 2024-04-15T15:21:14.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.425+0000 7f6b1f4f2640 1 -- 172.21.15.105:0/1823409453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b18071e60 msgr2=0x7f6b18141860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:14.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.425+0000 7f6b1f4f2640 1 --2- 172.21.15.105:0/1823409453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b18071e60 0x7f6b18141860 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f6b08000c00 tx=0x7f6b080046c0 comp rx=0 tx=0).stop 2024-04-15T15:21:14.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.425+0000 7f6b1f4f2640 1 -- 172.21.15.105:0/1823409453 shutdown_connections 2024-04-15T15:21:14.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.425+0000 7f6b1f4f2640 1 --2- 172.21.15.105:0/1823409453 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6b18145e00 0x7f6b18141da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:14.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.425+0000 7f6b1f4f2640 1 --2- 172.21.15.105:0/1823409453 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f6afc06c980 0x7f6afc06ee40 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:14.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.425+0000 7f6b1f4f2640 1 --2- 172.21.15.105:0/1823409453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b18071e60 0x7f6b18141860 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:14.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.425+0000 7f6b1f4f2640 1 --2- 172.21.15.105:0/1823409453 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b1806df80 0x7f6b181f5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:14.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.425+0000 7f6b1f4f2640 1 -- 172.21.15.105:0/1823409453 >> 172.21.15.105:0/1823409453 conn(0x7f6b18067d50 msgr2=0x7f6b1815e570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:14.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.425+0000 7f6b1f4f2640 1 -- 172.21.15.105:0/1823409453 shutdown_connections 2024-04-15T15:21:14.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.425+0000 7f6b1f4f2640 1 -- 172.21.15.105:0/1823409453 wait complete. 2024-04-15T15:21:14.437 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi105.front.sepia.ceph.com... 2024-04-15T15:21:14.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:21:14.438 DEBUG:teuthology.orchestra.run.smithi105:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-04-15T15:21:14.465 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi105.stderr:ceph-fuse[54191]: fuse finished with error 0 and tester_r 0 2024-04-15T15:21:14.467 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi105.front.sepia.ceph.com 2024-04-15T15:21:14.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:21:14.467 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-15T15:21:14.634 INFO:tasks.cephfs.mount:Terminating background process 2024-04-15T15:21:14.634 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-15T15:21:14.634 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:21:14.655 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:21:14.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.776+0000 7fb16c64a640 1 Processor -- start 2024-04-15T15:21:14.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.776+0000 7fb16c64a640 1 -- start start 2024-04-15T15:21:14.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.777+0000 7fb16c64a640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb164072160 0x7fb1640725f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:14.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.777+0000 7fb16c64a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1640aefa0 0x7fb1640af430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:14.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.777+0000 7fb16c64a640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1640afcd0 con 0x7fb164147730 2024-04-15T15:21:14.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.777+0000 7fb16c64a640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fb1640affb0 con 0x7fb164072160 2024-04-15T15:21:14.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.777+0000 7fb16c64a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fb16406e130 con 0x7fb1640aefa0 2024-04-15T15:21:14.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.777+0000 7fb16abe7640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1640aefa0 0x7fb1640af430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:14.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.777+0000 7fb169be5640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb164072160 0x7fb1640725f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:14.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.777+0000 7fb16a3e6640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fb164147730 0x7fb1640b3940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.105:36272/0 (socket says 172.21.15.105:36272) 2024-04-15T15:21:14.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.777+0000 7fb16abe7640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1640aefa0 0x7fb1640af430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.105:45984/0 (socket says 172.21.15.105:45984) 2024-04-15T15:21:14.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.777+0000 7fb16a3e6640 1 -- 172.21.15.105:0/699123347 learned_addr learned my addr 172.21.15.105:0/699123347 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:21:14.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.777+0000 7fb169be5640 1 -- 172.21.15.105:0/699123347 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1640aefa0 msgr2=0x7fb1640af430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:14.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.777+0000 7fb169be5640 1 --2- 172.21.15.105:0/699123347 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1640aefa0 0x7fb1640af430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:14.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.777+0000 7fb169be5640 1 -- 172.21.15.105:0/699123347 >> v1:172.21.15.154:6789/0 conn(0x7fb164147730 legacy=0x7fb1640b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:14.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.777+0000 7fb169be5640 1 -- 172.21.15.105:0/699123347 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb16406e410 con 0x7fb164072160 2024-04-15T15:21:14.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.778+0000 7fb169be5640 1 --2- 172.21.15.105:0/699123347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb164072160 0x7fb1640725f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb1540097d0 tx=0x7fb15402b810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd5af75668e025a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:14.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.778+0000 7fb1693e4640 1 -- 172.21.15.105:0/699123347 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb154031770 con 0x7fb164072160 2024-04-15T15:21:14.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.778+0000 7fb1693e4640 1 -- 172.21.15.105:0/699123347 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb154037070 con 0x7fb164072160 2024-04-15T15:21:14.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.778+0000 7fb1693e4640 1 -- 172.21.15.105:0/699123347 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb154031b30 con 0x7fb164072160 2024-04-15T15:21:14.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.779+0000 7fb16c64a640 1 -- 172.21.15.105:0/699123347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb164072160 msgr2=0x7fb1640725f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:14.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.779+0000 7fb16c64a640 1 --2- 172.21.15.105:0/699123347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb164072160 0x7fb1640725f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb1540097d0 tx=0x7fb15402b810 comp rx=0 tx=0).stop 2024-04-15T15:21:14.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.779+0000 7fb16c64a640 1 -- 172.21.15.105:0/699123347 shutdown_connections 2024-04-15T15:21:14.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.779+0000 7fb16c64a640 1 --2- 172.21.15.105:0/699123347 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1640aefa0 0x7fb1640af430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:14.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.779+0000 7fb16c64a640 1 --2- 172.21.15.105:0/699123347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb164072160 0x7fb1640725f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:14.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.779+0000 7fb16c64a640 1 -- 172.21.15.105:0/699123347 >> 172.21.15.105:0/699123347 conn(0x7fb16406b9c0 msgr2=0x7fb16406bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:14.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.779+0000 7fb16c64a640 1 -- 172.21.15.105:0/699123347 shutdown_connections 2024-04-15T15:21:14.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.779+0000 7fb16c64a640 1 -- 172.21.15.105:0/699123347 wait complete. 2024-04-15T15:21:14.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.780+0000 7fb16c64a640 1 Processor -- start 2024-04-15T15:21:14.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.780+0000 7fb16c64a640 1 -- start start 2024-04-15T15:21:14.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.781+0000 7fb16c64a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb164072160 0x7fb1641f8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:14.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.781+0000 7fb16c64a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1640aefa0 0x7fb1641f4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:14.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.781+0000 7fb16c64a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb164147730 0x7fb16413a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:14.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.781+0000 7fb16c64a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb1640af970 con 0x7fb164072160 2024-04-15T15:21:14.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.781+0000 7fb16c64a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb164147b10 con 0x7fb1640aefa0 2024-04-15T15:21:14.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.781+0000 7fb16c64a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fb16406ec10 con 0x7fb164147730 2024-04-15T15:21:14.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.781+0000 7fb16a3e6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb164072160 0x7fb1641f8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:14.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.781+0000 7fb16a3e6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb164072160 0x7fb1641f8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33300/0 (socket says 172.21.15.105:33300) 2024-04-15T15:21:14.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.781+0000 7fb16a3e6640 1 -- 172.21.15.105:0/2235822179 learned_addr learned my addr 172.21.15.105:0/2235822179 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:14.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.781+0000 7fb169be5640 1 --2- 172.21.15.105:0/2235822179 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1640aefa0 0x7fb1641f4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:14.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.781+0000 7fb16abe7640 1 --2- 172.21.15.105:0/2235822179 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb164147730 0x7fb16413a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:14.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.782+0000 7fb16a3e6640 1 -- 172.21.15.105:0/2235822179 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb164147730 msgr2=0x7fb16413a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:14.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.782+0000 7fb16a3e6640 1 --2- 172.21.15.105:0/2235822179 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb164147730 0x7fb16413a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:14.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.782+0000 7fb16a3e6640 1 -- 172.21.15.105:0/2235822179 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1640aefa0 msgr2=0x7fb1641f4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:14.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.782+0000 7fb16a3e6640 1 --2- 172.21.15.105:0/2235822179 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1640aefa0 0x7fb1641f4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:14.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.782+0000 7fb16a3e6640 1 -- 172.21.15.105:0/2235822179 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1640cf870 con 0x7fb164072160 2024-04-15T15:21:14.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.782+0000 7fb169be5640 1 --2- 172.21.15.105:0/2235822179 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1640aefa0 0x7fb1641f4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:21:14.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.782+0000 7fb16a3e6640 1 --2- 172.21.15.105:0/2235822179 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb164072160 0x7fb1641f8d20 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fb16000c480 tx=0x7fb16000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:14.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.782+0000 7fb15affd640 1 -- 172.21.15.105:0/2235822179 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb160017070 con 0x7fb164072160 2024-04-15T15:21:14.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.782+0000 7fb15affd640 1 -- 172.21.15.105:0/2235822179 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb160005ce0 con 0x7fb164072160 2024-04-15T15:21:14.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.782+0000 7fb15affd640 1 -- 172.21.15.105:0/2235822179 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1600072f0 con 0x7fb164072160 2024-04-15T15:21:14.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.782+0000 7fb16c64a640 1 -- 172.21.15.105:0/2235822179 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1640cfe70 con 0x7fb164072160 2024-04-15T15:21:14.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.782+0000 7fb16c64a640 1 -- 172.21.15.105:0/2235822179 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1641f8240 con 0x7fb164072160 2024-04-15T15:21:14.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.783+0000 7fb16c64a640 1 -- 172.21.15.105:0/2235822179 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb164068a40 con 0x7fb164072160 2024-04-15T15:21:14.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.784+0000 7fb15affd640 1 -- 172.21.15.105:0/2235822179 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fb16001a070 con 0x7fb164072160 2024-04-15T15:21:14.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.784+0000 7fb15affd640 1 --2- 172.21.15.105:0/2235822179 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fb14806c980 0x7fb14806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:14.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.784+0000 7fb169be5640 1 --2- 172.21.15.105:0/2235822179 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fb14806c980 0x7fb14806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:14.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.784+0000 7fb15affd640 1 -- 172.21.15.105:0/2235822179 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5795+0+0 (secure 0 0 0) 0x7fb16008ae40 con 0x7fb164072160 2024-04-15T15:21:14.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.784+0000 7fb169be5640 1 --2- 172.21.15.105:0/2235822179 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fb14806c980 0x7fb14806ee40 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb154036000 tx=0x7fb15403d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:14.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.788+0000 7fb15affd640 1 -- 172.21.15.105:0/2235822179 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb160058c60 con 0x7fb164072160 2024-04-15T15:21:14.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.964+0000 7fb16c64a640 1 -- 172.21.15.105:0/2235822179 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb1640b0c70 con 0x7fb164072160 2024-04-15T15:21:14.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.966+0000 7fb15affd640 1 -- 172.21.15.105:0/2235822179 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16846 (secure 0 0 0) 0x7fb16005cf60 con 0x7fb164072160 2024-04-15T15:21:14.968 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:21:14.968 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":35,"fsid":"09e4d2a4-a324-4793-af77-644b25e520e1","created":"2024-04-15T15:19:01.524543+0000","modified":"2024-04-15T15:21:03.660870+0000","last_up_change":"2024-04-15T15:19:12.825815+0000","last_in_change":"2024-04-15T15:19:03.788126+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:19:13.748585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:20:26.983353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-15T15:20:27.961802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6f6292b-2d7e-4aa0-9d1c-6ddd0f97287c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6801","nonce":2873040054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6803","nonce":2873040054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6807","nonce":2873040054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6805","nonce":2873040054}]},"public_addr":"172.21.15.105:6801/2873040054","cluster_addr":"172.21.15.105:6803/2873040054","heartbeat_back_addr":"172.21.15.105:6807/2873040054","heartbeat_front_addr":"172.21.15.105:6805/2873040054","state":["exists","up"]},{"osd":1,"uuid":"8900966a-b393-41e3-8f93-e5a43bd23622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6818","nonce":2494994479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6821","nonce":2494994479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6828","nonce":2494994479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6823","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6825","nonce":2494994479}]},"public_addr":"172.21.15.105:6818/2494994479","cluster_addr":"172.21.15.105:6821/2494994479","heartbeat_back_addr":"172.21.15.105:6828/2494994479","heartbeat_front_addr":"172.21.15.105:6825/2494994479","state":["exists","up"]},{"osd":2,"uuid":"81a5dc7a-9325-4ff5-941d-a2d5fba21f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6809","nonce":3798408699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6811","nonce":3798408699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6815","nonce":3798408699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6813","nonce":3798408699}]},"public_addr":"172.21.15.105:6809/3798408699","cluster_addr":"172.21.15.105:6811/3798408699","heartbeat_back_addr":"172.21.15.105:6815/3798408699","heartbeat_front_addr":"172.21.15.105:6813/3798408699","state":["exists","up"]},{"osd":3,"uuid":"df766ed2-5c71-4805-96f4-3d4970edc960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6819","nonce":1920720186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6824","nonce":1920720186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6831","nonce":1920720186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6827","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6829","nonce":1920720186}]},"public_addr":"172.21.15.105:6819/1920720186","cluster_addr":"172.21.15.105:6824/1920720186","heartbeat_back_addr":"172.21.15.105:6831/1920720186","heartbeat_front_addr":"172.21.15.105:6829/1920720186","state":["exists","up"]},{"osd":4,"uuid":"e1039e03-28b8-49df-b951-c91930b93c9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6810","nonce":3516494456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6813","nonce":3516494456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6820","nonce":3516494456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6817","nonce":3516494456}]},"public_addr":"172.21.15.154:6810/3516494456","cluster_addr":"172.21.15.154:6813/3516494456","heartbeat_back_addr":"172.21.15.154:6820/3516494456","heartbeat_front_addr":"172.21.15.154:6817/3516494456","state":["exists","up"]},{"osd":5,"uuid":"a9f5eaf1-66ff-405a-a9e4-ec3340812a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6801","nonce":540367751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6803","nonce":540367751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6807","nonce":540367751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6805","nonce":540367751}]},"public_addr":"172.21.15.154:6801/540367751","cluster_addr":"172.21.15.154:6803/540367751","heartbeat_back_addr":"172.21.15.154:6807/540367751","heartbeat_front_addr":"172.21.15.154:6805/540367751","state":["exists","up"]},{"osd":6,"uuid":"267d02ea-675f-428f-aae8-e0375e4d4feb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6825","nonce":1550829005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6827","nonce":1550829005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6831","nonce":1550829005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6829","nonce":1550829005}]},"public_addr":"172.21.15.154:6825/1550829005","cluster_addr":"172.21.15.154:6827/1550829005","heartbeat_back_addr":"172.21.15.154:6831/1550829005","heartbeat_front_addr":"172.21.15.154:6829/1550829005","state":["exists","up"]},{"osd":7,"uuid":"759ac6cc-896a-423c-885f-9fe90d8dcb14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6809","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6811","nonce":3920089903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6816","nonce":3920089903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6823","nonce":3920089903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6819","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6821","nonce":3920089903}]},"public_addr":"172.21.15.154:6811/3920089903","cluster_addr":"172.21.15.154:6816/3920089903","heartbeat_back_addr":"172.21.15.154:6823/3920089903","heartbeat_front_addr":"172.21.15.154:6821/3920089903","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.949509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.984072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.907278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.947391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.975447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.998896+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:09.028739+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.894119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.105:6837/958066041":"2024-04-16T15:21:03.660841+0000","172.21.15.105:6835/958066041":"2024-04-16T15:21:03.660841+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:21:14.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.968+0000 7fb16c64a640 1 -- 172.21.15.105:0/2235822179 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fb14806c980 msgr2=0x7fb14806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:14.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.968+0000 7fb16c64a640 1 --2- 172.21.15.105:0/2235822179 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fb14806c980 0x7fb14806ee40 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb154036000 tx=0x7fb15403d040 comp rx=0 tx=0).stop 2024-04-15T15:21:14.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.968+0000 7fb16c64a640 1 -- 172.21.15.105:0/2235822179 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb164072160 msgr2=0x7fb1641f8d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:14.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.968+0000 7fb16c64a640 1 --2- 172.21.15.105:0/2235822179 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb164072160 0x7fb1641f8d20 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fb16000c480 tx=0x7fb16000c940 comp rx=0 tx=0).stop 2024-04-15T15:21:14.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.968+0000 7fb16c64a640 1 -- 172.21.15.105:0/2235822179 shutdown_connections 2024-04-15T15:21:14.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.968+0000 7fb16c64a640 1 --2- 172.21.15.105:0/2235822179 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb164147730 0x7fb16413a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:14.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.968+0000 7fb16c64a640 1 --2- 172.21.15.105:0/2235822179 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fb14806c980 0x7fb14806ee40 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:14.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.968+0000 7fb16c64a640 1 --2- 172.21.15.105:0/2235822179 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1640aefa0 0x7fb1641f4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:14.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.968+0000 7fb16c64a640 1 --2- 172.21.15.105:0/2235822179 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb164072160 0x7fb1641f8d20 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:14.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.968+0000 7fb16c64a640 1 -- 172.21.15.105:0/2235822179 >> 172.21.15.105:0/2235822179 conn(0x7fb16406b9c0 msgr2=0x7fb1641405b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:14.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.968+0000 7fb16c64a640 1 -- 172.21.15.105:0/2235822179 shutdown_connections 2024-04-15T15:21:14.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:14.968+0000 7fb16c64a640 1 -- 172.21.15.105:0/2235822179 wait complete. 2024-04-15T15:21:14.981 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:19:13.748585+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-15T15:20:26.983353+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-15T15:20:27.961802+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:21:14.981 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:21:15.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.085+0000 7f4d86b15640 1 Processor -- start 2024-04-15T15:21:15.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.085+0000 7f4d86b15640 1 -- start start 2024-04-15T15:21:15.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.086+0000 7f4d86b15640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4d80145f00 0x7f4d80071e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:15.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.086+0000 7f4d86b15640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4d8006e590 0x7f4d8015d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:15.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.086+0000 7f4d86b15640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d8006ebb0 con 0x7f4d800723a0 2024-04-15T15:21:15.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.086+0000 7f4d86b15640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f4d80072800 con 0x7f4d8006e590 2024-04-15T15:21:15.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.086+0000 7f4d86b15640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f4d801462e0 con 0x7f4d80145f00 2024-04-15T15:21:15.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.086+0000 7f4d848b1640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4d80145f00 0x7f4d80071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:15.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.086+0000 7f4d850b2640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4d8006e590 0x7f4d8015d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:15.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.086+0000 7f4d77fff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f4d800723a0 0x7f4d8006de80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.105:36288/0 (socket says 172.21.15.105:36288) 2024-04-15T15:21:15.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.086+0000 7f4d77fff640 1 -- 172.21.15.105:0/3849195796 learned_addr learned my addr 172.21.15.105:0/3849195796 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:21:15.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.087+0000 7f4d850b2640 1 -- 172.21.15.105:0/3849195796 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4d80145f00 msgr2=0x7f4d80071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:15.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.087+0000 7f4d850b2640 1 --2- 172.21.15.105:0/3849195796 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4d80145f00 0x7f4d80071e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:15.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.087+0000 7f4d850b2640 1 -- 172.21.15.105:0/3849195796 >> v1:172.21.15.154:6789/0 conn(0x7f4d800723a0 legacy=0x7f4d8006de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:15.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.087+0000 7f4d850b2640 1 -- 172.21.15.105:0/3849195796 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d8006ee90 con 0x7f4d8006e590 2024-04-15T15:21:15.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.087+0000 7f4d777fe640 1 -- 172.21.15.105:0/3849195796 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 167342429 0 0) 0x7f4d8006ebb0 con 0x7f4d800723a0 2024-04-15T15:21:15.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.087+0000 7f4d850b2640 1 --2- 172.21.15.105:0/3849195796 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d8006e590 0x7f4d8015d5e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4d780097a0 tx=0x7f4d7802b7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=126d78935858a9b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:15.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.088+0000 7f4d777fe640 1 -- 172.21.15.105:0/3849195796 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d78038070 con 0x7f4d8006e590 2024-04-15T15:21:15.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.088+0000 7f4d777fe640 1 -- 172.21.15.105:0/3849195796 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d78031700 con 0x7f4d8006e590 2024-04-15T15:21:15.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.088+0000 7f4d777fe640 1 -- 172.21.15.105:0/3849195796 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d78031ad0 con 0x7f4d8006e590 2024-04-15T15:21:15.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.088+0000 7f4d86b15640 1 -- 172.21.15.105:0/3849195796 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d8006e590 msgr2=0x7f4d8015d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:15.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.088+0000 7f4d86b15640 1 --2- 172.21.15.105:0/3849195796 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d8006e590 0x7f4d8015d5e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4d780097a0 tx=0x7f4d7802b7a0 comp rx=0 tx=0).stop 2024-04-15T15:21:15.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.088+0000 7f4d86b15640 1 -- 172.21.15.105:0/3849195796 shutdown_connections 2024-04-15T15:21:15.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.088+0000 7f4d86b15640 1 --2- 172.21.15.105:0/3849195796 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d8006e590 0x7f4d8015d5e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:15.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.088+0000 7f4d86b15640 1 --2- 172.21.15.105:0/3849195796 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4d80145f00 0x7f4d80071e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:15.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.088+0000 7f4d86b15640 1 -- 172.21.15.105:0/3849195796 >> 172.21.15.105:0/3849195796 conn(0x7f4d80067d50 msgr2=0x7f4d8006b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:15.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.088+0000 7f4d86b15640 1 -- 172.21.15.105:0/3849195796 shutdown_connections 2024-04-15T15:21:15.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.088+0000 7f4d86b15640 1 -- 172.21.15.105:0/3849195796 wait complete. 2024-04-15T15:21:15.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.089+0000 7f4d86b15640 1 Processor -- start 2024-04-15T15:21:15.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.089+0000 7f4d86b15640 1 -- start start 2024-04-15T15:21:15.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.090+0000 7f4d86b15640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4d8006e590 0x7f4d8015bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:15.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.090+0000 7f4d86b15640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d800723a0 0x7f4d800cb390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:15.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.090+0000 7f4d86b15640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d80145f00 0x7f4d8013be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:15.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.090+0000 7f4d86b15640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8015db20 con 0x7f4d800723a0 2024-04-15T15:21:15.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.090+0000 7f4d86b15640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8015d0c0 con 0x7f4d80145f00 2024-04-15T15:21:15.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.090+0000 7f4d86b15640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f4d8015dca0 con 0x7f4d8006e590 2024-04-15T15:21:15.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.090+0000 7f4d77fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d800723a0 0x7f4d800cb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:15.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.090+0000 7f4d77fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d800723a0 0x7f4d800cb390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33308/0 (socket says 172.21.15.105:33308) 2024-04-15T15:21:15.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.090+0000 7f4d77fff640 1 -- 172.21.15.105:0/3225406887 learned_addr learned my addr 172.21.15.105:0/3225406887 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:15.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.091+0000 7f4d848b1640 1 --2- 172.21.15.105:0/3225406887 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4d8006e590 0x7f4d8015bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:15.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.091+0000 7f4d850b2640 1 --2- 172.21.15.105:0/3225406887 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d80145f00 0x7f4d8013be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:15.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.091+0000 7f4d77fff640 1 -- 172.21.15.105:0/3225406887 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4d8006e590 msgr2=0x7f4d8015bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:15.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.091+0000 7f4d77fff640 1 --2- 172.21.15.105:0/3225406887 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4d8006e590 0x7f4d8015bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:15.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.091+0000 7f4d77fff640 1 -- 172.21.15.105:0/3225406887 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d80145f00 msgr2=0x7f4d8013be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:15.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.091+0000 7f4d77fff640 1 --2- 172.21.15.105:0/3225406887 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d80145f00 0x7f4d8013be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:15.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.091+0000 7f4d77fff640 1 -- 172.21.15.105:0/3225406887 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d800cbb00 con 0x7f4d800723a0 2024-04-15T15:21:15.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.091+0000 7f4d77fff640 1 --2- 172.21.15.105:0/3225406887 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d800723a0 0x7f4d800cb390 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f4d7000c4d0 tx=0x7f4d7000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:15.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.091+0000 7f4d757fa640 1 -- 172.21.15.105:0/3225406887 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d70018070 con 0x7f4d800723a0 2024-04-15T15:21:15.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.091+0000 7f4d757fa640 1 -- 172.21.15.105:0/3225406887 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d70007500 con 0x7f4d800723a0 2024-04-15T15:21:15.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.091+0000 7f4d86b15640 1 -- 172.21.15.105:0/3225406887 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d800cbde0 con 0x7f4d800723a0 2024-04-15T15:21:15.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.091+0000 7f4d757fa640 1 -- 172.21.15.105:0/3225406887 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d70004a20 con 0x7f4d800723a0 2024-04-15T15:21:15.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.091+0000 7f4d86b15640 1 -- 172.21.15.105:0/3225406887 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d8013c3a0 con 0x7f4d800723a0 2024-04-15T15:21:15.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.092+0000 7f4d86b15640 1 -- 172.21.15.105:0/3225406887 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d80068ab0 con 0x7f4d800723a0 2024-04-15T15:21:15.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.093+0000 7f4d757fa640 1 -- 172.21.15.105:0/3225406887 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f4d7001b070 con 0x7f4d800723a0 2024-04-15T15:21:15.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.093+0000 7f4d757fa640 1 --2- 172.21.15.105:0/3225406887 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f4d6c06c980 0x7f4d6c06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:15.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.093+0000 7f4d757fa640 1 -- 172.21.15.105:0/3225406887 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5795+0+0 (secure 0 0 0) 0x7f4d7008aee0 con 0x7f4d800723a0 2024-04-15T15:21:15.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.093+0000 7f4d848b1640 1 --2- 172.21.15.105:0/3225406887 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f4d6c06c980 0x7f4d6c06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:15.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.093+0000 7f4d848b1640 1 --2- 172.21.15.105:0/3225406887 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f4d6c06c980 0x7f4d6c06ee40 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c007380 tx=0x7f4d7c00b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:15.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.097+0000 7f4d757fa640 1 -- 172.21.15.105:0/3225406887 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4d70059350 con 0x7f4d800723a0 2024-04-15T15:21:15.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.272+0000 7f4d86b15640 1 -- 172.21.15.105:0/3225406887 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4d800691f0 con 0x7f4d800723a0 2024-04-15T15:21:15.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.273+0000 7f4d757fa640 1 -- 172.21.15.105:0/3225406887 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) ==== 93+0+10 (secure 0 0 0) 0x7f4d70058c50 con 0x7f4d800723a0 2024-04-15T15:21:15.277 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-04-15T15:21:15.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.274+0000 7f4d86b15640 1 -- 172.21.15.105:0/3225406887 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f4d6c06c980 msgr2=0x7f4d6c06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:15.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.274+0000 7f4d86b15640 1 --2- 172.21.15.105:0/3225406887 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f4d6c06c980 0x7f4d6c06ee40 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c007380 tx=0x7f4d7c00b040 comp rx=0 tx=0).stop 2024-04-15T15:21:15.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.274+0000 7f4d86b15640 1 -- 172.21.15.105:0/3225406887 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d800723a0 msgr2=0x7f4d800cb390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:15.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.274+0000 7f4d86b15640 1 --2- 172.21.15.105:0/3225406887 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d800723a0 0x7f4d800cb390 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f4d7000c4d0 tx=0x7f4d7000c990 comp rx=0 tx=0).stop 2024-04-15T15:21:15.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.275+0000 7f4d86b15640 1 -- 172.21.15.105:0/3225406887 shutdown_connections 2024-04-15T15:21:15.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.275+0000 7f4d86b15640 1 --2- 172.21.15.105:0/3225406887 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f4d6c06c980 0x7f4d6c06ee40 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:15.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.275+0000 7f4d86b15640 1 --2- 172.21.15.105:0/3225406887 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d80145f00 0x7f4d8013be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:15.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.275+0000 7f4d86b15640 1 --2- 172.21.15.105:0/3225406887 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d800723a0 0x7f4d800cb390 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:15.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.275+0000 7f4d86b15640 1 --2- 172.21.15.105:0/3225406887 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4d8006e590 0x7f4d8015bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:15.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.275+0000 7f4d86b15640 1 -- 172.21.15.105:0/3225406887 >> 172.21.15.105:0/3225406887 conn(0x7f4d80067d50 msgr2=0x7f4d8006b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:15.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.275+0000 7f4d86b15640 1 -- 172.21.15.105:0/3225406887 shutdown_connections 2024-04-15T15:21:15.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.275+0000 7f4d86b15640 1 -- 172.21.15.105:0/3225406887 wait complete. 2024-04-15T15:21:15.288 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-15T15:21:15.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.391+0000 7f8785837640 1 Processor -- start 2024-04-15T15:21:15.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.391+0000 7f8785837640 1 -- start start 2024-04-15T15:21:15.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.392+0000 7f8785837640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f87800d1910 0x7f87800cfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:15.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.392+0000 7f8785837640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87800b7dd0 con 0x7f8780161df0 2024-04-15T15:21:15.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.392+0000 7f8785837640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87800b80b0 con 0x7f87800b9e80 2024-04-15T15:21:15.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.392+0000 7f8785837640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f87800b8390 con 0x7f87800d1910 2024-04-15T15:21:15.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.392+0000 7f877effd640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f8780161df0 0x7f87800d1200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:37562/0 (socket says 172.21.15.105:37562) 2024-04-15T15:21:15.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.392+0000 7f877effd640 1 -- 172.21.15.105:0/1880943879 learned_addr learned my addr 172.21.15.105:0/1880943879 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:21:15.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.392+0000 7f877f7fe640 1 --2- 172.21.15.105:0/1880943879 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f87800d1910 0x7f87800cfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:15.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.392+0000 7f877dffb640 1 -- 172.21.15.105:0/1880943879 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3083838451 0 0) 0x7f87800b7dd0 con 0x7f8780161df0 2024-04-15T15:21:15.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.392+0000 7f877dffb640 1 -- 172.21.15.105:0/1880943879 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8768003520 con 0x7f8780161df0 2024-04-15T15:21:15.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.392+0000 7f877dffb640 1 -- 172.21.15.105:0/1880943879 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3158259053 0 0) 0x7f87800b80b0 con 0x7f87800b9e80 2024-04-15T15:21:15.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.392+0000 7f877dffb640 1 -- 172.21.15.105:0/1880943879 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f87800b7dd0 con 0x7f87800b9e80 2024-04-15T15:21:15.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.393+0000 7f877f7fe640 1 -- 172.21.15.105:0/1880943879 >> v1:172.21.15.154:6789/0 conn(0x7f87800b9e80 legacy=0x7f8780166680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:15.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.393+0000 7f877f7fe640 1 -- 172.21.15.105:0/1880943879 >> v1:172.21.15.105:6789/0 conn(0x7f8780161df0 legacy=0x7f87800d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:15.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.393+0000 7f877f7fe640 1 -- 172.21.15.105:0/1880943879 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87800b8670 con 0x7f87800d1910 2024-04-15T15:21:15.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.393+0000 7f877f7fe640 1 --2- 172.21.15.105:0/1880943879 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f87800d1910 0x7f87800cfaf0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8778009dc0 tx=0x7f877802bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a85b63b3ee72a7ea server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:15.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.394+0000 7f877dffb640 1 -- 172.21.15.105:0/1880943879 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8778039070 con 0x7f87800d1910 2024-04-15T15:21:15.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.394+0000 7f877dffb640 1 -- 172.21.15.105:0/1880943879 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f877802ccd0 con 0x7f87800d1910 2024-04-15T15:21:15.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.394+0000 7f877dffb640 1 -- 172.21.15.105:0/1880943879 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8778034640 con 0x7f87800d1910 2024-04-15T15:21:15.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.394+0000 7f8785837640 1 -- 172.21.15.105:0/1880943879 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f87800d1910 msgr2=0x7f87800cfaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:15.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.394+0000 7f8785837640 1 --2- 172.21.15.105:0/1880943879 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f87800d1910 0x7f87800cfaf0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8778009dc0 tx=0x7f877802bf20 comp rx=0 tx=0).stop 2024-04-15T15:21:15.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.394+0000 7f8785837640 1 -- 172.21.15.105:0/1880943879 shutdown_connections 2024-04-15T15:21:15.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.394+0000 7f8785837640 1 --2- 172.21.15.105:0/1880943879 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f87800d1910 0x7f87800cfaf0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:15.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.394+0000 7f8785837640 1 -- 172.21.15.105:0/1880943879 >> 172.21.15.105:0/1880943879 conn(0x7f8780067720 msgr2=0x7f8780067b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:15.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.394+0000 7f8785837640 1 -- 172.21.15.105:0/1880943879 shutdown_connections 2024-04-15T15:21:15.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.394+0000 7f8785837640 1 -- 172.21.15.105:0/1880943879 wait complete. 2024-04-15T15:21:15.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.395+0000 7f8785837640 1 Processor -- start 2024-04-15T15:21:15.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.396+0000 7f8785837640 1 -- start start 2024-04-15T15:21:15.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.396+0000 7f8785837640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87800b9e80 0x7f87801f4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:15.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.396+0000 7f8785837640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f87800d1910 0x7f8780151cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:15.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.396+0000 7f8785837640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8780161df0 0x7f87800cc280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:15.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.396+0000 7f8785837640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f87801621d0 con 0x7f87800b9e80 2024-04-15T15:21:15.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.396+0000 7f8785837640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f87800b7a80 con 0x7f8780161df0 2024-04-15T15:21:15.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.396+0000 7f8785837640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f8780162350 con 0x7f87800d1910 2024-04-15T15:21:15.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.396+0000 7f877effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87800b9e80 0x7f87801f4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:15.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.396+0000 7f877effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87800b9e80 0x7f87801f4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33316/0 (socket says 172.21.15.105:33316) 2024-04-15T15:21:15.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.396+0000 7f877effd640 1 -- 172.21.15.105:0/940803283 learned_addr learned my addr 172.21.15.105:0/940803283 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:15.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.397+0000 7f877e7fc640 1 --2- 172.21.15.105:0/940803283 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f87800d1910 0x7f8780151cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:15.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.397+0000 7f877f7fe640 1 --2- 172.21.15.105:0/940803283 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8780161df0 0x7f87800cc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:15.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.397+0000 7f877effd640 1 -- 172.21.15.105:0/940803283 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f87800d1910 msgr2=0x7f8780151cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:15.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.397+0000 7f877effd640 1 --2- 172.21.15.105:0/940803283 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f87800d1910 0x7f8780151cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:15.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.397+0000 7f877effd640 1 -- 172.21.15.105:0/940803283 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8780161df0 msgr2=0x7f87800cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:15.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.397+0000 7f877effd640 1 --2- 172.21.15.105:0/940803283 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8780161df0 0x7f87800cc280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:15.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.397+0000 7f877effd640 1 -- 172.21.15.105:0/940803283 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87800cca20 con 0x7f87800b9e80 2024-04-15T15:21:15.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.397+0000 7f877effd640 1 --2- 172.21.15.105:0/940803283 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87800b9e80 0x7f87801f4630 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f877000c430 tx=0x7f877000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:15.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.397+0000 7f8767fff640 1 -- 172.21.15.105:0/940803283 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8770017070 con 0x7f87800b9e80 2024-04-15T15:21:15.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.397+0000 7f8767fff640 1 -- 172.21.15.105:0/940803283 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8770005ce0 con 0x7f87800b9e80 2024-04-15T15:21:15.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.397+0000 7f8785837640 1 -- 172.21.15.105:0/940803283 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87801f4ce0 con 0x7f87800b9e80 2024-04-15T15:21:15.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.397+0000 7f8767fff640 1 -- 172.21.15.105:0/940803283 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87700072f0 con 0x7f87800b9e80 2024-04-15T15:21:15.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.397+0000 7f8785837640 1 -- 172.21.15.105:0/940803283 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87801f52a0 con 0x7f87800b9e80 2024-04-15T15:21:15.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.398+0000 7f8785837640 1 -- 172.21.15.105:0/940803283 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8780068a40 con 0x7f87800b9e80 2024-04-15T15:21:15.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.398+0000 7f8767fff640 1 -- 172.21.15.105:0/940803283 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f877001a070 con 0x7f87800b9e80 2024-04-15T15:21:15.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.399+0000 7f8767fff640 1 --2- 172.21.15.105:0/940803283 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f876c06c980 0x7f876c06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:15.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.399+0000 7f8767fff640 1 -- 172.21.15.105:0/940803283 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5795+0+0 (secure 0 0 0) 0x7f877008b0a0 con 0x7f87800b9e80 2024-04-15T15:21:15.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.399+0000 7f877e7fc640 1 --2- 172.21.15.105:0/940803283 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f876c06c980 0x7f876c06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:15.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.399+0000 7f877e7fc640 1 --2- 172.21.15.105:0/940803283 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f876c06c980 0x7f876c06ee40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f87800b76d0 tx=0x7f8774009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:15.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.403+0000 7f8767fff640 1 -- 172.21.15.105:0/940803283 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8770059480 con 0x7f87800b9e80 2024-04-15T15:21:15.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.579+0000 7f8785837640 1 -- 172.21.15.105:0/940803283 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f87800b38c0 con 0x7f87800b9e80 2024-04-15T15:21:15.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.579+0000 7f8767fff640 1 -- 172.21.15.105:0/940803283 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v35) ==== 104+0+11 (secure 0 0 0) 0x7f8770058d80 con 0x7f87800b9e80 2024-04-15T15:21:15.581 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-04-15T15:21:15.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.581+0000 7f8785837640 1 -- 172.21.15.105:0/940803283 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f876c06c980 msgr2=0x7f876c06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:15.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.581+0000 7f8785837640 1 --2- 172.21.15.105:0/940803283 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f876c06c980 0x7f876c06ee40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f87800b76d0 tx=0x7f8774009040 comp rx=0 tx=0).stop 2024-04-15T15:21:15.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.581+0000 7f8785837640 1 -- 172.21.15.105:0/940803283 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87800b9e80 msgr2=0x7f87801f4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:15.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.581+0000 7f8785837640 1 --2- 172.21.15.105:0/940803283 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87800b9e80 0x7f87801f4630 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f877000c430 tx=0x7f877000c8f0 comp rx=0 tx=0).stop 2024-04-15T15:21:15.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.581+0000 7f8785837640 1 -- 172.21.15.105:0/940803283 shutdown_connections 2024-04-15T15:21:15.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.581+0000 7f8785837640 1 --2- 172.21.15.105:0/940803283 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f87800d1910 0x7f8780151cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:15.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.581+0000 7f8785837640 1 --2- 172.21.15.105:0/940803283 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f876c06c980 0x7f876c06ee40 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:15.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.581+0000 7f8785837640 1 --2- 172.21.15.105:0/940803283 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8780161df0 0x7f87800cc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:15.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.581+0000 7f8785837640 1 --2- 172.21.15.105:0/940803283 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87800b9e80 0x7f87801f4630 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:15.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.581+0000 7f8785837640 1 -- 172.21.15.105:0/940803283 >> 172.21.15.105:0/940803283 conn(0x7f8780067720 msgr2=0x7f8780160f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:15.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.581+0000 7f8785837640 1 -- 172.21.15.105:0/940803283 shutdown_connections 2024-04-15T15:21:15.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.581+0000 7f8785837640 1 -- 172.21.15.105:0/940803283 wait complete. 2024-04-15T15:21:15.594 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-15T15:21:15.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.698+0000 7f9fe8f0e640 1 Processor -- start 2024-04-15T15:21:15.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.698+0000 7f9fe8f0e640 1 -- start start 2024-04-15T15:21:15.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.699+0000 7f9fe8f0e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9fe4144c90 0x7f9fe40b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:15.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.699+0000 7f9fe8f0e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9fe40cd8e0 0x7f9fe406f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:15.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.699+0000 7f9fe8f0e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fe40b40a0 con 0x7f9fe406ef90 2024-04-15T15:21:15.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.699+0000 7f9fe8f0e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f9fe40b3e50 con 0x7f9fe40cd8e0 2024-04-15T15:21:15.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.699+0000 7f9fe8f0e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f9fe406f910 con 0x7f9fe4144c90 2024-04-15T15:21:15.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.699+0000 7f9fe259c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9fe4144c90 0x7f9fe40b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:15.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.699+0000 7f9fe2d9d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9fe40cd8e0 0x7f9fe406f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:15.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.699+0000 7f9fe259c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9fe4144c90 0x7f9fe40b4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.105:46032/0 (socket says 172.21.15.105:46032) 2024-04-15T15:21:15.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.699+0000 7f9fe259c640 1 -- 172.21.15.105:0/2251714464 learned_addr learned my addr 172.21.15.105:0/2251714464 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:15.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.699+0000 7f9fe1d9b640 1 --1- 172.21.15.105:0/2251714464 >> v1:172.21.15.154:6789/0 conn(0x7f9fe406ef90 0x7f9fe40cd1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.105:36304/0 (socket says 172.21.15.105:36304) 2024-04-15T15:21:15.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.700+0000 7f9fe2d9d640 1 -- 172.21.15.105:0/2251714464 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9fe4144c90 msgr2=0x7f9fe40b4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:15.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.700+0000 7f9fe2d9d640 1 --2- 172.21.15.105:0/2251714464 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9fe4144c90 0x7f9fe40b4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:15.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.700+0000 7f9fe2d9d640 1 -- 172.21.15.105:0/2251714464 >> v1:172.21.15.154:6789/0 conn(0x7f9fe406ef90 legacy=0x7f9fe40cd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:15.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.700+0000 7f9fe2d9d640 1 -- 172.21.15.105:0/2251714464 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fe40b4380 con 0x7f9fe40cd8e0 2024-04-15T15:21:15.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.701+0000 7f9fe2d9d640 1 --2- 172.21.15.105:0/2251714464 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9fe40cd8e0 0x7f9fe406f3d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc0097a0 tx=0x7f9fdc02b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b986494f586b69f server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:15.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.701+0000 7f9fe159a640 1 -- 172.21.15.105:0/2251714464 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc0315a0 con 0x7f9fe40cd8e0 2024-04-15T15:21:15.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.701+0000 7f9fe159a640 1 -- 172.21.15.105:0/2251714464 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fdc037070 con 0x7f9fe40cd8e0 2024-04-15T15:21:15.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.701+0000 7f9fe159a640 1 -- 172.21.15.105:0/2251714464 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc031960 con 0x7f9fe40cd8e0 2024-04-15T15:21:15.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.701+0000 7f9fe8f0e640 1 -- 172.21.15.105:0/2251714464 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9fe40cd8e0 msgr2=0x7f9fe406f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:15.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.701+0000 7f9fe8f0e640 1 --2- 172.21.15.105:0/2251714464 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9fe40cd8e0 0x7f9fe406f3d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc0097a0 tx=0x7f9fdc02b840 comp rx=0 tx=0).stop 2024-04-15T15:21:15.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.701+0000 7f9fe8f0e640 1 -- 172.21.15.105:0/2251714464 shutdown_connections 2024-04-15T15:21:15.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.701+0000 7f9fe8f0e640 1 --2- 172.21.15.105:0/2251714464 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9fe40cd8e0 0x7f9fe406f3d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:15.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.701+0000 7f9fe8f0e640 1 --2- 172.21.15.105:0/2251714464 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9fe4144c90 0x7f9fe40b4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:15.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.701+0000 7f9fe8f0e640 1 -- 172.21.15.105:0/2251714464 >> 172.21.15.105:0/2251714464 conn(0x7f9fe4067d50 msgr2=0x7f9fe406b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:15.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.701+0000 7f9fe8f0e640 1 -- 172.21.15.105:0/2251714464 shutdown_connections 2024-04-15T15:21:15.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.701+0000 7f9fe8f0e640 1 -- 172.21.15.105:0/2251714464 wait complete. 2024-04-15T15:21:15.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.703+0000 7f9fe8f0e640 1 Processor -- start 2024-04-15T15:21:15.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.703+0000 7f9fe8f0e640 1 -- start start 2024-04-15T15:21:15.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.703+0000 7f9fe8f0e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fe406ef90 0x7f9fe40af280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:15.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.703+0000 7f9fe8f0e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9fe40cd8e0 0x7f9fe414ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:15.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.703+0000 7f9fe8f0e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9fe4144c90 0x7f9fe414f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:15.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.703+0000 7f9fe8f0e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe4145070 con 0x7f9fe406ef90 2024-04-15T15:21:15.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.703+0000 7f9fe8f0e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe40b4940 con 0x7f9fe4144c90 2024-04-15T15:21:15.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.703+0000 7f9fe8f0e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f9fe41451f0 con 0x7f9fe40cd8e0 2024-04-15T15:21:15.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.704+0000 7f9fe259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fe406ef90 0x7f9fe40af280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:15.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.704+0000 7f9fe259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fe406ef90 0x7f9fe40af280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33324/0 (socket says 172.21.15.105:33324) 2024-04-15T15:21:15.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.704+0000 7f9fe259c640 1 -- 172.21.15.105:0/3081198382 learned_addr learned my addr 172.21.15.105:0/3081198382 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:15.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.704+0000 7f9fe1d9b640 1 --2- 172.21.15.105:0/3081198382 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9fe40cd8e0 0x7f9fe414ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:15.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.704+0000 7f9fe2d9d640 1 --2- 172.21.15.105:0/3081198382 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9fe4144c90 0x7f9fe414f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:15.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.704+0000 7f9fe259c640 1 -- 172.21.15.105:0/3081198382 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9fe40cd8e0 msgr2=0x7f9fe414ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:15.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.704+0000 7f9fe259c640 1 --2- 172.21.15.105:0/3081198382 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9fe40cd8e0 0x7f9fe414ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:15.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.704+0000 7f9fe259c640 1 -- 172.21.15.105:0/3081198382 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9fe4144c90 msgr2=0x7f9fe414f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:15.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.704+0000 7f9fe259c640 1 --2- 172.21.15.105:0/3081198382 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9fe4144c90 0x7f9fe414f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:15.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.704+0000 7f9fe259c640 1 -- 172.21.15.105:0/3081198382 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fe40af7c0 con 0x7f9fe406ef90 2024-04-15T15:21:15.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.704+0000 7f9fe259c640 1 --2- 172.21.15.105:0/3081198382 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fe406ef90 0x7f9fe40af280 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f9fd400efc0 tx=0x7f9fd400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:15.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.704+0000 7f9fd2ffd640 1 -- 172.21.15.105:0/3081198382 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd400cda0 con 0x7f9fe406ef90 2024-04-15T15:21:15.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.704+0000 7f9fd2ffd640 1 -- 172.21.15.105:0/3081198382 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fd4019070 con 0x7f9fe406ef90 2024-04-15T15:21:15.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.704+0000 7f9fe8f0e640 1 -- 172.21.15.105:0/3081198382 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fe414cd80 con 0x7f9fe406ef90 2024-04-15T15:21:15.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.704+0000 7f9fd2ffd640 1 -- 172.21.15.105:0/3081198382 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd40146a0 con 0x7f9fe406ef90 2024-04-15T15:21:15.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.704+0000 7f9fe8f0e640 1 -- 172.21.15.105:0/3081198382 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fe414d340 con 0x7f9fe406ef90 2024-04-15T15:21:15.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.705+0000 7f9fe8f0e640 1 -- 172.21.15.105:0/3081198382 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fe4068ab0 con 0x7f9fe406ef90 2024-04-15T15:21:15.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.706+0000 7f9fd2ffd640 1 -- 172.21.15.105:0/3081198382 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f9fd4005d10 con 0x7f9fe406ef90 2024-04-15T15:21:15.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.706+0000 7f9fd2ffd640 1 --2- 172.21.15.105:0/3081198382 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f9fc406c980 0x7f9fc406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:15.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.706+0000 7f9fd2ffd640 1 -- 172.21.15.105:0/3081198382 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5795+0+0 (secure 0 0 0) 0x7f9fd408c2a0 con 0x7f9fe406ef90 2024-04-15T15:21:15.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.706+0000 7f9fe1d9b640 1 --2- 172.21.15.105:0/3081198382 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f9fc406c980 0x7f9fc406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:15.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.707+0000 7f9fe1d9b640 1 --2- 172.21.15.105:0/3081198382 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f9fc406c980 0x7f9fc406ee40 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9fe415b790 tx=0x7f9fd8007cc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:15.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.712+0000 7f9fd2ffd640 1 -- 172.21.15.105:0/3081198382 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9fd4010040 con 0x7f9fe406ef90 2024-04-15T15:21:15.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.887+0000 7f9fe8f0e640 1 -- 172.21.15.105:0/3081198382 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9fe406ce40 con 0x7f9fe406ef90 2024-04-15T15:21:15.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.888+0000 7f9fd2ffd640 1 -- 172.21.15.105:0/3081198382 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v35) ==== 100+0+11 (secure 0 0 0) 0x7f9fd405a4f0 con 0x7f9fe406ef90 2024-04-15T15:21:15.890 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-04-15T15:21:15.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.890+0000 7f9fe8f0e640 1 -- 172.21.15.105:0/3081198382 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f9fc406c980 msgr2=0x7f9fc406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:15.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.890+0000 7f9fe8f0e640 1 --2- 172.21.15.105:0/3081198382 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f9fc406c980 0x7f9fc406ee40 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9fe415b790 tx=0x7f9fd8007cc0 comp rx=0 tx=0).stop 2024-04-15T15:21:15.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.890+0000 7f9fe8f0e640 1 -- 172.21.15.105:0/3081198382 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fe406ef90 msgr2=0x7f9fe40af280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:15.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.890+0000 7f9fe8f0e640 1 --2- 172.21.15.105:0/3081198382 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fe406ef90 0x7f9fe40af280 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f9fd400efc0 tx=0x7f9fd400c490 comp rx=0 tx=0).stop 2024-04-15T15:21:15.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.890+0000 7f9fe8f0e640 1 -- 172.21.15.105:0/3081198382 shutdown_connections 2024-04-15T15:21:15.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.890+0000 7f9fe8f0e640 1 --2- 172.21.15.105:0/3081198382 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9fe40cd8e0 0x7f9fe414ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:15.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.890+0000 7f9fe8f0e640 1 --2- 172.21.15.105:0/3081198382 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f9fc406c980 0x7f9fc406ee40 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:15.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.890+0000 7f9fe8f0e640 1 --2- 172.21.15.105:0/3081198382 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9fe4144c90 0x7f9fe414f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:15.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.890+0000 7f9fe8f0e640 1 --2- 172.21.15.105:0/3081198382 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fe406ef90 0x7f9fe40af280 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:15.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.890+0000 7f9fe8f0e640 1 -- 172.21.15.105:0/3081198382 >> 172.21.15.105:0/3081198382 conn(0x7f9fe4067d50 msgr2=0x7f9fe40b21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:15.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.890+0000 7f9fe8f0e640 1 -- 172.21.15.105:0/3081198382 shutdown_connections 2024-04-15T15:21:15.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:15.890+0000 7f9fe8f0e640 1 -- 172.21.15.105:0/3081198382 wait complete. 2024-04-15T15:21:15.903 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-15T15:21:16.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.004+0000 7fd25709a640 1 Processor -- start 2024-04-15T15:21:16.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.005+0000 7fd25709a640 1 -- start start 2024-04-15T15:21:16.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.005+0000 7fd25709a640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd250146380 0x7fd2500afbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:16.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.005+0000 7fd25709a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd25006c0d0 0x7fd2500ced50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:16.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.005+0000 7fd25709a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2500cf3a0 con 0x7fd2500ce8e0 2024-04-15T15:21:16.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.005+0000 7fd25709a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fd2500cffb0 con 0x7fd25006c0d0 2024-04-15T15:21:16.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.005+0000 7fd25709a640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fd2500cf680 con 0x7fd250146380 2024-04-15T15:21:16.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.006+0000 7fd255637640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd25006c0d0 0x7fd2500ced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:16.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.006+0000 7fd24ffff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fd2500ce8e0 0x7fd25006b9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:37570/0 (socket says 172.21.15.105:37570) 2024-04-15T15:21:16.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.006+0000 7fd255637640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd25006c0d0 0x7fd2500ced50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33336/0 (socket says 172.21.15.105:33336) 2024-04-15T15:21:16.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.006+0000 7fd24ffff640 1 -- 172.21.15.105:0/381294626 learned_addr learned my addr 172.21.15.105:0/381294626 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:21:16.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.006+0000 7fd254e36640 1 --2- 172.21.15.105:0/381294626 >> v2:172.21.15.154:3300/0 conn(0x7fd250146380 0x7fd2500afbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:16.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.006+0000 7fd255637640 1 -- 172.21.15.105:0/381294626 >> v2:172.21.15.154:3300/0 conn(0x7fd250146380 msgr2=0x7fd2500afbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:16.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.006+0000 7fd255637640 1 --2- 172.21.15.105:0/381294626 >> v2:172.21.15.154:3300/0 conn(0x7fd250146380 0x7fd2500afbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:16.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.006+0000 7fd255637640 1 -- 172.21.15.105:0/381294626 >> v1:172.21.15.105:6789/0 conn(0x7fd2500ce8e0 legacy=0x7fd25006b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:16.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.006+0000 7fd255637640 1 -- 172.21.15.105:0/381294626 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2500cf960 con 0x7fd25006c0d0 2024-04-15T15:21:16.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.006+0000 7fd24f7fe640 1 -- 172.21.15.105:0/381294626 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2397660698 0 0) 0x7fd2500cf3a0 con 0x7fd2500ce8e0 2024-04-15T15:21:16.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.007+0000 7fd255637640 1 --2- 172.21.15.105:0/381294626 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd25006c0d0 0x7fd2500ced50 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fd25006e740 tx=0x7fd24802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=178ce482e7a970ad server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:16.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.007+0000 7fd24f7fe640 1 -- 172.21.15.105:0/381294626 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd248039070 con 0x7fd25006c0d0 2024-04-15T15:21:16.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.007+0000 7fd24f7fe640 1 -- 172.21.15.105:0/381294626 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd24802cbf0 con 0x7fd25006c0d0 2024-04-15T15:21:16.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.007+0000 7fd24f7fe640 1 -- 172.21.15.105:0/381294626 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd248034440 con 0x7fd25006c0d0 2024-04-15T15:21:16.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.007+0000 7fd25709a640 1 -- 172.21.15.105:0/381294626 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd25006c0d0 msgr2=0x7fd2500ced50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:16.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.007+0000 7fd25709a640 1 --2- 172.21.15.105:0/381294626 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd25006c0d0 0x7fd2500ced50 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fd25006e740 tx=0x7fd24802be40 comp rx=0 tx=0).stop 2024-04-15T15:21:16.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.007+0000 7fd25709a640 1 -- 172.21.15.105:0/381294626 shutdown_connections 2024-04-15T15:21:16.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.007+0000 7fd25709a640 1 --2- 172.21.15.105:0/381294626 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd25006c0d0 0x7fd2500ced50 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:16.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.007+0000 7fd25709a640 1 --2- 172.21.15.105:0/381294626 >> v2:172.21.15.154:3300/0 conn(0x7fd250146380 0x7fd2500afbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:16.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.007+0000 7fd25709a640 1 -- 172.21.15.105:0/381294626 >> 172.21.15.105:0/381294626 conn(0x7fd25009aa40 msgr2=0x7fd2500b6950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:16.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.007+0000 7fd25709a640 1 -- 172.21.15.105:0/381294626 shutdown_connections 2024-04-15T15:21:16.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.007+0000 7fd25709a640 1 -- 172.21.15.105:0/381294626 wait complete. 2024-04-15T15:21:16.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.008+0000 7fd25709a640 1 Processor -- start 2024-04-15T15:21:16.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.009+0000 7fd25709a640 1 -- start start 2024-04-15T15:21:16.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.009+0000 7fd25709a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd25006c0d0 0x7fd25015c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:16.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.009+0000 7fd25709a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd2500ce8e0 0x7fd250144130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:16.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.009+0000 7fd25709a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd250146380 0x7fd250144670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:16.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.009+0000 7fd25709a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd2500b7290 con 0x7fd25006c0d0 2024-04-15T15:21:16.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.009+0000 7fd25709a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd2500b0130 con 0x7fd250146380 2024-04-15T15:21:16.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.009+0000 7fd25709a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fd2500d0440 con 0x7fd2500ce8e0 2024-04-15T15:21:16.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.009+0000 7fd254e36640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd25006c0d0 0x7fd25015c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:16.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.010+0000 7fd254e36640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd25006c0d0 0x7fd25015c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33346/0 (socket says 172.21.15.105:33346) 2024-04-15T15:21:16.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.010+0000 7fd254e36640 1 -- 172.21.15.105:0/898305622 learned_addr learned my addr 172.21.15.105:0/898305622 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:16.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.010+0000 7fd255637640 1 --2- 172.21.15.105:0/898305622 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd250146380 0x7fd250144670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:16.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.010+0000 7fd254e36640 1 -- 172.21.15.105:0/898305622 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd2500ce8e0 msgr2=0x7fd250144130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:16.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.010+0000 7fd254e36640 1 --2- 172.21.15.105:0/898305622 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd2500ce8e0 0x7fd250144130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:16.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.010+0000 7fd254e36640 1 -- 172.21.15.105:0/898305622 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd250146380 msgr2=0x7fd250144670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:16.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.010+0000 7fd254e36640 1 --2- 172.21.15.105:0/898305622 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd250146380 0x7fd250144670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:16.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.010+0000 7fd254e36640 1 -- 172.21.15.105:0/898305622 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd25015c6d0 con 0x7fd25006c0d0 2024-04-15T15:21:16.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.010+0000 7fd255637640 1 --2- 172.21.15.105:0/898305622 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd250146380 0x7fd250144670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:21:16.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.010+0000 7fd254e36640 1 --2- 172.21.15.105:0/898305622 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd25006c0d0 0x7fd25015c190 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fd24400d5f0 tx=0x7fd24400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:16.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.010+0000 7fd24d7fa640 1 -- 172.21.15.105:0/898305622 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2440125f0 con 0x7fd25006c0d0 2024-04-15T15:21:16.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.010+0000 7fd24d7fa640 1 -- 172.21.15.105:0/898305622 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd244019070 con 0x7fd25006c0d0 2024-04-15T15:21:16.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.010+0000 7fd24d7fa640 1 -- 172.21.15.105:0/898305622 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2440129d0 con 0x7fd25006c0d0 2024-04-15T15:21:16.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.010+0000 7fd25709a640 1 -- 172.21.15.105:0/898305622 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2500cbef0 con 0x7fd25006c0d0 2024-04-15T15:21:16.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.010+0000 7fd25709a640 1 -- 172.21.15.105:0/898305622 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2500cc4b0 con 0x7fd25006c0d0 2024-04-15T15:21:16.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.011+0000 7fd25709a640 1 -- 172.21.15.105:0/898305622 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd250069150 con 0x7fd25006c0d0 2024-04-15T15:21:16.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.012+0000 7fd24d7fa640 1 -- 172.21.15.105:0/898305622 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fd244012b70 con 0x7fd25006c0d0 2024-04-15T15:21:16.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.012+0000 7fd24d7fa640 1 --2- 172.21.15.105:0/898305622 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fd23406c980 0x7fd23406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:16.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.012+0000 7fd24ffff640 1 --2- 172.21.15.105:0/898305622 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fd23406c980 0x7fd23406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:16.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.012+0000 7fd24d7fa640 1 -- 172.21.15.105:0/898305622 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5795+0+0 (secure 0 0 0) 0x7fd24408bff0 con 0x7fd25006c0d0 2024-04-15T15:21:16.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.012+0000 7fd24ffff640 1 --2- 172.21.15.105:0/898305622 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fd23406c980 0x7fd23406ee40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd23c007fd0 tx=0x7fd23c007900 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:16.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.016+0000 7fd24d7fa640 1 -- 172.21.15.105:0/898305622 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd24405a3d0 con 0x7fd25006c0d0 2024-04-15T15:21:16.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.201+0000 7fd25709a640 1 -- 172.21.15.105:0/898305622 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd2500afef0 con 0x7fd25006c0d0 2024-04-15T15:21:16.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.202+0000 7fd24d7fa640 1 -- 172.21.15.105:0/898305622 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v35) ==== 80+0+122 (secure 0 0 0) 0x7fd244059cd0 con 0x7fd25006c0d0 2024-04-15T15:21:16.204 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.105:6837/958066041 2024-04-16T15:21:03.660841+0000 2024-04-15T15:21:16.204 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.105:6835/958066041 2024-04-16T15:21:03.660841+0000 2024-04-15T15:21:16.204 INFO:teuthology.orchestra.run.smithi105.stderr:listed 2 entries 2024-04-15T15:21:16.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.204+0000 7fd25709a640 1 -- 172.21.15.105:0/898305622 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fd23406c980 msgr2=0x7fd23406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:16.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.204+0000 7fd25709a640 1 --2- 172.21.15.105:0/898305622 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fd23406c980 0x7fd23406ee40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd23c007fd0 tx=0x7fd23c007900 comp rx=0 tx=0).stop 2024-04-15T15:21:16.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.204+0000 7fd25709a640 1 -- 172.21.15.105:0/898305622 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd25006c0d0 msgr2=0x7fd25015c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:16.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.204+0000 7fd25709a640 1 --2- 172.21.15.105:0/898305622 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd25006c0d0 0x7fd25015c190 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fd24400d5f0 tx=0x7fd24400dab0 comp rx=0 tx=0).stop 2024-04-15T15:21:16.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.204+0000 7fd25709a640 1 -- 172.21.15.105:0/898305622 shutdown_connections 2024-04-15T15:21:16.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.204+0000 7fd25709a640 1 --2- 172.21.15.105:0/898305622 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd2500ce8e0 0x7fd250144130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:16.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.204+0000 7fd25709a640 1 --2- 172.21.15.105:0/898305622 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fd23406c980 0x7fd23406ee40 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:16.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.204+0000 7fd25709a640 1 --2- 172.21.15.105:0/898305622 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd250146380 0x7fd250144670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:16.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.204+0000 7fd25709a640 1 --2- 172.21.15.105:0/898305622 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd25006c0d0 0x7fd25015c190 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:16.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.204+0000 7fd25709a640 1 -- 172.21.15.105:0/898305622 >> 172.21.15.105:0/898305622 conn(0x7fd25009aa40 msgr2=0x7fd250074a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:16.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.204+0000 7fd25709a640 1 -- 172.21.15.105:0/898305622 shutdown_connections 2024-04-15T15:21:16.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.204+0000 7fd25709a640 1 -- 172.21.15.105:0/898305622 wait complete. 2024-04-15T15:21:16.217 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi154.front.sepia.ceph.com... 2024-04-15T15:21:16.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:21:16.217 DEBUG:teuthology.orchestra.run.smithi154:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-04-15T15:21:16.250 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi154.stderr:ceph-fuse[45956]: fuse finished with error 0 and tester_r 0 2024-04-15T15:21:16.251 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi154.front.sepia.ceph.com 2024-04-15T15:21:16.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:21:16.252 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-15T15:21:16.422 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-15T15:21:16.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.522+0000 7fe3c361b640 1 Processor -- start 2024-04-15T15:21:16.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.523+0000 7fe3c361b640 1 -- start start 2024-04-15T15:21:16.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.524+0000 7fe3c361b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe3bc13e680 0x7fe3bc0cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:16.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.524+0000 7fe3c361b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3bc06f6a0 0x7fe3bc15f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:16.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.524+0000 7fe3c361b640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3bc15bc90 con 0x7fe3bc13ec60 2024-04-15T15:21:16.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.524+0000 7fe3c361b640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fe3bc15bf70 con 0x7fe3bc13e680 2024-04-15T15:21:16.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.524+0000 7fe3c361b640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fe3bc15c250 con 0x7fe3bc06f6a0 2024-04-15T15:21:16.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.524+0000 7fe3c13b7640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe3bc13e680 0x7fe3bc0cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:16.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.524+0000 7fe3c0bb6640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fe3bc13ec60 0x7fe3bc06ef90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:37584/0 (socket says 172.21.15.105:37584) 2024-04-15T15:21:16.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.524+0000 7fe3c0bb6640 1 -- 172.21.15.105:0/893443232 learned_addr learned my addr 172.21.15.105:0/893443232 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:21:16.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.524+0000 7fe3c13b7640 1 -- 172.21.15.105:0/893443232 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3bc06f6a0 msgr2=0x7fe3bc15f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:16.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.524+0000 7fe3c13b7640 1 --2- 172.21.15.105:0/893443232 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3bc06f6a0 0x7fe3bc15f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:16.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.524+0000 7fe3c13b7640 1 -- 172.21.15.105:0/893443232 >> v1:172.21.15.105:6789/0 conn(0x7fe3bc13ec60 legacy=0x7fe3bc06ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:16.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.524+0000 7fe3affff640 1 -- 172.21.15.105:0/893443232 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2763338449 0 0) 0x7fe3bc15bc90 con 0x7fe3bc13ec60 2024-04-15T15:21:16.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.524+0000 7fe3c13b7640 1 -- 172.21.15.105:0/893443232 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3bc15c530 con 0x7fe3bc13e680 2024-04-15T15:21:16.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.525+0000 7fe3c13b7640 1 --2- 172.21.15.105:0/893443232 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3bc13e680 0x7fe3bc0cfb70 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fe3b00097a0 tx=0x7fe3b002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b42fc3a28714337 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:16.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.525+0000 7fe3affff640 1 -- 172.21.15.105:0/893443232 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b0035020 con 0x7fe3bc13e680 2024-04-15T15:21:16.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.525+0000 7fe3affff640 1 -- 172.21.15.105:0/893443232 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3b0031830 con 0x7fe3bc13e680 2024-04-15T15:21:16.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.525+0000 7fe3affff640 1 -- 172.21.15.105:0/893443232 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b0031c00 con 0x7fe3bc13e680 2024-04-15T15:21:16.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.525+0000 7fe3c361b640 1 -- 172.21.15.105:0/893443232 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3bc13e680 msgr2=0x7fe3bc0cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:16.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.525+0000 7fe3c361b640 1 --2- 172.21.15.105:0/893443232 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3bc13e680 0x7fe3bc0cfb70 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fe3b00097a0 tx=0x7fe3b002b8d0 comp rx=0 tx=0).stop 2024-04-15T15:21:16.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.525+0000 7fe3c361b640 1 -- 172.21.15.105:0/893443232 shutdown_connections 2024-04-15T15:21:16.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.525+0000 7fe3c361b640 1 --2- 172.21.15.105:0/893443232 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3bc06f6a0 0x7fe3bc15f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:16.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.525+0000 7fe3c361b640 1 --2- 172.21.15.105:0/893443232 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3bc13e680 0x7fe3bc0cfb70 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:16.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.525+0000 7fe3c361b640 1 -- 172.21.15.105:0/893443232 >> 172.21.15.105:0/893443232 conn(0x7fe3bc067d50 msgr2=0x7fe3bc06b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:16.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.526+0000 7fe3c361b640 1 -- 172.21.15.105:0/893443232 shutdown_connections 2024-04-15T15:21:16.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.526+0000 7fe3c361b640 1 -- 172.21.15.105:0/893443232 wait complete. 2024-04-15T15:21:16.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.527+0000 7fe3c361b640 1 Processor -- start 2024-04-15T15:21:16.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.527+0000 7fe3c361b640 1 -- start start 2024-04-15T15:21:16.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.527+0000 7fe3c361b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3bc06f6a0 0x7fe3bc155320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:16.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.527+0000 7fe3c361b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3bc13e680 0x7fe3bc0b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:16.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.527+0000 7fe3c361b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3bc13ec60 0x7fe3bc158320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:16.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.527+0000 7fe3c361b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe3bc0d0230 con 0x7fe3bc13ec60 2024-04-15T15:21:16.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.527+0000 7fe3c361b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe3bc0d00b0 con 0x7fe3bc06f6a0 2024-04-15T15:21:16.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.527+0000 7fe3c361b640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fe3bc0cf590 con 0x7fe3bc13e680 2024-04-15T15:21:16.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.528+0000 7fe3c1bb8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3bc13ec60 0x7fe3bc158320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:16.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.528+0000 7fe3c1bb8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3bc13ec60 0x7fe3bc158320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33362/0 (socket says 172.21.15.105:33362) 2024-04-15T15:21:16.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.528+0000 7fe3c1bb8640 1 -- 172.21.15.105:0/2958983199 learned_addr learned my addr 172.21.15.105:0/2958983199 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:16.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.528+0000 7fe3c0bb6640 1 --2- 172.21.15.105:0/2958983199 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3bc13e680 0x7fe3bc0b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:16.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.528+0000 7fe3c13b7640 1 --2- 172.21.15.105:0/2958983199 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3bc06f6a0 0x7fe3bc155320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:16.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.528+0000 7fe3c1bb8640 1 -- 172.21.15.105:0/2958983199 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3bc13e680 msgr2=0x7fe3bc0b2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:16.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.528+0000 7fe3c1bb8640 1 --2- 172.21.15.105:0/2958983199 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3bc13e680 0x7fe3bc0b2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:16.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.528+0000 7fe3c1bb8640 1 -- 172.21.15.105:0/2958983199 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3bc06f6a0 msgr2=0x7fe3bc155320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:16.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.528+0000 7fe3c1bb8640 1 --2- 172.21.15.105:0/2958983199 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3bc06f6a0 0x7fe3bc155320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:16.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.528+0000 7fe3c1bb8640 1 -- 172.21.15.105:0/2958983199 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3bc13cfb0 con 0x7fe3bc13ec60 2024-04-15T15:21:16.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.528+0000 7fe3c13b7640 1 --2- 172.21.15.105:0/2958983199 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3bc06f6a0 0x7fe3bc155320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:21:16.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.528+0000 7fe3c1bb8640 1 --2- 172.21.15.105:0/2958983199 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3bc13ec60 0x7fe3bc158320 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fe3b400ba10 tx=0x7fe3b400bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:16.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.528+0000 7fe3adffb640 1 -- 172.21.15.105:0/2958983199 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b4019070 con 0x7fe3bc13ec60 2024-04-15T15:21:16.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.528+0000 7fe3adffb640 1 -- 172.21.15.105:0/2958983199 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3b400cb20 con 0x7fe3bc13ec60 2024-04-15T15:21:16.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.528+0000 7fe3c361b640 1 -- 172.21.15.105:0/2958983199 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3bc156f10 con 0x7fe3bc13ec60 2024-04-15T15:21:16.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.528+0000 7fe3adffb640 1 -- 172.21.15.105:0/2958983199 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b4014440 con 0x7fe3bc13ec60 2024-04-15T15:21:16.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.528+0000 7fe3c361b640 1 -- 172.21.15.105:0/2958983199 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3bc1574d0 con 0x7fe3bc13ec60 2024-04-15T15:21:16.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.529+0000 7fe3c361b640 1 -- 172.21.15.105:0/2958983199 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3bc0b0360 con 0x7fe3bc13ec60 2024-04-15T15:21:16.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.530+0000 7fe3adffb640 1 -- 172.21.15.105:0/2958983199 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fe3b401c070 con 0x7fe3bc13ec60 2024-04-15T15:21:16.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.530+0000 7fe3adffb640 1 --2- 172.21.15.105:0/2958983199 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fe3a006c980 0x7fe3a006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:16.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.530+0000 7fe3adffb640 1 -- 172.21.15.105:0/2958983199 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5795+0+0 (secure 0 0 0) 0x7fe3b408b7f0 con 0x7fe3bc13ec60 2024-04-15T15:21:16.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.530+0000 7fe3c13b7640 1 --2- 172.21.15.105:0/2958983199 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fe3a006c980 0x7fe3a006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:16.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.531+0000 7fe3c13b7640 1 --2- 172.21.15.105:0/2958983199 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fe3a006c980 0x7fe3a006ee40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe3b0036000 tx=0x7fe3b0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:16.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.535+0000 7fe3adffb640 1 -- 172.21.15.105:0/2958983199 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe3bc13cfb0 con 0x7fe3bc13ec60 2024-04-15T15:21:16.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.751+0000 7fe3c361b640 1 -- 172.21.15.105:0/2958983199 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe3bc0b4910 con 0x7fe3bc13ec60 2024-04-15T15:21:16.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.752+0000 7fe3adffb640 1 -- 172.21.15.105:0/2958983199 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+5184 (secure 0 0 0) 0x7fe3b4059c60 con 0x7fe3bc13ec60 2024-04-15T15:21:16.754 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:21:16.754 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4778,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/4848508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":4848508},{"type":"v1","addr":"172.21.15.154:6835","nonce":4848508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5005,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/1127070021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1127070021},{"type":"v1","addr":"172.21.15.154:6833","nonce":1127070021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5193,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6836/3825669416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3825669416},{"type":"v1","addr":"172.21.15.105:6836","nonce":3825669416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-15T15:20:29.000616+0000","modified":"2024-04-15T15:21:07.677485+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5712":{"gid":5712,"name":"a","rank":0,"incarnation":17,"state":"up:active","state_seq":4,"addr":"172.21.15.105:6837/4212477415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":4212477415},{"type":"v1","addr":"172.21.15.105:6837","nonce":4212477415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5712,"qdb_cluster":[5712]},"id":2}]} 2024-04-15T15:21:16.754 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 20 2024-04-15T15:21:16.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.754+0000 7fe3c361b640 1 -- 172.21.15.105:0/2958983199 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fe3a006c980 msgr2=0x7fe3a006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:16.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.754+0000 7fe3c361b640 1 --2- 172.21.15.105:0/2958983199 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fe3a006c980 0x7fe3a006ee40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe3b0036000 tx=0x7fe3b0008030 comp rx=0 tx=0).stop 2024-04-15T15:21:16.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.754+0000 7fe3c361b640 1 -- 172.21.15.105:0/2958983199 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3bc13ec60 msgr2=0x7fe3bc158320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:16.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.754+0000 7fe3c361b640 1 --2- 172.21.15.105:0/2958983199 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3bc13ec60 0x7fe3bc158320 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fe3b400ba10 tx=0x7fe3b400bed0 comp rx=0 tx=0).stop 2024-04-15T15:21:16.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.754+0000 7fe3c361b640 1 -- 172.21.15.105:0/2958983199 shutdown_connections 2024-04-15T15:21:16.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.754+0000 7fe3c361b640 1 --2- 172.21.15.105:0/2958983199 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3bc13e680 0x7fe3bc0b2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:16.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.754+0000 7fe3c361b640 1 --2- 172.21.15.105:0/2958983199 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fe3a006c980 0x7fe3a006ee40 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:16.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.754+0000 7fe3c361b640 1 --2- 172.21.15.105:0/2958983199 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3bc13ec60 0x7fe3bc158320 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:16.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.754+0000 7fe3c361b640 1 --2- 172.21.15.105:0/2958983199 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3bc06f6a0 0x7fe3bc155320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:16.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.754+0000 7fe3c361b640 1 -- 172.21.15.105:0/2958983199 >> 172.21.15.105:0/2958983199 conn(0x7fe3bc067d50 msgr2=0x7fe3bc1409a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:16.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.754+0000 7fe3c361b640 1 -- 172.21.15.105:0/2958983199 shutdown_connections 2024-04-15T15:21:16.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.754+0000 7fe3c361b640 1 -- 172.21.15.105:0/2958983199 wait complete. 2024-04-15T15:21:16.767 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:21:16.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.868+0000 7ff13698f640 1 Processor -- start 2024-04-15T15:21:16.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.868+0000 7ff13698f640 1 -- start start 2024-04-15T15:21:16.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.869+0000 7ff13698f640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff1300d18e0 0x7ff130162630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:16.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.869+0000 7ff13698f640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff13015ef40 con 0x7ff13015eb60 2024-04-15T15:21:16.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.869+0000 7ff13698f640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff130163260 con 0x7ff130162b70 2024-04-15T15:21:16.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.869+0000 7ff13698f640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7ff130163540 con 0x7ff1300d18e0 2024-04-15T15:21:16.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.869+0000 7ff12ffff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7ff13015eb60 0x7ff1300d11d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:37592/0 (socket says 172.21.15.105:37592) 2024-04-15T15:21:16.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.869+0000 7ff12ffff640 1 -- 172.21.15.105:0/3749112385 learned_addr learned my addr 172.21.15.105:0/3749112385 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:21:16.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.870+0000 7ff12effd640 1 -- 172.21.15.105:0/3749112385 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2877182788 0 0) 0x7ff13015ef40 con 0x7ff13015eb60 2024-04-15T15:21:16.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.870+0000 7ff12effd640 1 -- 172.21.15.105:0/3749112385 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff118003520 con 0x7ff13015eb60 2024-04-15T15:21:16.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.870+0000 7ff12f7fe640 1 --2- 172.21.15.105:0/3749112385 >> v2:172.21.15.154:3300/0 conn(0x7ff1300d18e0 0x7ff130162630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:16.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.870+0000 7ff12effd640 1 -- 172.21.15.105:0/3749112385 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2832160024 0 0) 0x7ff118003520 con 0x7ff13015eb60 2024-04-15T15:21:16.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.870+0000 7ff12effd640 1 -- 172.21.15.105:0/3749112385 >> v2:172.21.15.154:3300/0 conn(0x7ff1300d18e0 msgr2=0x7ff130162630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:16.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.870+0000 7ff12effd640 1 --2- 172.21.15.105:0/3749112385 >> v2:172.21.15.154:3300/0 conn(0x7ff1300d18e0 0x7ff130162630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:16.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.870+0000 7ff12effd640 1 -- 172.21.15.105:0/3749112385 >> v1:172.21.15.154:6789/0 conn(0x7ff130162b70 legacy=0x7ff1300d3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:16.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.870+0000 7ff12effd640 1 -- 172.21.15.105:0/3749112385 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1300d2970 con 0x7ff13015eb60 2024-04-15T15:21:16.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.870+0000 7ff12effd640 1 -- 172.21.15.105:0/3749112385 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7ff120001670 con 0x7ff13015eb60 2024-04-15T15:21:16.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.870+0000 7ff12effd640 1 -- 172.21.15.105:0/3749112385 >> v1:172.21.15.105:6789/0 conn(0x7ff13015eb60 legacy=0x7ff1300d11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:16.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.870+0000 7ff12effd640 1 --2- 172.21.15.105:0/3749112385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1180039b0 0x7ff118003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:16.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.870+0000 7ff12effd640 1 --2- 172.21.15.105:0/3749112385 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff118004800 0x7ff118004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:16.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.870+0000 7ff12effd640 1 --2- 172.21.15.105:0/3749112385 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff1180051c0 0x7ff11800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:16.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.870+0000 7ff12effd640 1 -- 172.21.15.105:0/3749112385 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff130163540 con 0x7ff1180039b0 2024-04-15T15:21:16.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.870+0000 7ff12effd640 1 -- 172.21.15.105:0/3749112385 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff11800c060 con 0x7ff118004800 2024-04-15T15:21:16.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.870+0000 7ff12effd640 1 -- 172.21.15.105:0/3749112385 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7ff11800c260 con 0x7ff1180051c0 2024-04-15T15:21:16.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.870+0000 7ff12ffff640 1 --2- 172.21.15.105:0/3749112385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1180039b0 0x7ff118003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:16.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.870+0000 7ff12f7fe640 1 --2- 172.21.15.105:0/3749112385 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff118004800 0x7ff118004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:16.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.870+0000 7ff134f2c640 1 --2- 172.21.15.105:0/3749112385 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff1180051c0 0x7ff11800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:16.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.870+0000 7ff12ffff640 1 -- 172.21.15.105:0/3749112385 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff1180051c0 msgr2=0x7ff11800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:16.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.870+0000 7ff12ffff640 1 --2- 172.21.15.105:0/3749112385 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff1180051c0 0x7ff11800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:16.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.870+0000 7ff12ffff640 1 -- 172.21.15.105:0/3749112385 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff118004800 msgr2=0x7ff118004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:16.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.870+0000 7ff12ffff640 1 --2- 172.21.15.105:0/3749112385 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff118004800 0x7ff118004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:16.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.870+0000 7ff134f2c640 1 -- 172.21.15.105:0/3749112385 reap_dead start 2024-04-15T15:21:16.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.870+0000 7ff12ffff640 1 -- 172.21.15.105:0/3749112385 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff118003520 con 0x7ff1180039b0 2024-04-15T15:21:16.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.871+0000 7ff12ffff640 1 --2- 172.21.15.105:0/3749112385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1180039b0 0x7ff118003dd0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7ff12000c210 tx=0x7ff118005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52c9dad1b9541239 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:16.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.871+0000 7ff12effd640 1 -- 172.21.15.105:0/3749112385 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff120007d30 con 0x7ff1180039b0 2024-04-15T15:21:16.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.871+0000 7ff12effd640 1 -- 172.21.15.105:0/3749112385 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff120030070 con 0x7ff1180039b0 2024-04-15T15:21:16.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.871+0000 7ff12effd640 1 -- 172.21.15.105:0/3749112385 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff120005d70 con 0x7ff1180039b0 2024-04-15T15:21:16.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.872+0000 7ff13698f640 1 -- 172.21.15.105:0/3749112385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1180039b0 msgr2=0x7ff118003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:16.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.872+0000 7ff13698f640 1 --2- 172.21.15.105:0/3749112385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1180039b0 0x7ff118003dd0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7ff12000c210 tx=0x7ff118005830 comp rx=0 tx=0).stop 2024-04-15T15:21:16.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.872+0000 7ff13698f640 1 -- 172.21.15.105:0/3749112385 shutdown_connections 2024-04-15T15:21:16.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.872+0000 7ff13698f640 1 --2- 172.21.15.105:0/3749112385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1180039b0 0x7ff118003dd0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:16.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.872+0000 7ff13698f640 1 -- 172.21.15.105:0/3749112385 >> 172.21.15.105:0/3749112385 conn(0x7ff130067720 msgr2=0x7ff130067b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:16.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.872+0000 7ff13698f640 1 -- 172.21.15.105:0/3749112385 shutdown_connections 2024-04-15T15:21:16.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.872+0000 7ff13698f640 1 -- 172.21.15.105:0/3749112385 wait complete. 2024-04-15T15:21:16.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.873+0000 7ff13698f640 1 Processor -- start 2024-04-15T15:21:16.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.873+0000 7ff13698f640 1 -- start start 2024-04-15T15:21:16.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.874+0000 7ff13698f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1180039b0 0x7ff1300cd650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:16.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.874+0000 7ff13698f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff13015f130 0x7ff130155be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:16.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.874+0000 7ff13698f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff130160950 0x7ff13015ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:16.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.874+0000 7ff13698f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff130162f80 con 0x7ff1180039b0 2024-04-15T15:21:16.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.874+0000 7ff13698f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff1300d2fc0 con 0x7ff130160950 2024-04-15T15:21:16.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.874+0000 7ff13698f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7ff1300b78f0 con 0x7ff13015f130 2024-04-15T15:21:16.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.874+0000 7ff12ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1180039b0 0x7ff1300cd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:16.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.874+0000 7ff12ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1180039b0 0x7ff1300cd650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33384/0 (socket says 172.21.15.105:33384) 2024-04-15T15:21:16.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.874+0000 7ff12ffff640 1 -- 172.21.15.105:0/3368267720 learned_addr learned my addr 172.21.15.105:0/3368267720 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:16.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.874+0000 7ff12f7fe640 1 --2- 172.21.15.105:0/3368267720 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff13015f130 0x7ff130155be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:16.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.874+0000 7ff134f2c640 1 --2- 172.21.15.105:0/3368267720 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff130160950 0x7ff13015ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:16.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.874+0000 7ff12ffff640 1 -- 172.21.15.105:0/3368267720 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff13015f130 msgr2=0x7ff130155be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:16.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.874+0000 7ff12ffff640 1 --2- 172.21.15.105:0/3368267720 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff13015f130 0x7ff130155be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:16.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.874+0000 7ff12ffff640 1 -- 172.21.15.105:0/3368267720 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff130160950 msgr2=0x7ff13015ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:16.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.874+0000 7ff12ffff640 1 --2- 172.21.15.105:0/3368267720 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff130160950 0x7ff13015ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:16.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.874+0000 7ff12ffff640 1 -- 172.21.15.105:0/3368267720 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff130156120 con 0x7ff1180039b0 2024-04-15T15:21:16.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.874+0000 7ff12f7fe640 1 --2- 172.21.15.105:0/3368267720 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff13015f130 0x7ff130155be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:21:16.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.874+0000 7ff134f2c640 1 --2- 172.21.15.105:0/3368267720 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff130160950 0x7ff13015ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:21:16.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.875+0000 7ff12ffff640 1 --2- 172.21.15.105:0/3368267720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1180039b0 0x7ff1300cd650 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7ff12000c210 tx=0x7ff12002e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:16.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.875+0000 7ff12cff9640 1 -- 172.21.15.105:0/3368267720 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff120004bf0 con 0x7ff1180039b0 2024-04-15T15:21:16.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.875+0000 7ff12cff9640 1 -- 172.21.15.105:0/3368267720 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff120004d90 con 0x7ff1180039b0 2024-04-15T15:21:16.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.875+0000 7ff12cff9640 1 -- 172.21.15.105:0/3368267720 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff120007270 con 0x7ff1180039b0 2024-04-15T15:21:16.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.875+0000 7ff13698f640 1 -- 172.21.15.105:0/3368267720 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff13015f8b0 con 0x7ff1180039b0 2024-04-15T15:21:16.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.875+0000 7ff13698f640 1 -- 172.21.15.105:0/3368267720 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff13015fe70 con 0x7ff1180039b0 2024-04-15T15:21:16.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.876+0000 7ff13698f640 1 -- 172.21.15.105:0/3368267720 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff130073c20 con 0x7ff1180039b0 2024-04-15T15:21:16.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.876+0000 7ff12cff9640 1 -- 172.21.15.105:0/3368267720 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7ff120048020 con 0x7ff1180039b0 2024-04-15T15:21:16.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.877+0000 7ff12cff9640 1 --2- 172.21.15.105:0/3368267720 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff11c06c9b0 0x7ff11c06ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:16.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.877+0000 7ff12cff9640 1 -- 172.21.15.105:0/3368267720 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5795+0+0 (secure 0 0 0) 0x7ff1200ad080 con 0x7ff1180039b0 2024-04-15T15:21:16.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.877+0000 7ff12f7fe640 1 --2- 172.21.15.105:0/3368267720 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff11c06c9b0 0x7ff11c06ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:16.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.877+0000 7ff12f7fe640 1 --2- 172.21.15.105:0/3368267720 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff11c06c9b0 0x7ff11c06ee70 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff13015c640 tx=0x7ff124008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:16.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:16.881+0000 7ff12cff9640 1 -- 172.21.15.105:0/3368267720 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff130073c20 con 0x7ff1180039b0 2024-04-15T15:21:17.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.061+0000 7ff13698f640 1 -- 172.21.15.105:0/3368267720 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff1300d23f0 con 0x7ff1180039b0 2024-04-15T15:21:17.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.062+0000 7ff12cff9640 1 -- 172.21.15.105:0/3368267720 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16846 (secure 0 0 0) 0x7ff12007b4f0 con 0x7ff1180039b0 2024-04-15T15:21:17.064 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:21:17.064 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":35,"fsid":"09e4d2a4-a324-4793-af77-644b25e520e1","created":"2024-04-15T15:19:01.524543+0000","modified":"2024-04-15T15:21:03.660870+0000","last_up_change":"2024-04-15T15:19:12.825815+0000","last_in_change":"2024-04-15T15:19:03.788126+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:19:13.748585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:20:26.983353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-15T15:20:27.961802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6f6292b-2d7e-4aa0-9d1c-6ddd0f97287c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6801","nonce":2873040054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6803","nonce":2873040054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6807","nonce":2873040054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6805","nonce":2873040054}]},"public_addr":"172.21.15.105:6801/2873040054","cluster_addr":"172.21.15.105:6803/2873040054","heartbeat_back_addr":"172.21.15.105:6807/2873040054","heartbeat_front_addr":"172.21.15.105:6805/2873040054","state":["exists","up"]},{"osd":1,"uuid":"8900966a-b393-41e3-8f93-e5a43bd23622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6818","nonce":2494994479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6821","nonce":2494994479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6828","nonce":2494994479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6823","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6825","nonce":2494994479}]},"public_addr":"172.21.15.105:6818/2494994479","cluster_addr":"172.21.15.105:6821/2494994479","heartbeat_back_addr":"172.21.15.105:6828/2494994479","heartbeat_front_addr":"172.21.15.105:6825/2494994479","state":["exists","up"]},{"osd":2,"uuid":"81a5dc7a-9325-4ff5-941d-a2d5fba21f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6809","nonce":3798408699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6811","nonce":3798408699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6815","nonce":3798408699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6813","nonce":3798408699}]},"public_addr":"172.21.15.105:6809/3798408699","cluster_addr":"172.21.15.105:6811/3798408699","heartbeat_back_addr":"172.21.15.105:6815/3798408699","heartbeat_front_addr":"172.21.15.105:6813/3798408699","state":["exists","up"]},{"osd":3,"uuid":"df766ed2-5c71-4805-96f4-3d4970edc960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6819","nonce":1920720186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6824","nonce":1920720186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6831","nonce":1920720186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6827","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6829","nonce":1920720186}]},"public_addr":"172.21.15.105:6819/1920720186","cluster_addr":"172.21.15.105:6824/1920720186","heartbeat_back_addr":"172.21.15.105:6831/1920720186","heartbeat_front_addr":"172.21.15.105:6829/1920720186","state":["exists","up"]},{"osd":4,"uuid":"e1039e03-28b8-49df-b951-c91930b93c9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6810","nonce":3516494456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6813","nonce":3516494456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6820","nonce":3516494456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6817","nonce":3516494456}]},"public_addr":"172.21.15.154:6810/3516494456","cluster_addr":"172.21.15.154:6813/3516494456","heartbeat_back_addr":"172.21.15.154:6820/3516494456","heartbeat_front_addr":"172.21.15.154:6817/3516494456","state":["exists","up"]},{"osd":5,"uuid":"a9f5eaf1-66ff-405a-a9e4-ec3340812a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6801","nonce":540367751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6803","nonce":540367751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6807","nonce":540367751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6805","nonce":540367751}]},"public_addr":"172.21.15.154:6801/540367751","cluster_addr":"172.21.15.154:6803/540367751","heartbeat_back_addr":"172.21.15.154:6807/540367751","heartbeat_front_addr":"172.21.15.154:6805/540367751","state":["exists","up"]},{"osd":6,"uuid":"267d02ea-675f-428f-aae8-e0375e4d4feb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6825","nonce":1550829005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6827","nonce":1550829005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6831","nonce":1550829005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6829","nonce":1550829005}]},"public_addr":"172.21.15.154:6825/1550829005","cluster_addr":"172.21.15.154:6827/1550829005","heartbeat_back_addr":"172.21.15.154:6831/1550829005","heartbeat_front_addr":"172.21.15.154:6829/1550829005","state":["exists","up"]},{"osd":7,"uuid":"759ac6cc-896a-423c-885f-9fe90d8dcb14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6809","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6811","nonce":3920089903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6816","nonce":3920089903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6823","nonce":3920089903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6819","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6821","nonce":3920089903}]},"public_addr":"172.21.15.154:6811/3920089903","cluster_addr":"172.21.15.154:6816/3920089903","heartbeat_back_addr":"172.21.15.154:6823/3920089903","heartbeat_front_addr":"172.21.15.154:6821/3920089903","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.949509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.984072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.907278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.947391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.975447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.998896+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:09.028739+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.894119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.105:6837/958066041":"2024-04-16T15:21:03.660841+0000","172.21.15.105:6835/958066041":"2024-04-16T15:21:03.660841+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:21:17.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.064+0000 7ff13698f640 1 -- 172.21.15.105:0/3368267720 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff11c06c9b0 msgr2=0x7ff11c06ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:17.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.064+0000 7ff13698f640 1 --2- 172.21.15.105:0/3368267720 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff11c06c9b0 0x7ff11c06ee70 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff13015c640 tx=0x7ff124008040 comp rx=0 tx=0).stop 2024-04-15T15:21:17.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.064+0000 7ff13698f640 1 -- 172.21.15.105:0/3368267720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1180039b0 msgr2=0x7ff1300cd650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:17.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.064+0000 7ff13698f640 1 --2- 172.21.15.105:0/3368267720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1180039b0 0x7ff1300cd650 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7ff12000c210 tx=0x7ff12002e070 comp rx=0 tx=0).stop 2024-04-15T15:21:17.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.064+0000 7ff13698f640 1 -- 172.21.15.105:0/3368267720 shutdown_connections 2024-04-15T15:21:17.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.064+0000 7ff13698f640 1 --2- 172.21.15.105:0/3368267720 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff13015f130 0x7ff130155be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:17.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.064+0000 7ff13698f640 1 --2- 172.21.15.105:0/3368267720 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ff11c06c9b0 0x7ff11c06ee70 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:17.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.064+0000 7ff13698f640 1 --2- 172.21.15.105:0/3368267720 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff130160950 0x7ff13015ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:17.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.064+0000 7ff13698f640 1 --2- 172.21.15.105:0/3368267720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1180039b0 0x7ff1300cd650 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:17.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.064+0000 7ff13698f640 1 -- 172.21.15.105:0/3368267720 >> 172.21.15.105:0/3368267720 conn(0x7ff130067720 msgr2=0x7ff13015de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:17.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.064+0000 7ff13698f640 1 -- 172.21.15.105:0/3368267720 shutdown_connections 2024-04-15T15:21:17.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.064+0000 7ff13698f640 1 -- 172.21.15.105:0/3368267720 wait complete. 2024-04-15T15:21:17.077 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:19:13.748585+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-15T15:20:26.983353+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-15T15:20:27.961802+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:21:17.078 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:21:17.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.181+0000 7fe9d507e640 1 Processor -- start 2024-04-15T15:21:17.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.181+0000 7fe9d507e640 1 -- start start 2024-04-15T15:21:17.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.182+0000 7fe9d507e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe9d015eaf0 0x7fe9d00b6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:17.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.182+0000 7fe9d507e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe9d00d3f00 0x7fe9d00d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:17.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.182+0000 7fe9d507e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9d0162a10 con 0x7fe9d00d1560 2024-04-15T15:21:17.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.182+0000 7fe9d507e640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fe9d00b71b0 con 0x7fe9d015eaf0 2024-04-15T15:21:17.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.182+0000 7fe9d507e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fe9d0162cf0 con 0x7fe9d00d3f00 2024-04-15T15:21:17.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.182+0000 7fe9ced9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe9d015eaf0 0x7fe9d00b6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:17.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.182+0000 7fe9ced9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe9d015eaf0 0x7fe9d00b6c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33396/0 (socket says 172.21.15.105:33396) 2024-04-15T15:21:17.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.182+0000 7fe9ced9d640 1 -- 172.21.15.105:0/494533183 learned_addr learned my addr 172.21.15.105:0/494533183 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:17.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.182+0000 7fe9cf59e640 1 --2- 172.21.15.105:0/494533183 >> v2:172.21.15.154:3300/0 conn(0x7fe9d00d3f00 0x7fe9d00d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:17.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.183+0000 7fe9ced9d640 1 -- 172.21.15.105:0/494533183 >> v2:172.21.15.154:3300/0 conn(0x7fe9d00d3f00 msgr2=0x7fe9d00d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:17.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.183+0000 7fe9ced9d640 1 --2- 172.21.15.105:0/494533183 >> v2:172.21.15.154:3300/0 conn(0x7fe9d00d3f00 0x7fe9d00d19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:17.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.183+0000 7fe9ced9d640 1 -- 172.21.15.105:0/494533183 >> v1:172.21.15.154:6789/0 conn(0x7fe9d00d1560 legacy=0x7fe9d00cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:17.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.183+0000 7fe9ced9d640 1 -- 172.21.15.105:0/494533183 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9d0162fd0 con 0x7fe9d015eaf0 2024-04-15T15:21:17.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.183+0000 7fe9ced9d640 1 --2- 172.21.15.105:0/494533183 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9d015eaf0 0x7fe9d00b6c70 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fe9d008d8b0 tx=0x7fe9c002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cad205d219cfba9 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:17.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.183+0000 7fe9cdd9b640 1 -- 172.21.15.105:0/494533183 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c002c9f0 con 0x7fe9d015eaf0 2024-04-15T15:21:17.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.183+0000 7fe9cdd9b640 1 -- 172.21.15.105:0/494533183 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9c002cb90 con 0x7fe9d015eaf0 2024-04-15T15:21:17.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.183+0000 7fe9cdd9b640 1 -- 172.21.15.105:0/494533183 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c00343f0 con 0x7fe9d015eaf0 2024-04-15T15:21:17.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.184+0000 7fe9d507e640 1 -- 172.21.15.105:0/494533183 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9d015eaf0 msgr2=0x7fe9d00b6c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:17.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.184+0000 7fe9d507e640 1 --2- 172.21.15.105:0/494533183 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9d015eaf0 0x7fe9d00b6c70 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fe9d008d8b0 tx=0x7fe9c002be50 comp rx=0 tx=0).stop 2024-04-15T15:21:17.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.184+0000 7fe9d507e640 1 -- 172.21.15.105:0/494533183 shutdown_connections 2024-04-15T15:21:17.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.184+0000 7fe9d507e640 1 --2- 172.21.15.105:0/494533183 >> v2:172.21.15.154:3300/0 conn(0x7fe9d00d3f00 0x7fe9d00d19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:17.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.184+0000 7fe9d507e640 1 --2- 172.21.15.105:0/494533183 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9d015eaf0 0x7fe9d00b6c70 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:17.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.184+0000 7fe9d507e640 1 -- 172.21.15.105:0/494533183 >> 172.21.15.105:0/494533183 conn(0x7fe9d006cf90 msgr2=0x7fe9d006d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:17.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.184+0000 7fe9d507e640 1 -- 172.21.15.105:0/494533183 shutdown_connections 2024-04-15T15:21:17.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.184+0000 7fe9d507e640 1 -- 172.21.15.105:0/494533183 wait complete. 2024-04-15T15:21:17.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.185+0000 7fe9d507e640 1 Processor -- start 2024-04-15T15:21:17.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.186+0000 7fe9d507e640 1 -- start start 2024-04-15T15:21:17.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.186+0000 7fe9d507e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9d00d1560 0x7fe9d014c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:17.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.186+0000 7fe9d507e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9d00d3f00 0x7fe9d0149ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:17.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.186+0000 7fe9d507e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9d015eaf0 0x7fe9d014b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:17.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.186+0000 7fe9d507e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d00b3d60 con 0x7fe9d00d3f00 2024-04-15T15:21:17.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.186+0000 7fe9d507e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d015eed0 con 0x7fe9d015eaf0 2024-04-15T15:21:17.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.186+0000 7fe9d507e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fe9d0162650 con 0x7fe9d00d1560 2024-04-15T15:21:17.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.186+0000 7fe9ce59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9d00d3f00 0x7fe9d0149ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:17.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.186+0000 7fe9ce59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9d00d3f00 0x7fe9d0149ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33412/0 (socket says 172.21.15.105:33412) 2024-04-15T15:21:17.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.186+0000 7fe9ce59c640 1 -- 172.21.15.105:0/1114443746 learned_addr learned my addr 172.21.15.105:0/1114443746 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:17.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.187+0000 7fe9cf59e640 1 --2- 172.21.15.105:0/1114443746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9d015eaf0 0x7fe9d014b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:17.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.187+0000 7fe9ced9d640 1 --2- 172.21.15.105:0/1114443746 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9d00d1560 0x7fe9d014c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:17.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.187+0000 7fe9ce59c640 1 -- 172.21.15.105:0/1114443746 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9d00d1560 msgr2=0x7fe9d014c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:17.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.187+0000 7fe9ce59c640 1 --2- 172.21.15.105:0/1114443746 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9d00d1560 0x7fe9d014c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:17.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.187+0000 7fe9ce59c640 1 -- 172.21.15.105:0/1114443746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9d015eaf0 msgr2=0x7fe9d014b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:17.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.187+0000 7fe9ce59c640 1 --2- 172.21.15.105:0/1114443746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9d015eaf0 0x7fe9d014b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:17.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.187+0000 7fe9ce59c640 1 -- 172.21.15.105:0/1114443746 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9d014a250 con 0x7fe9d00d3f00 2024-04-15T15:21:17.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.187+0000 7fe9ce59c640 1 --2- 172.21.15.105:0/1114443746 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9d00d3f00 0x7fe9d0149ad0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fe9c400b3e0 tx=0x7fe9c400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:17.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.187+0000 7fe9b77fe640 1 -- 172.21.15.105:0/1114443746 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c4015020 con 0x7fe9d00d3f00 2024-04-15T15:21:17.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.187+0000 7fe9b77fe640 1 -- 172.21.15.105:0/1114443746 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9c4011850 con 0x7fe9d00d3f00 2024-04-15T15:21:17.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.187+0000 7fe9b77fe640 1 -- 172.21.15.105:0/1114443746 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c4011c40 con 0x7fe9d00d3f00 2024-04-15T15:21:17.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.187+0000 7fe9d507e640 1 -- 172.21.15.105:0/1114443746 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9d014a530 con 0x7fe9d00d3f00 2024-04-15T15:21:17.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.187+0000 7fe9d507e640 1 -- 172.21.15.105:0/1114443746 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9d014aaf0 con 0x7fe9d00d3f00 2024-04-15T15:21:17.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.188+0000 7fe9d507e640 1 -- 172.21.15.105:0/1114443746 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9d00cfed0 con 0x7fe9d00d3f00 2024-04-15T15:21:17.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.188+0000 7fe9b77fe640 1 -- 172.21.15.105:0/1114443746 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fe9c4018020 con 0x7fe9d00d3f00 2024-04-15T15:21:17.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.189+0000 7fe9b77fe640 1 --2- 172.21.15.105:0/1114443746 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fe9bc06c980 0x7fe9bc06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:17.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.189+0000 7fe9b77fe640 1 -- 172.21.15.105:0/1114443746 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5795+0+0 (secure 0 0 0) 0x7fe9c408bb20 con 0x7fe9d00d3f00 2024-04-15T15:21:17.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.189+0000 7fe9ced9d640 1 --2- 172.21.15.105:0/1114443746 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fe9bc06c980 0x7fe9bc06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:17.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.189+0000 7fe9ced9d640 1 --2- 172.21.15.105:0/1114443746 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fe9bc06c980 0x7fe9bc06ee40 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0035000 tx=0x7fe9c0030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:17.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.193+0000 7fe9b77fe640 1 -- 172.21.15.105:0/1114443746 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe9d00cfed0 con 0x7fe9d00d3f00 2024-04-15T15:21:17.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.370+0000 7fe9d507e640 1 -- 172.21.15.105:0/1114443746 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe9d0068ab0 con 0x7fe9d00d3f00 2024-04-15T15:21:17.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.370+0000 7fe9b77fe640 1 -- 172.21.15.105:0/1114443746 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) ==== 93+0+10 (secure 0 0 0) 0x7fe9c401f020 con 0x7fe9d00d3f00 2024-04-15T15:21:17.372 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-04-15T15:21:17.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.372+0000 7fe9d507e640 1 -- 172.21.15.105:0/1114443746 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fe9bc06c980 msgr2=0x7fe9bc06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:17.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.372+0000 7fe9d507e640 1 --2- 172.21.15.105:0/1114443746 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fe9bc06c980 0x7fe9bc06ee40 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0035000 tx=0x7fe9c0030000 comp rx=0 tx=0).stop 2024-04-15T15:21:17.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.372+0000 7fe9d507e640 1 -- 172.21.15.105:0/1114443746 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9d00d3f00 msgr2=0x7fe9d0149ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:17.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.372+0000 7fe9d507e640 1 --2- 172.21.15.105:0/1114443746 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9d00d3f00 0x7fe9d0149ad0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fe9c400b3e0 tx=0x7fe9c400b8a0 comp rx=0 tx=0).stop 2024-04-15T15:21:17.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.372+0000 7fe9d507e640 1 -- 172.21.15.105:0/1114443746 shutdown_connections 2024-04-15T15:21:17.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.372+0000 7fe9d507e640 1 --2- 172.21.15.105:0/1114443746 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fe9bc06c980 0x7fe9bc06ee40 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:17.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.372+0000 7fe9d507e640 1 --2- 172.21.15.105:0/1114443746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9d015eaf0 0x7fe9d014b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:17.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.372+0000 7fe9d507e640 1 --2- 172.21.15.105:0/1114443746 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9d00d3f00 0x7fe9d0149ad0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:17.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.372+0000 7fe9d507e640 1 --2- 172.21.15.105:0/1114443746 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9d00d1560 0x7fe9d014c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:17.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.372+0000 7fe9d507e640 1 -- 172.21.15.105:0/1114443746 >> 172.21.15.105:0/1114443746 conn(0x7fe9d006cf90 msgr2=0x7fe9d00d19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:17.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.372+0000 7fe9d507e640 1 -- 172.21.15.105:0/1114443746 shutdown_connections 2024-04-15T15:21:17.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.372+0000 7fe9d507e640 1 -- 172.21.15.105:0/1114443746 wait complete. 2024-04-15T15:21:17.385 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-15T15:21:17.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.489+0000 7f91d9952640 1 Processor -- start 2024-04-15T15:21:17.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.489+0000 7f91d9952640 1 -- start start 2024-04-15T15:21:17.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.490+0000 7f91d9952640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f91d40b7910 0x7f91d40d2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:17.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.490+0000 7f91d9952640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f91d41625c0 0x7f91d4162a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:17.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.490+0000 7f91d9952640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91d40b7d70 con 0x7f91d41620c0 2024-04-15T15:21:17.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.490+0000 7f91d9952640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f91d4162f50 con 0x7f91d41625c0 2024-04-15T15:21:17.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.490+0000 7f91d9952640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f91d40b8c00 con 0x7f91d40b7910 2024-04-15T15:21:17.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.490+0000 7f91d37fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f91d41625c0 0x7f91d4162a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:17.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.490+0000 7f91d37fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f91d41625c0 0x7f91d4162a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33416/0 (socket says 172.21.15.105:33416) 2024-04-15T15:21:17.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.490+0000 7f91d2ffd640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f91d41620c0 0x7f91d40ba190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:37598/0 (socket says 172.21.15.105:37598) 2024-04-15T15:21:17.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.490+0000 7f91d37fe640 1 -- 172.21.15.105:0/3967929490 learned_addr learned my addr 172.21.15.105:0/3967929490 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:17.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.490+0000 7f91d37fe640 1 -- 172.21.15.105:0/3967929490 >> v2:172.21.15.154:3300/0 conn(0x7f91d40b7910 msgr2=0x7f91d40d2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:17.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.490+0000 7f91d27fc640 1 --2- 172.21.15.105:0/3967929490 >> v2:172.21.15.154:3300/0 conn(0x7f91d40b7910 0x7f91d40d2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:17.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.490+0000 7f91d1ffb640 1 -- 172.21.15.105:0/3967929490 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1447594562 0 0) 0x7f91c4001490 con 0x7f91d41620c0 2024-04-15T15:21:17.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.490+0000 7f91d37fe640 1 --2- 172.21.15.105:0/3967929490 >> v2:172.21.15.154:3300/0 conn(0x7f91d40b7910 0x7f91d40d2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:17.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.490+0000 7f91d37fe640 1 -- 172.21.15.105:0/3967929490 >> v1:172.21.15.105:6789/0 conn(0x7f91d41620c0 legacy=0x7f91d40ba190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:17.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.490+0000 7f91d37fe640 1 -- 172.21.15.105:0/3967929490 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91d40b7ff0 con 0x7f91d41625c0 2024-04-15T15:21:17.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.491+0000 7f91d27fc640 1 --2- 172.21.15.105:0/3967929490 >> v2:172.21.15.154:3300/0 conn(0x7f91d40b7910 0x7f91d40d2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:21:17.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.491+0000 7f91d37fe640 1 --2- 172.21.15.105:0/3967929490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91d41625c0 0x7f91d4162a10 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f91d40d0b50 tx=0x7f91cc02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a41575ce699ae0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:17.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.491+0000 7f91d1ffb640 1 -- 172.21.15.105:0/3967929490 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91cc034070 con 0x7f91d41625c0 2024-04-15T15:21:17.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.491+0000 7f91d1ffb640 1 -- 172.21.15.105:0/3967929490 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91cc030860 con 0x7f91d41625c0 2024-04-15T15:21:17.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.491+0000 7f91d1ffb640 1 -- 172.21.15.105:0/3967929490 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91cc030c30 con 0x7f91d41625c0 2024-04-15T15:21:17.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.492+0000 7f91d9952640 1 -- 172.21.15.105:0/3967929490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91d41625c0 msgr2=0x7f91d4162a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:17.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.492+0000 7f91d9952640 1 --2- 172.21.15.105:0/3967929490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91d41625c0 0x7f91d4162a10 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f91d40d0b50 tx=0x7f91cc02b9a0 comp rx=0 tx=0).stop 2024-04-15T15:21:17.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.492+0000 7f91d9952640 1 -- 172.21.15.105:0/3967929490 shutdown_connections 2024-04-15T15:21:17.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.492+0000 7f91d9952640 1 --2- 172.21.15.105:0/3967929490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91d41625c0 0x7f91d4162a10 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:17.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.492+0000 7f91d9952640 1 --2- 172.21.15.105:0/3967929490 >> v2:172.21.15.154:3300/0 conn(0x7f91d40b7910 0x7f91d40d2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:17.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.492+0000 7f91d9952640 1 -- 172.21.15.105:0/3967929490 >> 172.21.15.105:0/3967929490 conn(0x7f91d406ccb0 msgr2=0x7f91d406d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:17.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.492+0000 7f91d9952640 1 -- 172.21.15.105:0/3967929490 shutdown_connections 2024-04-15T15:21:17.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.492+0000 7f91d9952640 1 -- 172.21.15.105:0/3967929490 wait complete. 2024-04-15T15:21:17.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.493+0000 7f91d9952640 1 Processor -- start 2024-04-15T15:21:17.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.493+0000 7f91d9952640 1 -- start start 2024-04-15T15:21:17.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.494+0000 7f91d9952640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91d40b7910 0x7f91d415e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:17.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.494+0000 7f91d9952640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91d41620c0 0x7f91d4161040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:17.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.494+0000 7f91d9952640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f91d41625c0 0x7f91d41f83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:17.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.494+0000 7f91d9952640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f91d40b8890 con 0x7f91d40b7910 2024-04-15T15:21:17.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.494+0000 7f91d9952640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f91d40b85b0 con 0x7f91d41620c0 2024-04-15T15:21:17.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.494+0000 7f91d9952640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f91d40b9090 con 0x7f91d41625c0 2024-04-15T15:21:17.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.494+0000 7f91d2ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91d40b7910 0x7f91d415e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:17.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.494+0000 7f91d2ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91d40b7910 0x7f91d415e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33420/0 (socket says 172.21.15.105:33420) 2024-04-15T15:21:17.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.494+0000 7f91d2ffd640 1 -- 172.21.15.105:0/3064384718 learned_addr learned my addr 172.21.15.105:0/3064384718 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:17.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.494+0000 7f91d2ffd640 1 -- 172.21.15.105:0/3064384718 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f91d41625c0 msgr2=0x7f91d41f83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:17.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.494+0000 7f91d2ffd640 1 --2- 172.21.15.105:0/3064384718 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f91d41625c0 0x7f91d41f83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:17.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.494+0000 7f91d2ffd640 1 -- 172.21.15.105:0/3064384718 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91d41620c0 msgr2=0x7f91d4161040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:17.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.494+0000 7f91d2ffd640 1 --2- 172.21.15.105:0/3064384718 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91d41620c0 0x7f91d4161040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:17.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.494+0000 7f91d2ffd640 1 -- 172.21.15.105:0/3064384718 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91d4155500 con 0x7f91d40b7910 2024-04-15T15:21:17.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.495+0000 7f91d2ffd640 1 --2- 172.21.15.105:0/3064384718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91d40b7910 0x7f91d415e0c0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f91c400c850 tx=0x7f91c400cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:17.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.495+0000 7f91bbfff640 1 -- 172.21.15.105:0/3064384718 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c4018070 con 0x7f91d40b7910 2024-04-15T15:21:17.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.495+0000 7f91bbfff640 1 -- 172.21.15.105:0/3064384718 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91c400a640 con 0x7f91d40b7910 2024-04-15T15:21:17.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.495+0000 7f91d9952640 1 -- 172.21.15.105:0/3064384718 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91d4161580 con 0x7f91d40b7910 2024-04-15T15:21:17.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.495+0000 7f91bbfff640 1 -- 172.21.15.105:0/3064384718 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c400aa70 con 0x7f91d40b7910 2024-04-15T15:21:17.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.495+0000 7f91d9952640 1 -- 172.21.15.105:0/3064384718 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91d41557a0 con 0x7f91d40b7910 2024-04-15T15:21:17.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.496+0000 7f91d9952640 1 -- 172.21.15.105:0/3064384718 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91d4068a40 con 0x7f91d40b7910 2024-04-15T15:21:17.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.496+0000 7f91bbfff640 1 -- 172.21.15.105:0/3064384718 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f91c401b070 con 0x7f91d40b7910 2024-04-15T15:21:17.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.497+0000 7f91bbfff640 1 --2- 172.21.15.105:0/3064384718 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f91c006c980 0x7f91c006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:17.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.497+0000 7f91bbfff640 1 -- 172.21.15.105:0/3064384718 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5795+0+0 (secure 0 0 0) 0x7f91c408b200 con 0x7f91d40b7910 2024-04-15T15:21:17.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.497+0000 7f91d27fc640 1 --2- 172.21.15.105:0/3064384718 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f91c006c980 0x7f91c006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:17.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.497+0000 7f91d27fc640 1 --2- 172.21.15.105:0/3064384718 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f91c006c980 0x7f91c006ee40 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f91c8006fd0 tx=0x7f91c8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:17.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.501+0000 7f91bbfff640 1 -- 172.21.15.105:0/3064384718 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f91d40b7d70 con 0x7f91d40b7910 2024-04-15T15:21:17.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.679+0000 7f91d9952640 1 -- 172.21.15.105:0/3064384718 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f91d40d2410 con 0x7f91d40b7910 2024-04-15T15:21:17.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.679+0000 7f91bbfff640 1 -- 172.21.15.105:0/3064384718 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v35) ==== 104+0+11 (secure 0 0 0) 0x7f91c4059670 con 0x7f91d40b7910 2024-04-15T15:21:17.681 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-04-15T15:21:17.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.681+0000 7f91d9952640 1 -- 172.21.15.105:0/3064384718 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f91c006c980 msgr2=0x7f91c006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:17.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.681+0000 7f91d9952640 1 --2- 172.21.15.105:0/3064384718 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f91c006c980 0x7f91c006ee40 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f91c8006fd0 tx=0x7f91c8008040 comp rx=0 tx=0).stop 2024-04-15T15:21:17.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.681+0000 7f91d9952640 1 -- 172.21.15.105:0/3064384718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91d40b7910 msgr2=0x7f91d415e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:17.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.681+0000 7f91d9952640 1 --2- 172.21.15.105:0/3064384718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91d40b7910 0x7f91d415e0c0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f91c400c850 tx=0x7f91c400cd10 comp rx=0 tx=0).stop 2024-04-15T15:21:17.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.681+0000 7f91d9952640 1 -- 172.21.15.105:0/3064384718 shutdown_connections 2024-04-15T15:21:17.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.681+0000 7f91d9952640 1 --2- 172.21.15.105:0/3064384718 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f91d41625c0 0x7f91d41f83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:17.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.681+0000 7f91d9952640 1 --2- 172.21.15.105:0/3064384718 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f91c006c980 0x7f91c006ee40 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:17.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.681+0000 7f91d9952640 1 --2- 172.21.15.105:0/3064384718 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91d41620c0 0x7f91d4161040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:17.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.681+0000 7f91d9952640 1 --2- 172.21.15.105:0/3064384718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91d40b7910 0x7f91d415e0c0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:17.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.681+0000 7f91d9952640 1 -- 172.21.15.105:0/3064384718 >> 172.21.15.105:0/3064384718 conn(0x7f91d406ccb0 msgr2=0x7f91d40d3c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:17.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.681+0000 7f91d9952640 1 -- 172.21.15.105:0/3064384718 shutdown_connections 2024-04-15T15:21:17.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.681+0000 7f91d9952640 1 -- 172.21.15.105:0/3064384718 wait complete. 2024-04-15T15:21:17.694 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-15T15:21:17.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.799+0000 7f38d3c60640 1 Processor -- start 2024-04-15T15:21:17.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.799+0000 7f38d3c60640 1 -- start start 2024-04-15T15:21:17.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.800+0000 7f38d3c60640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f38cc0af820 0x7f38cc15d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:17.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.800+0000 7f38d3c60640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38cc0ade60 con 0x7f38cc15a3f0 2024-04-15T15:21:17.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.800+0000 7f38d3c60640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38cc0ae140 con 0x7f38cc0cd290 2024-04-15T15:21:17.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.800+0000 7f38d3c60640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f38cc0aef00 con 0x7f38cc0af820 2024-04-15T15:21:17.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.800+0000 7f38d19fc640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f38cc15a3f0 0x7f38cc0af110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:37612/0 (socket says 172.21.15.105:37612) 2024-04-15T15:21:17.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.800+0000 7f38d19fc640 1 -- 172.21.15.105:0/3590996116 learned_addr learned my addr 172.21.15.105:0/3590996116 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:21:17.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.800+0000 7f38d09fa640 1 -- 172.21.15.105:0/3590996116 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4156497458 0 0) 0x7f38cc0ade60 con 0x7f38cc15a3f0 2024-04-15T15:21:17.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.800+0000 7f38d11fb640 1 --2- 172.21.15.105:0/3590996116 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f38cc0af820 0x7f38cc15d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:17.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.800+0000 7f38d09fa640 1 -- 172.21.15.105:0/3590996116 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f38b8003520 con 0x7f38cc15a3f0 2024-04-15T15:21:17.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.800+0000 7f38d09fa640 1 -- 172.21.15.105:0/3590996116 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1745275487 0 0) 0x7f38b8003520 con 0x7f38cc15a3f0 2024-04-15T15:21:17.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.800+0000 7f38d09fa640 1 -- 172.21.15.105:0/3590996116 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f38cc0af820 msgr2=0x7f38cc15d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:17.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.800+0000 7f38d09fa640 1 --2- 172.21.15.105:0/3590996116 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f38cc0af820 0x7f38cc15d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:17.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.800+0000 7f38d09fa640 1 -- 172.21.15.105:0/3590996116 >> v1:172.21.15.154:6789/0 conn(0x7f38cc0cd290 legacy=0x7f38cc06e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:17.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.800+0000 7f38d09fa640 1 -- 172.21.15.105:0/3590996116 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38cc0ae420 con 0x7f38cc15a3f0 2024-04-15T15:21:17.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.800+0000 7f38d09fa640 1 -- 172.21.15.105:0/3590996116 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7f38c8001690 con 0x7f38cc15a3f0 2024-04-15T15:21:17.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.800+0000 7f38d09fa640 1 -- 172.21.15.105:0/3590996116 >> v1:172.21.15.105:6789/0 conn(0x7f38cc15a3f0 legacy=0x7f38cc0af110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:17.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.801+0000 7f38d09fa640 1 --2- 172.21.15.105:0/3590996116 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38b80039e0 0x7f38b8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:17.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.801+0000 7f38d09fa640 1 --2- 172.21.15.105:0/3590996116 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f38cc0af820 0x7f38b8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:17.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.801+0000 7f38d09fa640 1 --2- 172.21.15.105:0/3590996116 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f38b8004d40 0x7f38b80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:17.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.801+0000 7f38d09fa640 1 -- 172.21.15.105:0/3590996116 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f38b800ba70 con 0x7f38b80039e0 2024-04-15T15:21:17.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.801+0000 7f38d09fa640 1 -- 172.21.15.105:0/3590996116 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f38b800bbf0 con 0x7f38b8004d40 2024-04-15T15:21:17.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.801+0000 7f38d09fa640 1 -- 172.21.15.105:0/3590996116 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f38b800bdf0 con 0x7f38cc0af820 2024-04-15T15:21:17.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.801+0000 7f38d19fc640 1 --2- 172.21.15.105:0/3590996116 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38b80039e0 0x7f38b8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:17.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.801+0000 7f38d11fb640 1 --2- 172.21.15.105:0/3590996116 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f38cc0af820 0x7f38b8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:17.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.801+0000 7f38d21fd640 1 --2- 172.21.15.105:0/3590996116 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f38b8004d40 0x7f38b80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:17.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.801+0000 7f38d19fc640 1 -- 172.21.15.105:0/3590996116 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f38cc0af820 msgr2=0x7f38b8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:17.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.801+0000 7f38d19fc640 1 --2- 172.21.15.105:0/3590996116 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f38cc0af820 0x7f38b8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:17.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.801+0000 7f38d19fc640 1 -- 172.21.15.105:0/3590996116 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f38b8004d40 msgr2=0x7f38b80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:17.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.801+0000 7f38d19fc640 1 --2- 172.21.15.105:0/3590996116 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f38b8004d40 0x7f38b80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:17.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.801+0000 7f38d19fc640 1 -- 172.21.15.105:0/3590996116 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38b8003520 con 0x7f38b80039e0 2024-04-15T15:21:17.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.802+0000 7f38d19fc640 1 --2- 172.21.15.105:0/3590996116 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38b80039e0 0x7f38b8003e00 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f38cc15a800 tx=0x7f38b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=142ad4953f2a4a0e server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:17.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.802+0000 7f38d09fa640 1 -- 172.21.15.105:0/3590996116 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c8007d10 con 0x7f38b80039e0 2024-04-15T15:21:17.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.802+0000 7f38d09fa640 1 -- 172.21.15.105:0/3590996116 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38c8030070 con 0x7f38b80039e0 2024-04-15T15:21:17.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.802+0000 7f38d09fa640 1 -- 172.21.15.105:0/3590996116 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c8005d10 con 0x7f38b80039e0 2024-04-15T15:21:17.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.802+0000 7f38d3c60640 1 -- 172.21.15.105:0/3590996116 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38b80039e0 msgr2=0x7f38b8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:17.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.802+0000 7f38d3c60640 1 --2- 172.21.15.105:0/3590996116 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38b80039e0 0x7f38b8003e00 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f38cc15a800 tx=0x7f38b8005830 comp rx=0 tx=0).stop 2024-04-15T15:21:17.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.802+0000 7f38d21fd640 1 -- 172.21.15.105:0/3590996116 reap_dead start 2024-04-15T15:21:17.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.802+0000 7f38d3c60640 1 -- 172.21.15.105:0/3590996116 shutdown_connections 2024-04-15T15:21:17.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.802+0000 7f38d3c60640 1 -- 172.21.15.105:0/3590996116 >> 172.21.15.105:0/3590996116 conn(0x7f38cc073570 msgr2=0x7f38cc06b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:17.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.802+0000 7f38d3c60640 1 -- 172.21.15.105:0/3590996116 shutdown_connections 2024-04-15T15:21:17.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.802+0000 7f38d3c60640 1 -- 172.21.15.105:0/3590996116 wait complete. 2024-04-15T15:21:17.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.804+0000 7f38d3c60640 1 Processor -- start 2024-04-15T15:21:17.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.804+0000 7f38d3c60640 1 -- start start 2024-04-15T15:21:17.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.804+0000 7f38d3c60640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38b80039e0 0x7f38cc0cea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:17.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.804+0000 7f38d3c60640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f38cc09e3a0 0x7f38cc1496c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:17.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.804+0000 7f38d3c60640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f38cc15b850 0x7f38cc15bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:17.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.804+0000 7f38d3c60640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f38cc0704c0 con 0x7f38b80039e0 2024-04-15T15:21:17.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.804+0000 7f38d3c60640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f38cc0ae9e0 con 0x7f38cc15b850 2024-04-15T15:21:17.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.804+0000 7f38d3c60640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f38cc15d720 con 0x7f38cc09e3a0 2024-04-15T15:21:17.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.804+0000 7f38d19fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38b80039e0 0x7f38cc0cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:17.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.804+0000 7f38d19fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38b80039e0 0x7f38cc0cea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33444/0 (socket says 172.21.15.105:33444) 2024-04-15T15:21:17.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.804+0000 7f38d19fc640 1 -- 172.21.15.105:0/448548407 learned_addr learned my addr 172.21.15.105:0/448548407 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:17.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.805+0000 7f38d19fc640 1 -- 172.21.15.105:0/448548407 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f38cc09e3a0 msgr2=0x7f38cc1496c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:17.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.805+0000 7f38d19fc640 1 --2- 172.21.15.105:0/448548407 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f38cc09e3a0 0x7f38cc1496c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:17.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.805+0000 7f38d19fc640 1 -- 172.21.15.105:0/448548407 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f38cc15b850 msgr2=0x7f38cc15bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:17.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.805+0000 7f38d21fd640 1 --2- 172.21.15.105:0/448548407 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f38cc15b850 0x7f38cc15bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:17.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.805+0000 7f38d19fc640 1 --2- 172.21.15.105:0/448548407 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f38cc15b850 0x7f38cc15bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:17.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.805+0000 7f38d19fc640 1 -- 172.21.15.105:0/448548407 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38cc0cf070 con 0x7f38b80039e0 2024-04-15T15:21:17.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.805+0000 7f38d21fd640 1 --2- 172.21.15.105:0/448548407 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f38cc15b850 0x7f38cc15bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:21:17.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.805+0000 7f38d19fc640 1 --2- 172.21.15.105:0/448548407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38b80039e0 0x7f38cc0cea90 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f38c802c500 tx=0x7f38cc0ae030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:17.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.805+0000 7f38c27fc640 1 -- 172.21.15.105:0/448548407 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c8045070 con 0x7f38b80039e0 2024-04-15T15:21:17.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.805+0000 7f38c27fc640 1 -- 172.21.15.105:0/448548407 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38c8004bf0 con 0x7f38b80039e0 2024-04-15T15:21:17.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.805+0000 7f38d3c60640 1 -- 172.21.15.105:0/448548407 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38cc0cf350 con 0x7f38b80039e0 2024-04-15T15:21:17.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.805+0000 7f38c27fc640 1 -- 172.21.15.105:0/448548407 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c8007d10 con 0x7f38b80039e0 2024-04-15T15:21:17.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.805+0000 7f38d3c60640 1 -- 172.21.15.105:0/448548407 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38cc0cf910 con 0x7f38b80039e0 2024-04-15T15:21:17.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.806+0000 7f38d3c60640 1 -- 172.21.15.105:0/448548407 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38cc068ab0 con 0x7f38b80039e0 2024-04-15T15:21:17.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.807+0000 7f38c27fc640 1 -- 172.21.15.105:0/448548407 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f38c8007070 con 0x7f38b80039e0 2024-04-15T15:21:17.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.807+0000 7f38c27fc640 1 --2- 172.21.15.105:0/448548407 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f38b006c9b0 0x7f38b006ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:17.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.807+0000 7f38c27fc640 1 -- 172.21.15.105:0/448548407 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5795+0+0 (secure 0 0 0) 0x7f38c80acac0 con 0x7f38b80039e0 2024-04-15T15:21:17.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.807+0000 7f38d11fb640 1 --2- 172.21.15.105:0/448548407 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f38b006c9b0 0x7f38b006ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:17.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.807+0000 7f38d11fb640 1 --2- 172.21.15.105:0/448548407 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f38b006c9b0 0x7f38b006ee70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f38cc0d02a0 tx=0x7f38bc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:17.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.811+0000 7f38c27fc640 1 -- 172.21.15.105:0/448548407 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f38c807afb0 con 0x7f38b80039e0 2024-04-15T15:21:17.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.987+0000 7f38d3c60640 1 -- 172.21.15.105:0/448548407 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f38cc0b1f50 con 0x7f38b80039e0 2024-04-15T15:21:17.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.987+0000 7f38c27fc640 1 -- 172.21.15.105:0/448548407 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v35) ==== 100+0+11 (secure 0 0 0) 0x7f38c807a8b0 con 0x7f38b80039e0 2024-04-15T15:21:17.989 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-04-15T15:21:17.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.990+0000 7f38d3c60640 1 -- 172.21.15.105:0/448548407 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f38b006c9b0 msgr2=0x7f38b006ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:17.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.990+0000 7f38d3c60640 1 --2- 172.21.15.105:0/448548407 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f38b006c9b0 0x7f38b006ee70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f38cc0d02a0 tx=0x7f38bc008040 comp rx=0 tx=0).stop 2024-04-15T15:21:17.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.990+0000 7f38d3c60640 1 -- 172.21.15.105:0/448548407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38b80039e0 msgr2=0x7f38cc0cea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:17.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.990+0000 7f38d3c60640 1 --2- 172.21.15.105:0/448548407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38b80039e0 0x7f38cc0cea90 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f38c802c500 tx=0x7f38cc0ae030 comp rx=0 tx=0).stop 2024-04-15T15:21:17.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.990+0000 7f38d3c60640 1 -- 172.21.15.105:0/448548407 shutdown_connections 2024-04-15T15:21:17.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.990+0000 7f38d3c60640 1 --2- 172.21.15.105:0/448548407 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f38cc09e3a0 0x7f38cc1496c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:17.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.990+0000 7f38d3c60640 1 --2- 172.21.15.105:0/448548407 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f38b006c9b0 0x7f38b006ee70 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:17.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.990+0000 7f38d3c60640 1 --2- 172.21.15.105:0/448548407 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f38cc15b850 0x7f38cc15bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:17.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.990+0000 7f38d3c60640 1 --2- 172.21.15.105:0/448548407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38b80039e0 0x7f38cc0cea90 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:17.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.990+0000 7f38d3c60640 1 -- 172.21.15.105:0/448548407 >> 172.21.15.105:0/448548407 conn(0x7f38cc073570 msgr2=0x7f38cc0af110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:17.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.990+0000 7f38d3c60640 1 -- 172.21.15.105:0/448548407 shutdown_connections 2024-04-15T15:21:17.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:17.990+0000 7f38d3c60640 1 -- 172.21.15.105:0/448548407 wait complete. 2024-04-15T15:21:18.003 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-15T15:21:18.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.104+0000 7f066ddd5640 1 Processor -- start 2024-04-15T15:21:18.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.104+0000 7f066ddd5640 1 -- start start 2024-04-15T15:21:18.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.105+0000 7f066ddd5640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f066815ecf0 0x7f066806e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:18.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.105+0000 7f066ddd5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f06680d0f10 0x7f06680ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:18.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.105+0000 7f066ddd5640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f066806d2a0 con 0x7f06680cd190 2024-04-15T15:21:18.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.105+0000 7f066ddd5640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f066806d580 con 0x7f066815ecf0 2024-04-15T15:21:18.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.105+0000 7f066ddd5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f066806d860 con 0x7f06680d0f10 2024-04-15T15:21:18.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.105+0000 7f06677fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f066815ecf0 0x7f066806e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:18.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.105+0000 7f06677fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f066815ecf0 0x7f066806e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33456/0 (socket says 172.21.15.105:33456) 2024-04-15T15:21:18.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.105+0000 7f06677fe640 1 -- 172.21.15.105:0/1704620139 learned_addr learned my addr 172.21.15.105:0/1704620139 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:18.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.105+0000 7f0666ffd640 1 --2- 172.21.15.105:0/1704620139 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f06680d0f10 0x7f06680ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:18.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.105+0000 7f06677fe640 1 -- 172.21.15.105:0/1704620139 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f06680d0f10 msgr2=0x7f06680ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:18.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.105+0000 7f06677fe640 1 --2- 172.21.15.105:0/1704620139 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f06680d0f10 0x7f06680ccc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:18.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.105+0000 7f06677fe640 1 -- 172.21.15.105:0/1704620139 >> v1:172.21.15.154:6789/0 conn(0x7f06680cd190 legacy=0x7f066809aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:18.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.105+0000 7f06677fe640 1 -- 172.21.15.105:0/1704620139 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f066806db40 con 0x7f066815ecf0 2024-04-15T15:21:18.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.105+0000 7f0666ffd640 1 --2- 172.21.15.105:0/1704620139 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f06680d0f10 0x7f06680ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:21:18.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.106+0000 7f06677fe640 1 --2- 172.21.15.105:0/1704620139 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f066815ecf0 0x7f066806e060 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f0658009870 tx=0x7f065802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46829687b4c82897 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:18.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.106+0000 7f06667fc640 1 -- 172.21.15.105:0/1704620139 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0658035020 con 0x7f066815ecf0 2024-04-15T15:21:18.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.106+0000 7f06667fc640 1 -- 172.21.15.105:0/1704620139 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0658031640 con 0x7f066815ecf0 2024-04-15T15:21:18.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.106+0000 7f06667fc640 1 -- 172.21.15.105:0/1704620139 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065802d040 con 0x7f066815ecf0 2024-04-15T15:21:18.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.107+0000 7f066ddd5640 1 -- 172.21.15.105:0/1704620139 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f066815ecf0 msgr2=0x7f066806e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:18.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.107+0000 7f066ddd5640 1 --2- 172.21.15.105:0/1704620139 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f066815ecf0 0x7f066806e060 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f0658009870 tx=0x7f065802b8f0 comp rx=0 tx=0).stop 2024-04-15T15:21:18.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.107+0000 7f066ddd5640 1 -- 172.21.15.105:0/1704620139 shutdown_connections 2024-04-15T15:21:18.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.107+0000 7f066ddd5640 1 --2- 172.21.15.105:0/1704620139 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f06680d0f10 0x7f06680ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:18.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.107+0000 7f066ddd5640 1 --2- 172.21.15.105:0/1704620139 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f066815ecf0 0x7f066806e060 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:18.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.107+0000 7f066ddd5640 1 -- 172.21.15.105:0/1704620139 >> 172.21.15.105:0/1704620139 conn(0x7f06680d2600 msgr2=0x7f06680d2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:18.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.107+0000 7f066ddd5640 1 -- 172.21.15.105:0/1704620139 shutdown_connections 2024-04-15T15:21:18.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.107+0000 7f066ddd5640 1 -- 172.21.15.105:0/1704620139 wait complete. 2024-04-15T15:21:18.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.108+0000 7f066ddd5640 1 Processor -- start 2024-04-15T15:21:18.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.108+0000 7f066ddd5640 1 -- start start 2024-04-15T15:21:18.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.109+0000 7f066ddd5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06680cd190 0x7f06681f89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:18.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.109+0000 7f066ddd5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06680d0f10 0x7f06680b5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:18.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.109+0000 7f066ddd5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f066815ecf0 0x7f066809acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:18.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.109+0000 7f066ddd5640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f066815f0d0 con 0x7f06680cd190 2024-04-15T15:21:18.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.109+0000 7f066ddd5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f06680d1370 con 0x7f06680d0f10 2024-04-15T15:21:18.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.109+0000 7f066ddd5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f066815f250 con 0x7f066815ecf0 2024-04-15T15:21:18.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.109+0000 7f06677fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06680cd190 0x7f06681f89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:18.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.109+0000 7f06677fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06680cd190 0x7f06681f89c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33468/0 (socket says 172.21.15.105:33468) 2024-04-15T15:21:18.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.109+0000 7f06677fe640 1 -- 172.21.15.105:0/3947132251 learned_addr learned my addr 172.21.15.105:0/3947132251 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:18.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.109+0000 7f0666ffd640 1 --2- 172.21.15.105:0/3947132251 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06680d0f10 0x7f06680b5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:18.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.109+0000 7f0667fff640 1 --2- 172.21.15.105:0/3947132251 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f066815ecf0 0x7f066809acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:18.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.109+0000 7f06677fe640 1 -- 172.21.15.105:0/3947132251 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f066815ecf0 msgr2=0x7f066809acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:18.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.109+0000 7f06677fe640 1 --2- 172.21.15.105:0/3947132251 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f066815ecf0 0x7f066809acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:18.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.109+0000 7f06677fe640 1 -- 172.21.15.105:0/3947132251 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06680d0f10 msgr2=0x7f06680b5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:18.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.109+0000 7f06677fe640 1 --2- 172.21.15.105:0/3947132251 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06680d0f10 0x7f06680b5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:18.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.109+0000 7f06677fe640 1 -- 172.21.15.105:0/3947132251 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06680c6130 con 0x7f06680cd190 2024-04-15T15:21:18.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.109+0000 7f0666ffd640 1 --2- 172.21.15.105:0/3947132251 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06680d0f10 0x7f06680b5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:21:18.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.110+0000 7f06677fe640 1 --2- 172.21.15.105:0/3947132251 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06680cd190 0x7f06681f89c0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f065802b930 tx=0x7f06580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:18.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.110+0000 7f064ffff640 1 -- 172.21.15.105:0/3947132251 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0658040070 con 0x7f06680cd190 2024-04-15T15:21:18.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.110+0000 7f064ffff640 1 -- 172.21.15.105:0/3947132251 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0658039c10 con 0x7f06680cd190 2024-04-15T15:21:18.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.110+0000 7f066ddd5640 1 -- 172.21.15.105:0/3947132251 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06680b6c80 con 0x7f06680cd190 2024-04-15T15:21:18.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.110+0000 7f064ffff640 1 -- 172.21.15.105:0/3947132251 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0658035040 con 0x7f06680cd190 2024-04-15T15:21:18.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.110+0000 7f066ddd5640 1 -- 172.21.15.105:0/3947132251 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06680b5a30 con 0x7f06680cd190 2024-04-15T15:21:18.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.111+0000 7f066ddd5640 1 -- 172.21.15.105:0/3947132251 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0668071710 con 0x7f06680cd190 2024-04-15T15:21:18.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.111+0000 7f064ffff640 1 -- 172.21.15.105:0/3947132251 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f065804b060 con 0x7f06680cd190 2024-04-15T15:21:18.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.112+0000 7f064ffff640 1 --2- 172.21.15.105:0/3947132251 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f065406c980 0x7f065406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:18.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.112+0000 7f064ffff640 1 -- 172.21.15.105:0/3947132251 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5795+0+0 (secure 0 0 0) 0x7f06580ad790 con 0x7f06680cd190 2024-04-15T15:21:18.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.112+0000 7f0666ffd640 1 --2- 172.21.15.105:0/3947132251 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f065406c980 0x7f065406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:18.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.112+0000 7f0666ffd640 1 --2- 172.21.15.105:0/3947132251 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f065406c980 0x7f065406ee40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f06680cc760 tx=0x7f065c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:18.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.116+0000 7f064ffff640 1 -- 172.21.15.105:0/3947132251 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0668071710 con 0x7f06680cd190 2024-04-15T15:21:18.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.330+0000 7f066ddd5640 1 -- 172.21.15.105:0/3947132251 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0668068a40 con 0x7f06680cd190 2024-04-15T15:21:18.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.331+0000 7f064ffff640 1 -- 172.21.15.105:0/3947132251 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+5184 (secure 0 0 0) 0x7f065807bb70 con 0x7f06680cd190 2024-04-15T15:21:18.333 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:21:18.333 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4778,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/4848508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":4848508},{"type":"v1","addr":"172.21.15.154:6835","nonce":4848508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5005,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/1127070021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1127070021},{"type":"v1","addr":"172.21.15.154:6833","nonce":1127070021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5193,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6836/3825669416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3825669416},{"type":"v1","addr":"172.21.15.105:6836","nonce":3825669416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-15T15:20:29.000616+0000","modified":"2024-04-15T15:21:07.677485+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5712":{"gid":5712,"name":"a","rank":0,"incarnation":17,"state":"up:active","state_seq":4,"addr":"172.21.15.105:6837/4212477415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":4212477415},{"type":"v1","addr":"172.21.15.105:6837","nonce":4212477415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5712,"qdb_cluster":[5712]},"id":2}]} 2024-04-15T15:21:18.333 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 20 2024-04-15T15:21:18.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.333+0000 7f066ddd5640 1 -- 172.21.15.105:0/3947132251 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f065406c980 msgr2=0x7f065406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:18.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.333+0000 7f066ddd5640 1 --2- 172.21.15.105:0/3947132251 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f065406c980 0x7f065406ee40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f06680cc760 tx=0x7f065c008040 comp rx=0 tx=0).stop 2024-04-15T15:21:18.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.333+0000 7f066ddd5640 1 -- 172.21.15.105:0/3947132251 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06680cd190 msgr2=0x7f06681f89c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:18.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.333+0000 7f066ddd5640 1 --2- 172.21.15.105:0/3947132251 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06680cd190 0x7f06681f89c0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f065802b930 tx=0x7f06580046c0 comp rx=0 tx=0).stop 2024-04-15T15:21:18.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.333+0000 7f066ddd5640 1 -- 172.21.15.105:0/3947132251 shutdown_connections 2024-04-15T15:21:18.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.333+0000 7f066ddd5640 1 --2- 172.21.15.105:0/3947132251 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f066815ecf0 0x7f066809acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:18.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.333+0000 7f066ddd5640 1 --2- 172.21.15.105:0/3947132251 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f065406c980 0x7f065406ee40 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:18.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.333+0000 7f066ddd5640 1 --2- 172.21.15.105:0/3947132251 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06680d0f10 0x7f06680b5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:18.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.333+0000 7f066ddd5640 1 --2- 172.21.15.105:0/3947132251 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06680cd190 0x7f06681f89c0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:18.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.333+0000 7f066ddd5640 1 -- 172.21.15.105:0/3947132251 >> 172.21.15.105:0/3947132251 conn(0x7f06680d2600 msgr2=0x7f0668070980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:18.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.333+0000 7f066ddd5640 1 -- 172.21.15.105:0/3947132251 shutdown_connections 2024-04-15T15:21:18.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.333+0000 7f066ddd5640 1 -- 172.21.15.105:0/3947132251 wait complete. 2024-04-15T15:21:18.346 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:21:18.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.447+0000 7f0a598f2640 1 Processor -- start 2024-04-15T15:21:18.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.447+0000 7f0a598f2640 1 -- start start 2024-04-15T15:21:18.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.448+0000 7f0a598f2640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0a5415eb60 0x7f0a540d11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:18.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.448+0000 7f0a598f2640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0a540d1710 0x7f0a54162630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:18.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.448+0000 7f0a598f2640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f0a54162b70 0x7f0a540b6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:18.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.448+0000 7f0a598f2640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f0a54163540 con 0x7f0a540d1710 2024-04-15T15:21:18.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.448+0000 7f0a598f2640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f0a540d2fc0 con 0x7f0a54162b70 2024-04-15T15:21:18.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.448+0000 7f0a598f2640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f0a540d1b30 con 0x7f0a5415eb60 2024-04-15T15:21:18.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.448+0000 7f0a4bfff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0a540d1710 0x7f0a54162630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:18.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.448+0000 7f0a4bfff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0a540d1710 0x7f0a54162630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33474/0 (socket says 172.21.15.105:33474) 2024-04-15T15:21:18.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.448+0000 7f0a4bfff640 1 -- 172.21.15.105:0/3505177789 learned_addr learned my addr 172.21.15.105:0/3505177789 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:18.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.448+0000 7f0a537fe640 1 --2- 172.21.15.105:0/3505177789 >> v2:172.21.15.154:3300/0 conn(0x7f0a54162b70 0x7f0a540b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:18.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.448+0000 7f0a52ffd640 1 --2- 172.21.15.105:0/3505177789 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0a5415eb60 0x7f0a540d11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:18.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.448+0000 7f0a4bfff640 1 -- 172.21.15.105:0/3505177789 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0a5415eb60 msgr2=0x7f0a540d11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:18.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.448+0000 7f0a4bfff640 1 --2- 172.21.15.105:0/3505177789 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0a5415eb60 0x7f0a540d11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:18.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.448+0000 7f0a4bfff640 1 -- 172.21.15.105:0/3505177789 >> v2:172.21.15.154:3300/0 conn(0x7f0a54162b70 msgr2=0x7f0a540b6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:18.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.448+0000 7f0a4bfff640 1 --2- 172.21.15.105:0/3505177789 >> v2:172.21.15.154:3300/0 conn(0x7f0a54162b70 0x7f0a540b6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:18.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.448+0000 7f0a4bfff640 1 -- 172.21.15.105:0/3505177789 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a5415ef40 con 0x7f0a540d1710 2024-04-15T15:21:18.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.449+0000 7f0a4bfff640 1 --2- 172.21.15.105:0/3505177789 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a540d1710 0x7f0a54162630 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f0a5406e830 tx=0x7f0a3c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f349b6910bbec49a server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:18.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.449+0000 7f0a527fc640 1 -- 172.21.15.105:0/3505177789 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a3c035020 con 0x7f0a540d1710 2024-04-15T15:21:18.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.449+0000 7f0a527fc640 1 -- 172.21.15.105:0/3505177789 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a3c031870 con 0x7f0a540d1710 2024-04-15T15:21:18.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.449+0000 7f0a527fc640 1 -- 172.21.15.105:0/3505177789 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a3c031c40 con 0x7f0a540d1710 2024-04-15T15:21:18.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.450+0000 7f0a598f2640 1 -- 172.21.15.105:0/3505177789 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a540d1710 msgr2=0x7f0a54162630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:18.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.450+0000 7f0a598f2640 1 --2- 172.21.15.105:0/3505177789 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a540d1710 0x7f0a54162630 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f0a5406e830 tx=0x7f0a3c02b910 comp rx=0 tx=0).stop 2024-04-15T15:21:18.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.450+0000 7f0a598f2640 1 -- 172.21.15.105:0/3505177789 shutdown_connections 2024-04-15T15:21:18.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.450+0000 7f0a598f2640 1 --2- 172.21.15.105:0/3505177789 >> v2:172.21.15.154:3300/0 conn(0x7f0a54162b70 0x7f0a540b6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:18.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.450+0000 7f0a598f2640 1 --2- 172.21.15.105:0/3505177789 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a540d1710 0x7f0a54162630 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:18.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.450+0000 7f0a598f2640 1 --2- 172.21.15.105:0/3505177789 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0a5415eb60 0x7f0a540d11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:18.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.450+0000 7f0a598f2640 1 -- 172.21.15.105:0/3505177789 >> 172.21.15.105:0/3505177789 conn(0x7f0a54067720 msgr2=0x7f0a54067b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:18.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.450+0000 7f0a598f2640 1 -- 172.21.15.105:0/3505177789 shutdown_connections 2024-04-15T15:21:18.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.450+0000 7f0a598f2640 1 -- 172.21.15.105:0/3505177789 wait complete. 2024-04-15T15:21:18.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.451+0000 7f0a598f2640 1 Processor -- start 2024-04-15T15:21:18.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.451+0000 7f0a598f2640 1 -- start start 2024-04-15T15:21:18.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.451+0000 7f0a598f2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a540d1710 0x7f0a540cd590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:18.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.451+0000 7f0a598f2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a5415eb60 0x7f0a540cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:18.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.452+0000 7f0a598f2640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0a54162b70 0x7f0a5414ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:18.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.452+0000 7f0a598f2640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0a540d2970 con 0x7f0a540d1710 2024-04-15T15:21:18.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.452+0000 7f0a598f2640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0a54162fc0 con 0x7f0a5415eb60 2024-04-15T15:21:18.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.452+0000 7f0a598f2640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f0a540d2c50 con 0x7f0a54162b70 2024-04-15T15:21:18.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.452+0000 7f0a52ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a540d1710 0x7f0a540cd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:18.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.452+0000 7f0a52ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a540d1710 0x7f0a540cd590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33488/0 (socket says 172.21.15.105:33488) 2024-04-15T15:21:18.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.452+0000 7f0a52ffd640 1 -- 172.21.15.105:0/2880641505 learned_addr learned my addr 172.21.15.105:0/2880641505 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:18.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.452+0000 7f0a4bfff640 1 --2- 172.21.15.105:0/2880641505 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a5415eb60 0x7f0a540cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:18.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.452+0000 7f0a537fe640 1 --2- 172.21.15.105:0/2880641505 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0a54162b70 0x7f0a5414ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:18.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.452+0000 7f0a52ffd640 1 -- 172.21.15.105:0/2880641505 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0a54162b70 msgr2=0x7f0a5414ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:18.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.452+0000 7f0a52ffd640 1 --2- 172.21.15.105:0/2880641505 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0a54162b70 0x7f0a5414ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:18.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.452+0000 7f0a52ffd640 1 -- 172.21.15.105:0/2880641505 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a5415eb60 msgr2=0x7f0a540cdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:18.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.452+0000 7f0a52ffd640 1 --2- 172.21.15.105:0/2880641505 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a5415eb60 0x7f0a540cdad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:18.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.452+0000 7f0a52ffd640 1 -- 172.21.15.105:0/2880641505 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a540ce0e0 con 0x7f0a540d1710 2024-04-15T15:21:18.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.452+0000 7f0a52ffd640 1 --2- 172.21.15.105:0/2880641505 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a540d1710 0x7f0a540cd590 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f0a4400d800 tx=0x7f0a4400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:18.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.452+0000 7f0a50ff9640 1 -- 172.21.15.105:0/2880641505 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a44018020 con 0x7f0a540d1710 2024-04-15T15:21:18.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.452+0000 7f0a50ff9640 1 -- 172.21.15.105:0/2880641505 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a440096a0 con 0x7f0a540d1710 2024-04-15T15:21:18.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.452+0000 7f0a50ff9640 1 -- 172.21.15.105:0/2880641505 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a44009a70 con 0x7f0a540d1710 2024-04-15T15:21:18.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.452+0000 7f0a598f2640 1 -- 172.21.15.105:0/2880641505 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a540ce3c0 con 0x7f0a540d1710 2024-04-15T15:21:18.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.452+0000 7f0a598f2640 1 -- 172.21.15.105:0/2880641505 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a540ce980 con 0x7f0a540d1710 2024-04-15T15:21:18.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.453+0000 7f0a598f2640 1 -- 172.21.15.105:0/2880641505 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a540cebc0 con 0x7f0a540d1710 2024-04-15T15:21:18.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.454+0000 7f0a50ff9640 1 -- 172.21.15.105:0/2880641505 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f0a4401b020 con 0x7f0a540d1710 2024-04-15T15:21:18.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.454+0000 7f0a50ff9640 1 --2- 172.21.15.105:0/2880641505 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f0a3406c9b0 0x7f0a3406ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:18.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.454+0000 7f0a4bfff640 1 --2- 172.21.15.105:0/2880641505 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f0a3406c9b0 0x7f0a3406ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:18.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.454+0000 7f0a50ff9640 1 -- 172.21.15.105:0/2880641505 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5795+0+0 (secure 0 0 0) 0x7f0a4408be50 con 0x7f0a540d1710 2024-04-15T15:21:18.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.455+0000 7f0a4bfff640 1 --2- 172.21.15.105:0/2880641505 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f0a3406c9b0 0x7f0a3406ee70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c036000 tx=0x7f0a3c03c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:18.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.459+0000 7f0a50ff9640 1 -- 172.21.15.105:0/2880641505 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0a540cebc0 con 0x7f0a540d1710 2024-04-15T15:21:18.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.635+0000 7f0a598f2640 1 -- 172.21.15.105:0/2880641505 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0a54068a40 con 0x7f0a540d1710 2024-04-15T15:21:18.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.636+0000 7f0a50ff9640 1 -- 172.21.15.105:0/2880641505 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16846 (secure 0 0 0) 0x7f0a44022030 con 0x7f0a540d1710 2024-04-15T15:21:18.638 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:21:18.638 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":35,"fsid":"09e4d2a4-a324-4793-af77-644b25e520e1","created":"2024-04-15T15:19:01.524543+0000","modified":"2024-04-15T15:21:03.660870+0000","last_up_change":"2024-04-15T15:19:12.825815+0000","last_in_change":"2024-04-15T15:19:03.788126+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:19:13.748585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:20:26.983353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-15T15:20:27.961802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6f6292b-2d7e-4aa0-9d1c-6ddd0f97287c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6801","nonce":2873040054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6803","nonce":2873040054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6807","nonce":2873040054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6805","nonce":2873040054}]},"public_addr":"172.21.15.105:6801/2873040054","cluster_addr":"172.21.15.105:6803/2873040054","heartbeat_back_addr":"172.21.15.105:6807/2873040054","heartbeat_front_addr":"172.21.15.105:6805/2873040054","state":["exists","up"]},{"osd":1,"uuid":"8900966a-b393-41e3-8f93-e5a43bd23622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6818","nonce":2494994479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6821","nonce":2494994479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6828","nonce":2494994479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6823","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6825","nonce":2494994479}]},"public_addr":"172.21.15.105:6818/2494994479","cluster_addr":"172.21.15.105:6821/2494994479","heartbeat_back_addr":"172.21.15.105:6828/2494994479","heartbeat_front_addr":"172.21.15.105:6825/2494994479","state":["exists","up"]},{"osd":2,"uuid":"81a5dc7a-9325-4ff5-941d-a2d5fba21f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6809","nonce":3798408699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6811","nonce":3798408699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6815","nonce":3798408699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6813","nonce":3798408699}]},"public_addr":"172.21.15.105:6809/3798408699","cluster_addr":"172.21.15.105:6811/3798408699","heartbeat_back_addr":"172.21.15.105:6815/3798408699","heartbeat_front_addr":"172.21.15.105:6813/3798408699","state":["exists","up"]},{"osd":3,"uuid":"df766ed2-5c71-4805-96f4-3d4970edc960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6819","nonce":1920720186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6824","nonce":1920720186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6831","nonce":1920720186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6827","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6829","nonce":1920720186}]},"public_addr":"172.21.15.105:6819/1920720186","cluster_addr":"172.21.15.105:6824/1920720186","heartbeat_back_addr":"172.21.15.105:6831/1920720186","heartbeat_front_addr":"172.21.15.105:6829/1920720186","state":["exists","up"]},{"osd":4,"uuid":"e1039e03-28b8-49df-b951-c91930b93c9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6810","nonce":3516494456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6813","nonce":3516494456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6820","nonce":3516494456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6817","nonce":3516494456}]},"public_addr":"172.21.15.154:6810/3516494456","cluster_addr":"172.21.15.154:6813/3516494456","heartbeat_back_addr":"172.21.15.154:6820/3516494456","heartbeat_front_addr":"172.21.15.154:6817/3516494456","state":["exists","up"]},{"osd":5,"uuid":"a9f5eaf1-66ff-405a-a9e4-ec3340812a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6801","nonce":540367751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6803","nonce":540367751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6807","nonce":540367751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6805","nonce":540367751}]},"public_addr":"172.21.15.154:6801/540367751","cluster_addr":"172.21.15.154:6803/540367751","heartbeat_back_addr":"172.21.15.154:6807/540367751","heartbeat_front_addr":"172.21.15.154:6805/540367751","state":["exists","up"]},{"osd":6,"uuid":"267d02ea-675f-428f-aae8-e0375e4d4feb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6825","nonce":1550829005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6827","nonce":1550829005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6831","nonce":1550829005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6829","nonce":1550829005}]},"public_addr":"172.21.15.154:6825/1550829005","cluster_addr":"172.21.15.154:6827/1550829005","heartbeat_back_addr":"172.21.15.154:6831/1550829005","heartbeat_front_addr":"172.21.15.154:6829/1550829005","state":["exists","up"]},{"osd":7,"uuid":"759ac6cc-896a-423c-885f-9fe90d8dcb14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6809","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6811","nonce":3920089903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6816","nonce":3920089903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6823","nonce":3920089903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6819","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6821","nonce":3920089903}]},"public_addr":"172.21.15.154:6811/3920089903","cluster_addr":"172.21.15.154:6816/3920089903","heartbeat_back_addr":"172.21.15.154:6823/3920089903","heartbeat_front_addr":"172.21.15.154:6821/3920089903","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.949509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.984072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.907278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.947391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.975447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.998896+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:09.028739+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.894119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.105:6837/958066041":"2024-04-16T15:21:03.660841+0000","172.21.15.105:6835/958066041":"2024-04-16T15:21:03.660841+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:21:18.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.638+0000 7f0a598f2640 1 -- 172.21.15.105:0/2880641505 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f0a3406c9b0 msgr2=0x7f0a3406ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:18.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.638+0000 7f0a598f2640 1 --2- 172.21.15.105:0/2880641505 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f0a3406c9b0 0x7f0a3406ee70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c036000 tx=0x7f0a3c03c040 comp rx=0 tx=0).stop 2024-04-15T15:21:18.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.638+0000 7f0a598f2640 1 -- 172.21.15.105:0/2880641505 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a540d1710 msgr2=0x7f0a540cd590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:18.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.638+0000 7f0a598f2640 1 --2- 172.21.15.105:0/2880641505 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a540d1710 0x7f0a540cd590 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f0a4400d800 tx=0x7f0a4400dcc0 comp rx=0 tx=0).stop 2024-04-15T15:21:18.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.638+0000 7f0a598f2640 1 -- 172.21.15.105:0/2880641505 shutdown_connections 2024-04-15T15:21:18.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.638+0000 7f0a598f2640 1 --2- 172.21.15.105:0/2880641505 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0a54162b70 0x7f0a5414ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:18.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.638+0000 7f0a598f2640 1 --2- 172.21.15.105:0/2880641505 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f0a3406c9b0 0x7f0a3406ee70 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:18.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.638+0000 7f0a598f2640 1 --2- 172.21.15.105:0/2880641505 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a5415eb60 0x7f0a540cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:18.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.638+0000 7f0a598f2640 1 --2- 172.21.15.105:0/2880641505 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a540d1710 0x7f0a540cd590 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:18.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.638+0000 7f0a598f2640 1 -- 172.21.15.105:0/2880641505 >> 172.21.15.105:0/2880641505 conn(0x7f0a54067720 msgr2=0x7f0a540b8990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:18.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.638+0000 7f0a598f2640 1 -- 172.21.15.105:0/2880641505 shutdown_connections 2024-04-15T15:21:18.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.638+0000 7f0a598f2640 1 -- 172.21.15.105:0/2880641505 wait complete. 2024-04-15T15:21:18.652 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-15T15:21:18.652 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-15T15:21:18.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.755+0000 7f1708fba640 1 Processor -- start 2024-04-15T15:21:18.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.755+0000 7f1708fba640 1 -- start start 2024-04-15T15:21:18.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.756+0000 7f1708fba640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f17041554d0 0x7f17040afaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:18.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.756+0000 7f1708fba640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f17040726f0 0x7f1704072af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:18.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.756+0000 7f1708fba640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17040b5a30 con 0x7f17040b5580 2024-04-15T15:21:18.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.756+0000 7f1708fba640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f17040cf280 con 0x7f17041554d0 2024-04-15T15:21:18.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.756+0000 7f1708fba640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f1704073030 con 0x7f17040726f0 2024-04-15T15:21:18.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.756+0000 7f170259c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f17041554d0 0x7f17040afaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:18.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.756+0000 7f170259c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f17041554d0 0x7f17040afaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33492/0 (socket says 172.21.15.105:33492) 2024-04-15T15:21:18.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.756+0000 7f170259c640 1 -- 172.21.15.105:0/2920941271 learned_addr learned my addr 172.21.15.105:0/2920941271 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:18.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.756+0000 7f1702d9d640 1 --2- 172.21.15.105:0/2920941271 >> v2:172.21.15.154:3300/0 conn(0x7f17040726f0 0x7f1704072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:18.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.756+0000 7f170259c640 1 -- 172.21.15.105:0/2920941271 >> v2:172.21.15.154:3300/0 conn(0x7f17040726f0 msgr2=0x7f1704072af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:18.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.756+0000 7f170259c640 1 --2- 172.21.15.105:0/2920941271 >> v2:172.21.15.154:3300/0 conn(0x7f17040726f0 0x7f1704072af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:18.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.756+0000 7f170259c640 1 -- 172.21.15.105:0/2920941271 >> v1:172.21.15.154:6789/0 conn(0x7f17040b5580 legacy=0x7f17040cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:18.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.756+0000 7f170259c640 1 -- 172.21.15.105:0/2920941271 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f170415bf70 con 0x7f17041554d0 2024-04-15T15:21:18.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.757+0000 7f170259c640 1 --2- 172.21.15.105:0/2920941271 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17041554d0 0x7f17040afaa0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f1704158f10 tx=0x7f16f402e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d965d47e732f978d server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:18.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.757+0000 7f1701d9b640 1 -- 172.21.15.105:0/2920941271 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f403b070 con 0x7f17041554d0 2024-04-15T15:21:18.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.757+0000 7f1701d9b640 1 -- 172.21.15.105:0/2920941271 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16f4036440 con 0x7f17041554d0 2024-04-15T15:21:18.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.757+0000 7f1701d9b640 1 -- 172.21.15.105:0/2920941271 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f4036810 con 0x7f17041554d0 2024-04-15T15:21:18.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.758+0000 7f1708fba640 1 -- 172.21.15.105:0/2920941271 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17041554d0 msgr2=0x7f17040afaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:18.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.758+0000 7f1708fba640 1 --2- 172.21.15.105:0/2920941271 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17041554d0 0x7f17040afaa0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f1704158f10 tx=0x7f16f402e0e0 comp rx=0 tx=0).stop 2024-04-15T15:21:18.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.758+0000 7f1708fba640 1 -- 172.21.15.105:0/2920941271 shutdown_connections 2024-04-15T15:21:18.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.758+0000 7f1708fba640 1 --2- 172.21.15.105:0/2920941271 >> v2:172.21.15.154:3300/0 conn(0x7f17040726f0 0x7f1704072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:18.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.758+0000 7f1708fba640 1 --2- 172.21.15.105:0/2920941271 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17041554d0 0x7f17040afaa0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:18.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.758+0000 7f1708fba640 1 -- 172.21.15.105:0/2920941271 >> 172.21.15.105:0/2920941271 conn(0x7f1704067d50 msgr2=0x7f170406b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:18.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.758+0000 7f1708fba640 1 -- 172.21.15.105:0/2920941271 shutdown_connections 2024-04-15T15:21:18.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.758+0000 7f1708fba640 1 -- 172.21.15.105:0/2920941271 wait complete. 2024-04-15T15:21:18.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.759+0000 7f1708fba640 1 Processor -- start 2024-04-15T15:21:18.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.759+0000 7f1708fba640 1 -- start start 2024-04-15T15:21:18.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.760+0000 7f1708fba640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17040726f0 0x7f17041f3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:18.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.760+0000 7f1708fba640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17040b5580 0x7f17040ad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:18.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.760+0000 7f1708fba640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f17041554d0 0x7f17040addf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:18.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.760+0000 7f1708fba640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f17040cfe20 con 0x7f17040726f0 2024-04-15T15:21:18.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.760+0000 7f1708fba640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f17040cef10 con 0x7f17040b5580 2024-04-15T15:21:18.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.760+0000 7f1708fba640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f170415c250 con 0x7f17041554d0 2024-04-15T15:21:18.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.760+0000 7f170259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17040726f0 0x7f17041f3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:18.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.760+0000 7f170259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17040726f0 0x7f17041f3c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33500/0 (socket says 172.21.15.105:33500) 2024-04-15T15:21:18.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.760+0000 7f170259c640 1 -- 172.21.15.105:0/430900045 learned_addr learned my addr 172.21.15.105:0/430900045 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:18.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.760+0000 7f170259c640 1 -- 172.21.15.105:0/430900045 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f17041554d0 msgr2=0x7f17040addf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:18.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.760+0000 7f170259c640 1 --2- 172.21.15.105:0/430900045 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f17041554d0 0x7f17040addf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:18.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.760+0000 7f170259c640 1 -- 172.21.15.105:0/430900045 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17040b5580 msgr2=0x7f17040ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:18.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.760+0000 7f170259c640 1 --2- 172.21.15.105:0/430900045 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17040b5580 0x7f17040ad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:18.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.760+0000 7f170259c640 1 -- 172.21.15.105:0/430900045 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17040ae420 con 0x7f17040726f0 2024-04-15T15:21:18.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.760+0000 7f170259c640 1 --2- 172.21.15.105:0/430900045 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17040726f0 0x7f17041f3c50 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f16f402e120 tx=0x7f16f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:18.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.760+0000 7f16faffd640 1 -- 172.21.15.105:0/430900045 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f4046070 con 0x7f17040726f0 2024-04-15T15:21:18.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.760+0000 7f16faffd640 1 -- 172.21.15.105:0/430900045 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16f403d050 con 0x7f17040726f0 2024-04-15T15:21:18.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.761+0000 7f1708fba640 1 -- 172.21.15.105:0/430900045 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17040ae700 con 0x7f17040726f0 2024-04-15T15:21:18.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.761+0000 7f16faffd640 1 -- 172.21.15.105:0/430900045 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f403b030 con 0x7f17040726f0 2024-04-15T15:21:18.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.761+0000 7f1708fba640 1 -- 172.21.15.105:0/430900045 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1704156d10 con 0x7f17040726f0 2024-04-15T15:21:18.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.762+0000 7f1708fba640 1 -- 172.21.15.105:0/430900045 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1704068a40 con 0x7f17040726f0 2024-04-15T15:21:18.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.762+0000 7f16faffd640 1 -- 172.21.15.105:0/430900045 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f16f4049070 con 0x7f17040726f0 2024-04-15T15:21:18.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.763+0000 7f16faffd640 1 --2- 172.21.15.105:0/430900045 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f16e406c980 0x7f16e406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:18.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.763+0000 7f16faffd640 1 -- 172.21.15.105:0/430900045 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5795+0+0 (secure 0 0 0) 0x7f16f40adac0 con 0x7f17040726f0 2024-04-15T15:21:18.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.763+0000 7f16fbfff640 1 --2- 172.21.15.105:0/430900045 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f16e406c980 0x7f16e406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:18.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.763+0000 7f16fbfff640 1 --2- 172.21.15.105:0/430900045 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f16e406c980 0x7f16e406ee40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f17040cd740 tx=0x7f16ec0079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:18.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.767+0000 7f16faffd640 1 -- 172.21.15.105:0/430900045 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f16f407be20 con 0x7f17040726f0 2024-04-15T15:21:18.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.978+0000 7f1708fba640 1 -- 172.21.15.105:0/430900045 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f17040b5000 con 0x7f17040726f0 2024-04-15T15:21:18.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.978+0000 7f16faffd640 1 -- 172.21.15.105:0/430900045 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+5184 (secure 0 0 0) 0x7f16f407b720 con 0x7f17040726f0 2024-04-15T15:21:18.980 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:21:18.980 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4778,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/4848508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":4848508},{"type":"v1","addr":"172.21.15.154:6835","nonce":4848508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5005,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/1127070021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1127070021},{"type":"v1","addr":"172.21.15.154:6833","nonce":1127070021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5193,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6836/3825669416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3825669416},{"type":"v1","addr":"172.21.15.105:6836","nonce":3825669416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-15T15:20:29.000616+0000","modified":"2024-04-15T15:21:07.677485+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5712":{"gid":5712,"name":"a","rank":0,"incarnation":17,"state":"up:active","state_seq":4,"addr":"172.21.15.105:6837/4212477415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":4212477415},{"type":"v1","addr":"172.21.15.105:6837","nonce":4212477415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5712,"qdb_cluster":[5712]},"id":2}]} 2024-04-15T15:21:18.981 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 20 2024-04-15T15:21:18.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.980+0000 7f1708fba640 1 -- 172.21.15.105:0/430900045 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f16e406c980 msgr2=0x7f16e406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:18.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.980+0000 7f1708fba640 1 --2- 172.21.15.105:0/430900045 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f16e406c980 0x7f16e406ee40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f17040cd740 tx=0x7f16ec0079b0 comp rx=0 tx=0).stop 2024-04-15T15:21:18.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.980+0000 7f1708fba640 1 -- 172.21.15.105:0/430900045 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17040726f0 msgr2=0x7f17041f3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:18.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.980+0000 7f1708fba640 1 --2- 172.21.15.105:0/430900045 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17040726f0 0x7f17041f3c50 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f16f402e120 tx=0x7f16f40046c0 comp rx=0 tx=0).stop 2024-04-15T15:21:18.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.980+0000 7f1708fba640 1 -- 172.21.15.105:0/430900045 shutdown_connections 2024-04-15T15:21:18.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.980+0000 7f1708fba640 1 --2- 172.21.15.105:0/430900045 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f17041554d0 0x7f17040addf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:18.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.980+0000 7f1708fba640 1 --2- 172.21.15.105:0/430900045 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f16e406c980 0x7f16e406ee40 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:18.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.980+0000 7f1708fba640 1 --2- 172.21.15.105:0/430900045 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17040b5580 0x7f17040ad8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:18.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.980+0000 7f1708fba640 1 --2- 172.21.15.105:0/430900045 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17040726f0 0x7f17041f3c50 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:18.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.980+0000 7f1708fba640 1 -- 172.21.15.105:0/430900045 >> 172.21.15.105:0/430900045 conn(0x7f1704067d50 msgr2=0x7f17041540f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:18.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.980+0000 7f1708fba640 1 -- 172.21.15.105:0/430900045 shutdown_connections 2024-04-15T15:21:18.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:18.980+0000 7f1708fba640 1 -- 172.21.15.105:0/430900045 wait complete. 2024-04-15T15:21:18.993 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-15T15:21:19.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.094+0000 7f74baff4640 1 Processor -- start 2024-04-15T15:21:19.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.094+0000 7f74baff4640 1 -- start start 2024-04-15T15:21:19.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.095+0000 7f74baff4640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f74b4071e60 0x7f74b40722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:19.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.095+0000 7f74baff4640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74b406df80 0x7f74b406e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:19.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.095+0000 7f74baff4640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74b40ccfb0 con 0x7f74b4145e00 2024-04-15T15:21:19.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.095+0000 7f74baff4640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f74b4072800 con 0x7f74b4071e60 2024-04-15T15:21:19.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.095+0000 7f74baff4640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f74b40cdbc0 con 0x7f74b406df80 2024-04-15T15:21:19.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.095+0000 7f74b9591640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74b406df80 0x7f74b406e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:19.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.095+0000 7f74abfff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f74b4071e60 0x7f74b40722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:19.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.095+0000 7f74b8d90640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f74b4145e00 0x7f74b40ce7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.105:36354/0 (socket says 172.21.15.105:36354) 2024-04-15T15:21:19.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.095+0000 7f74b8d90640 1 -- 172.21.15.105:0/2284591732 learned_addr learned my addr 172.21.15.105:0/2284591732 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:21:19.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.095+0000 7f74b9591640 1 --2- 172.21.15.105:0/2284591732 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74b406df80 0x7f74b406e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.105:46190/0 (socket says 172.21.15.105:46190) 2024-04-15T15:21:19.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.096+0000 7f74b9591640 1 -- 172.21.15.105:0/2284591732 >> v2:172.21.15.154:3300/0 conn(0x7f74b4071e60 msgr2=0x7f74b40722c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:19.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.096+0000 7f74b9591640 1 --2- 172.21.15.105:0/2284591732 >> v2:172.21.15.154:3300/0 conn(0x7f74b4071e60 0x7f74b40722c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:19.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.096+0000 7f74b9591640 1 -- 172.21.15.105:0/2284591732 >> v1:172.21.15.154:6789/0 conn(0x7f74b4145e00 legacy=0x7f74b40ce7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:19.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.096+0000 7f74b9591640 1 -- 172.21.15.105:0/2284591732 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74b40cd290 con 0x7f74b406df80 2024-04-15T15:21:19.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.096+0000 7f74abfff640 1 --2- 172.21.15.105:0/2284591732 >> v2:172.21.15.154:3300/0 conn(0x7f74b4071e60 0x7f74b40722c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:21:19.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.096+0000 7f74b9591640 1 --2- 172.21.15.105:0/2284591732 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74b406df80 0x7f74b406e3d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f74ac009d90 tx=0x7f74ac02bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e76f208b43ca48b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:19.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.097+0000 7f74ab7fe640 1 -- 172.21.15.105:0/2284591732 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ac039070 con 0x7f74b406df80 2024-04-15T15:21:19.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.097+0000 7f74ab7fe640 1 -- 172.21.15.105:0/2284591732 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74ac02cc50 con 0x7f74b406df80 2024-04-15T15:21:19.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.097+0000 7f74ab7fe640 1 -- 172.21.15.105:0/2284591732 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ac034440 con 0x7f74b406df80 2024-04-15T15:21:19.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.097+0000 7f74baff4640 1 -- 172.21.15.105:0/2284591732 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74b406df80 msgr2=0x7f74b406e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:19.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.097+0000 7f74baff4640 1 --2- 172.21.15.105:0/2284591732 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74b406df80 0x7f74b406e3d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f74ac009d90 tx=0x7f74ac02bea0 comp rx=0 tx=0).stop 2024-04-15T15:21:19.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.097+0000 7f74baff4640 1 -- 172.21.15.105:0/2284591732 shutdown_connections 2024-04-15T15:21:19.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.097+0000 7f74baff4640 1 --2- 172.21.15.105:0/2284591732 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74b406df80 0x7f74b406e3d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:19.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.097+0000 7f74baff4640 1 --2- 172.21.15.105:0/2284591732 >> v2:172.21.15.154:3300/0 conn(0x7f74b4071e60 0x7f74b40722c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:19.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.097+0000 7f74baff4640 1 -- 172.21.15.105:0/2284591732 >> 172.21.15.105:0/2284591732 conn(0x7f74b4067d50 msgr2=0x7f74b406b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:19.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.097+0000 7f74baff4640 1 -- 172.21.15.105:0/2284591732 shutdown_connections 2024-04-15T15:21:19.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.097+0000 7f74baff4640 1 -- 172.21.15.105:0/2284591732 wait complete. 2024-04-15T15:21:19.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.098+0000 7f74baff4640 1 Processor -- start 2024-04-15T15:21:19.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.098+0000 7f74baff4640 1 -- start start 2024-04-15T15:21:19.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.099+0000 7f74baff4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f74b406df80 0x7f74b40a9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:19.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.099+0000 7f74baff4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74b4071e60 0x7f74b4137780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:19.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.099+0000 7f74baff4640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74b4145e00 0x7f74b4137cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:19.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.099+0000 7f74baff4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f74b40cef00 con 0x7f74b4071e60 2024-04-15T15:21:19.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.099+0000 7f74baff4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f74b40cd850 con 0x7f74b406df80 2024-04-15T15:21:19.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.099+0000 7f74baff4640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f74b40ce370 con 0x7f74b4145e00 2024-04-15T15:21:19.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.099+0000 7f74abfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74b4071e60 0x7f74b4137780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:19.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.099+0000 7f74abfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74b4071e60 0x7f74b4137780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33516/0 (socket says 172.21.15.105:33516) 2024-04-15T15:21:19.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.099+0000 7f74abfff640 1 -- 172.21.15.105:0/2306442891 learned_addr learned my addr 172.21.15.105:0/2306442891 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:19.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.100+0000 7f74b8d90640 1 --2- 172.21.15.105:0/2306442891 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f74b406df80 0x7f74b40a9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:19.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.100+0000 7f74b9591640 1 --2- 172.21.15.105:0/2306442891 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74b4145e00 0x7f74b4137cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:19.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.100+0000 7f74abfff640 1 -- 172.21.15.105:0/2306442891 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74b4145e00 msgr2=0x7f74b4137cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:19.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.100+0000 7f74abfff640 1 --2- 172.21.15.105:0/2306442891 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74b4145e00 0x7f74b4137cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:19.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.100+0000 7f74abfff640 1 -- 172.21.15.105:0/2306442891 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f74b406df80 msgr2=0x7f74b40a9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:19.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.100+0000 7f74abfff640 1 --2- 172.21.15.105:0/2306442891 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f74b406df80 0x7f74b40a9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:19.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.100+0000 7f74abfff640 1 -- 172.21.15.105:0/2306442891 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74b41f51a0 con 0x7f74b4071e60 2024-04-15T15:21:19.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.100+0000 7f74b9591640 1 --2- 172.21.15.105:0/2306442891 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74b4145e00 0x7f74b4137cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:21:19.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.100+0000 7f74abfff640 1 --2- 172.21.15.105:0/2306442891 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74b4071e60 0x7f74b4137780 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f74a400ebe0 tx=0x7f74a400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:19.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.100+0000 7f74a97fa640 1 -- 172.21.15.105:0/2306442891 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a4017070 con 0x7f74b4071e60 2024-04-15T15:21:19.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.100+0000 7f74a97fa640 1 -- 172.21.15.105:0/2306442891 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74a400cc30 con 0x7f74b4071e60 2024-04-15T15:21:19.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.100+0000 7f74baff4640 1 -- 172.21.15.105:0/2306442891 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74b41f5480 con 0x7f74b4071e60 2024-04-15T15:21:19.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.100+0000 7f74a97fa640 1 -- 172.21.15.105:0/2306442891 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a4013470 con 0x7f74b4071e60 2024-04-15T15:21:19.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.100+0000 7f74baff4640 1 -- 172.21.15.105:0/2306442891 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74b41f5a40 con 0x7f74b4071e60 2024-04-15T15:21:19.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.101+0000 7f74baff4640 1 -- 172.21.15.105:0/2306442891 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74b4068a40 con 0x7f74b4071e60 2024-04-15T15:21:19.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.102+0000 7f74a97fa640 1 -- 172.21.15.105:0/2306442891 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f74a401a070 con 0x7f74b4071e60 2024-04-15T15:21:19.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.102+0000 7f74a97fa640 1 --2- 172.21.15.105:0/2306442891 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f74a006c9b0 0x7f74a006ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:19.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.102+0000 7f74a97fa640 1 -- 172.21.15.105:0/2306442891 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5795+0+0 (secure 0 0 0) 0x7f74a408be20 con 0x7f74b4071e60 2024-04-15T15:21:19.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.102+0000 7f74b8d90640 1 --2- 172.21.15.105:0/2306442891 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f74a006c9b0 0x7f74a006ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:19.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.102+0000 7f74b8d90640 1 --2- 172.21.15.105:0/2306442891 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f74a006c9b0 0x7f74a006ee70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f74b0000f60 tx=0x7f74b0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:19.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.106+0000 7f74a97fa640 1 -- 172.21.15.105:0/2306442891 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f74a405a290 con 0x7f74b4071e60 2024-04-15T15:21:19.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.319+0000 7f74baff4640 1 -- 172.21.15.105:0/2306442891 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f74b40aff30 con 0x7f74b4071e60 2024-04-15T15:21:19.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.320+0000 7f74a97fa640 1 -- 172.21.15.105:0/2306442891 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+5184 (secure 0 0 0) 0x7f74a4059b90 con 0x7f74b4071e60 2024-04-15T15:21:19.322 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:21:19.322 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4778,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/4848508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":4848508},{"type":"v1","addr":"172.21.15.154:6835","nonce":4848508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5005,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/1127070021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1127070021},{"type":"v1","addr":"172.21.15.154:6833","nonce":1127070021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5193,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6836/3825669416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3825669416},{"type":"v1","addr":"172.21.15.105:6836","nonce":3825669416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-15T15:20:29.000616+0000","modified":"2024-04-15T15:21:07.677485+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5712":{"gid":5712,"name":"a","rank":0,"incarnation":17,"state":"up:active","state_seq":4,"addr":"172.21.15.105:6837/4212477415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":4212477415},{"type":"v1","addr":"172.21.15.105:6837","nonce":4212477415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5712,"qdb_cluster":[5712]},"id":2}]} 2024-04-15T15:21:19.322 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 20 2024-04-15T15:21:19.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.322+0000 7f74baff4640 1 -- 172.21.15.105:0/2306442891 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f74a006c9b0 msgr2=0x7f74a006ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:19.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.322+0000 7f74baff4640 1 --2- 172.21.15.105:0/2306442891 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f74a006c9b0 0x7f74a006ee70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f74b0000f60 tx=0x7f74b0009040 comp rx=0 tx=0).stop 2024-04-15T15:21:19.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.322+0000 7f74baff4640 1 -- 172.21.15.105:0/2306442891 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74b4071e60 msgr2=0x7f74b4137780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:19.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.322+0000 7f74baff4640 1 --2- 172.21.15.105:0/2306442891 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74b4071e60 0x7f74b4137780 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f74a400ebe0 tx=0x7f74a400c520 comp rx=0 tx=0).stop 2024-04-15T15:21:19.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.322+0000 7f74baff4640 1 -- 172.21.15.105:0/2306442891 shutdown_connections 2024-04-15T15:21:19.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.322+0000 7f74baff4640 1 --2- 172.21.15.105:0/2306442891 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74b4145e00 0x7f74b4137cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:19.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.322+0000 7f74baff4640 1 --2- 172.21.15.105:0/2306442891 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f74a006c9b0 0x7f74a006ee70 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:19.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.322+0000 7f74baff4640 1 --2- 172.21.15.105:0/2306442891 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74b4071e60 0x7f74b4137780 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:19.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.322+0000 7f74baff4640 1 --2- 172.21.15.105:0/2306442891 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f74b406df80 0x7f74b40a9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:19.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.322+0000 7f74baff4640 1 -- 172.21.15.105:0/2306442891 >> 172.21.15.105:0/2306442891 conn(0x7f74b4067d50 msgr2=0x7f74b415e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:19.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.322+0000 7f74baff4640 1 -- 172.21.15.105:0/2306442891 shutdown_connections 2024-04-15T15:21:19.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.322+0000 7f74baff4640 1 -- 172.21.15.105:0/2306442891 wait complete. 2024-04-15T15:21:19.335 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:21:19.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.436+0000 7fcfd62a5640 1 Processor -- start 2024-04-15T15:21:19.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.436+0000 7fcfd62a5640 1 -- start start 2024-04-15T15:21:19.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.436+0000 7fcfd62a5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcfd0162d40 0x7fcfd0169960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:19.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.436+0000 7fcfd62a5640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfd015ef40 con 0x7fcfd015eb60 2024-04-15T15:21:19.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.436+0000 7fcfd62a5640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfd0163260 con 0x7fcfd00d18e0 2024-04-15T15:21:19.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.436+0000 7fcfd62a5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fcfd0163540 con 0x7fcfd0162d40 2024-04-15T15:21:19.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.437+0000 7fcfcffff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fcfd015eb60 0x7fcfd00d11d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:37628/0 (socket says 172.21.15.105:37628) 2024-04-15T15:21:19.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.437+0000 7fcfcffff640 1 -- 172.21.15.105:0/1440443753 learned_addr learned my addr 172.21.15.105:0/1440443753 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:21:19.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.437+0000 7fcfd4842640 1 --2- 172.21.15.105:0/1440443753 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcfd0162d40 0x7fcfd0169960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:19.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.437+0000 7fcfceffd640 1 -- 172.21.15.105:0/1440443753 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3485610085 0 0) 0x7fcfd015ef40 con 0x7fcfd015eb60 2024-04-15T15:21:19.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.437+0000 7fcfceffd640 1 -- 172.21.15.105:0/1440443753 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcfbc003520 con 0x7fcfd015eb60 2024-04-15T15:21:19.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.437+0000 7fcfd4842640 1 -- 172.21.15.105:0/1440443753 >> v1:172.21.15.154:6789/0 conn(0x7fcfd00d18e0 legacy=0x7fcfd0162630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:19.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.437+0000 7fcfd4842640 1 -- 172.21.15.105:0/1440443753 >> v1:172.21.15.105:6789/0 conn(0x7fcfd015eb60 legacy=0x7fcfd00d11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:19.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.437+0000 7fcfd4842640 1 -- 172.21.15.105:0/1440443753 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfd00d2970 con 0x7fcfd0162d40 2024-04-15T15:21:19.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.438+0000 7fcfd4842640 1 --2- 172.21.15.105:0/1440443753 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcfd0162d40 0x7fcfd0169960 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8009dc0 tx=0x7fcfc802bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7098397af16c4a4b server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:19.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.438+0000 7fcfceffd640 1 -- 172.21.15.105:0/1440443753 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc8039070 con 0x7fcfd0162d40 2024-04-15T15:21:19.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.438+0000 7fcfceffd640 1 -- 172.21.15.105:0/1440443753 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfc802ccd0 con 0x7fcfd0162d40 2024-04-15T15:21:19.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.438+0000 7fcfceffd640 1 -- 172.21.15.105:0/1440443753 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc8034640 con 0x7fcfd0162d40 2024-04-15T15:21:19.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.438+0000 7fcfd62a5640 1 -- 172.21.15.105:0/1440443753 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcfd0162d40 msgr2=0x7fcfd0169960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:19.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.438+0000 7fcfd62a5640 1 --2- 172.21.15.105:0/1440443753 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcfd0162d40 0x7fcfd0169960 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8009dc0 tx=0x7fcfc802bf20 comp rx=0 tx=0).stop 2024-04-15T15:21:19.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.438+0000 7fcfd62a5640 1 -- 172.21.15.105:0/1440443753 shutdown_connections 2024-04-15T15:21:19.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.438+0000 7fcfd62a5640 1 --2- 172.21.15.105:0/1440443753 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcfd0162d40 0x7fcfd0169960 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:19.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.439+0000 7fcfd62a5640 1 -- 172.21.15.105:0/1440443753 >> 172.21.15.105:0/1440443753 conn(0x7fcfd0067720 msgr2=0x7fcfd0067b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:19.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.439+0000 7fcfd62a5640 1 -- 172.21.15.105:0/1440443753 shutdown_connections 2024-04-15T15:21:19.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.439+0000 7fcfd62a5640 1 -- 172.21.15.105:0/1440443753 wait complete. 2024-04-15T15:21:19.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.440+0000 7fcfd62a5640 1 Processor -- start 2024-04-15T15:21:19.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.440+0000 7fcfd62a5640 1 -- start start 2024-04-15T15:21:19.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.440+0000 7fcfd62a5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcfd00d18e0 0x7fcfd014cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:19.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.441+0000 7fcfd62a5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcfd015eb60 0x7fcfd00cc830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:19.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.441+0000 7fcfd62a5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcfd0162d40 0x7fcfd00d46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:19.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.441+0000 7fcfd62a5640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd00b78f0 con 0x7fcfd00d18e0 2024-04-15T15:21:19.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.441+0000 7fcfd62a5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd00d2fc0 con 0x7fcfd0162d40 2024-04-15T15:21:19.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.441+0000 7fcfd62a5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fcfd00d4460 con 0x7fcfd015eb60 2024-04-15T15:21:19.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.441+0000 7fcfcffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcfd00d18e0 0x7fcfd014cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:19.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.441+0000 7fcfcffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcfd00d18e0 0x7fcfd014cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33528/0 (socket says 172.21.15.105:33528) 2024-04-15T15:21:19.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.441+0000 7fcfcffff640 1 -- 172.21.15.105:0/1718310340 learned_addr learned my addr 172.21.15.105:0/1718310340 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:19.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.441+0000 7fcfd4842640 1 --2- 172.21.15.105:0/1718310340 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcfd0162d40 0x7fcfd00d46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:19.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.441+0000 7fcfcf7fe640 1 --2- 172.21.15.105:0/1718310340 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcfd015eb60 0x7fcfd00cc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:19.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.441+0000 7fcfcffff640 1 -- 172.21.15.105:0/1718310340 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcfd015eb60 msgr2=0x7fcfd00cc830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:19.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.441+0000 7fcfcffff640 1 --2- 172.21.15.105:0/1718310340 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcfd015eb60 0x7fcfd00cc830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:19.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.441+0000 7fcfcffff640 1 -- 172.21.15.105:0/1718310340 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcfd0162d40 msgr2=0x7fcfd00d46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:19.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.441+0000 7fcfcffff640 1 --2- 172.21.15.105:0/1718310340 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcfd0162d40 0x7fcfd00d46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:19.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.441+0000 7fcfcffff640 1 -- 172.21.15.105:0/1718310340 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfd00cd340 con 0x7fcfd00d18e0 2024-04-15T15:21:19.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.441+0000 7fcfcffff640 1 --2- 172.21.15.105:0/1718310340 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcfd00d18e0 0x7fcfd014cc50 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fcfc000c530 tx=0x7fcfc000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:19.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.441+0000 7fcfccff9640 1 -- 172.21.15.105:0/1718310340 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc0017070 con 0x7fcfd00d18e0 2024-04-15T15:21:19.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.441+0000 7fcfccff9640 1 -- 172.21.15.105:0/1718310340 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfc0005ce0 con 0x7fcfd00d18e0 2024-04-15T15:21:19.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.441+0000 7fcfccff9640 1 -- 172.21.15.105:0/1718310340 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc00072b0 con 0x7fcfd00d18e0 2024-04-15T15:21:19.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.441+0000 7fcfd62a5640 1 -- 172.21.15.105:0/1718310340 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfd00cd620 con 0x7fcfd00d18e0 2024-04-15T15:21:19.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.441+0000 7fcfd62a5640 1 -- 172.21.15.105:0/1718310340 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfd00cdbe0 con 0x7fcfd00d18e0 2024-04-15T15:21:19.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.442+0000 7fcfd62a5640 1 -- 172.21.15.105:0/1718310340 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfd0068a40 con 0x7fcfd00d18e0 2024-04-15T15:21:19.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.443+0000 7fcfccff9640 1 -- 172.21.15.105:0/1718310340 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fcfc001a070 con 0x7fcfd00d18e0 2024-04-15T15:21:19.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.443+0000 7fcfccff9640 1 --2- 172.21.15.105:0/1718310340 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fcfb006c980 0x7fcfb006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:19.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.443+0000 7fcfcf7fe640 1 --2- 172.21.15.105:0/1718310340 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fcfb006c980 0x7fcfb006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:19.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.443+0000 7fcfccff9640 1 -- 172.21.15.105:0/1718310340 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5795+0+0 (secure 0 0 0) 0x7fcfc008b010 con 0x7fcfd00d18e0 2024-04-15T15:21:19.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.444+0000 7fcfcf7fe640 1 --2- 172.21.15.105:0/1718310340 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fcfb006c980 0x7fcfb006ee40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8007fd0 tx=0x7fcfb8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:19.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.448+0000 7fcfccff9640 1 -- 172.21.15.105:0/1718310340 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcfc00594b0 con 0x7fcfd00d18e0 2024-04-15T15:21:19.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.623+0000 7fcfd62a5640 1 -- 172.21.15.105:0/1718310340 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcfd00d23f0 con 0x7fcfd00d18e0 2024-04-15T15:21:19.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.624+0000 7fcfccff9640 1 -- 172.21.15.105:0/1718310340 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16846 (secure 0 0 0) 0x7fcfc0058db0 con 0x7fcfd00d18e0 2024-04-15T15:21:19.626 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:21:19.626 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":35,"fsid":"09e4d2a4-a324-4793-af77-644b25e520e1","created":"2024-04-15T15:19:01.524543+0000","modified":"2024-04-15T15:21:03.660870+0000","last_up_change":"2024-04-15T15:19:12.825815+0000","last_in_change":"2024-04-15T15:19:03.788126+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:19:13.748585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:20:26.983353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-15T15:20:27.961802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6f6292b-2d7e-4aa0-9d1c-6ddd0f97287c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6801","nonce":2873040054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6803","nonce":2873040054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6807","nonce":2873040054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6805","nonce":2873040054}]},"public_addr":"172.21.15.105:6801/2873040054","cluster_addr":"172.21.15.105:6803/2873040054","heartbeat_back_addr":"172.21.15.105:6807/2873040054","heartbeat_front_addr":"172.21.15.105:6805/2873040054","state":["exists","up"]},{"osd":1,"uuid":"8900966a-b393-41e3-8f93-e5a43bd23622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6818","nonce":2494994479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6821","nonce":2494994479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6828","nonce":2494994479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6823","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6825","nonce":2494994479}]},"public_addr":"172.21.15.105:6818/2494994479","cluster_addr":"172.21.15.105:6821/2494994479","heartbeat_back_addr":"172.21.15.105:6828/2494994479","heartbeat_front_addr":"172.21.15.105:6825/2494994479","state":["exists","up"]},{"osd":2,"uuid":"81a5dc7a-9325-4ff5-941d-a2d5fba21f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6809","nonce":3798408699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6811","nonce":3798408699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6815","nonce":3798408699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6813","nonce":3798408699}]},"public_addr":"172.21.15.105:6809/3798408699","cluster_addr":"172.21.15.105:6811/3798408699","heartbeat_back_addr":"172.21.15.105:6815/3798408699","heartbeat_front_addr":"172.21.15.105:6813/3798408699","state":["exists","up"]},{"osd":3,"uuid":"df766ed2-5c71-4805-96f4-3d4970edc960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6819","nonce":1920720186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6824","nonce":1920720186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6831","nonce":1920720186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6827","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6829","nonce":1920720186}]},"public_addr":"172.21.15.105:6819/1920720186","cluster_addr":"172.21.15.105:6824/1920720186","heartbeat_back_addr":"172.21.15.105:6831/1920720186","heartbeat_front_addr":"172.21.15.105:6829/1920720186","state":["exists","up"]},{"osd":4,"uuid":"e1039e03-28b8-49df-b951-c91930b93c9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6810","nonce":3516494456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6813","nonce":3516494456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6820","nonce":3516494456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6817","nonce":3516494456}]},"public_addr":"172.21.15.154:6810/3516494456","cluster_addr":"172.21.15.154:6813/3516494456","heartbeat_back_addr":"172.21.15.154:6820/3516494456","heartbeat_front_addr":"172.21.15.154:6817/3516494456","state":["exists","up"]},{"osd":5,"uuid":"a9f5eaf1-66ff-405a-a9e4-ec3340812a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6801","nonce":540367751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6803","nonce":540367751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6807","nonce":540367751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6805","nonce":540367751}]},"public_addr":"172.21.15.154:6801/540367751","cluster_addr":"172.21.15.154:6803/540367751","heartbeat_back_addr":"172.21.15.154:6807/540367751","heartbeat_front_addr":"172.21.15.154:6805/540367751","state":["exists","up"]},{"osd":6,"uuid":"267d02ea-675f-428f-aae8-e0375e4d4feb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6825","nonce":1550829005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6827","nonce":1550829005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6831","nonce":1550829005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6829","nonce":1550829005}]},"public_addr":"172.21.15.154:6825/1550829005","cluster_addr":"172.21.15.154:6827/1550829005","heartbeat_back_addr":"172.21.15.154:6831/1550829005","heartbeat_front_addr":"172.21.15.154:6829/1550829005","state":["exists","up"]},{"osd":7,"uuid":"759ac6cc-896a-423c-885f-9fe90d8dcb14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6809","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6811","nonce":3920089903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6816","nonce":3920089903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6823","nonce":3920089903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6819","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6821","nonce":3920089903}]},"public_addr":"172.21.15.154:6811/3920089903","cluster_addr":"172.21.15.154:6816/3920089903","heartbeat_back_addr":"172.21.15.154:6823/3920089903","heartbeat_front_addr":"172.21.15.154:6821/3920089903","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.949509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.984072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.907278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.947391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.975447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.998896+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:09.028739+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.894119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.105:6837/958066041":"2024-04-16T15:21:03.660841+0000","172.21.15.105:6835/958066041":"2024-04-16T15:21:03.660841+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:21:19.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.626+0000 7fcfd62a5640 1 -- 172.21.15.105:0/1718310340 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fcfb006c980 msgr2=0x7fcfb006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:19.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.626+0000 7fcfd62a5640 1 --2- 172.21.15.105:0/1718310340 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fcfb006c980 0x7fcfb006ee40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8007fd0 tx=0x7fcfb8009040 comp rx=0 tx=0).stop 2024-04-15T15:21:19.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.626+0000 7fcfd62a5640 1 -- 172.21.15.105:0/1718310340 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcfd00d18e0 msgr2=0x7fcfd014cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:19.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.626+0000 7fcfd62a5640 1 --2- 172.21.15.105:0/1718310340 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcfd00d18e0 0x7fcfd014cc50 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fcfc000c530 tx=0x7fcfc000c9f0 comp rx=0 tx=0).stop 2024-04-15T15:21:19.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.626+0000 7fcfd62a5640 1 -- 172.21.15.105:0/1718310340 shutdown_connections 2024-04-15T15:21:19.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.626+0000 7fcfd62a5640 1 --2- 172.21.15.105:0/1718310340 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcfd015eb60 0x7fcfd00cc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:19.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.626+0000 7fcfd62a5640 1 --2- 172.21.15.105:0/1718310340 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fcfb006c980 0x7fcfb006ee40 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:19.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.626+0000 7fcfd62a5640 1 --2- 172.21.15.105:0/1718310340 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcfd0162d40 0x7fcfd00d46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:19.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.626+0000 7fcfd62a5640 1 --2- 172.21.15.105:0/1718310340 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcfd00d18e0 0x7fcfd014cc50 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:19.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.626+0000 7fcfd62a5640 1 -- 172.21.15.105:0/1718310340 >> 172.21.15.105:0/1718310340 conn(0x7fcfd0067720 msgr2=0x7fcfd00b7a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:19.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.626+0000 7fcfd62a5640 1 -- 172.21.15.105:0/1718310340 shutdown_connections 2024-04-15T15:21:19.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.626+0000 7fcfd62a5640 1 -- 172.21.15.105:0/1718310340 wait complete. 2024-04-15T15:21:19.639 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-15T15:21:19.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.742+0000 7f4a0c731640 1 Processor -- start 2024-04-15T15:21:19.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.743+0000 7f4a0c731640 1 -- start start 2024-04-15T15:21:19.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.743+0000 7f4a0c731640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4a0415b700 0x7f4a040af590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:19.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.743+0000 7f4a0c731640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4a040b0490 0x7f4a040b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:19.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.743+0000 7f4a0c731640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4a040b58d0 0x7f4a040d1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:19.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.743+0000 7f4a0c731640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f4a040b4f00 con 0x7f4a040b0490 2024-04-15T15:21:19.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.743+0000 7f4a0c731640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f4a040b45d0 con 0x7f4a0415b700 2024-04-15T15:21:19.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.743+0000 7f4a0c731640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f4a040b48b0 con 0x7f4a040b58d0 2024-04-15T15:21:19.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.744+0000 7f4a09ccc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4a040b0490 0x7f4a040b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:19.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.744+0000 7f4a09ccc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4a040b0490 0x7f4a040b5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33542/0 (socket says 172.21.15.105:33542) 2024-04-15T15:21:19.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.744+0000 7f4a09ccc640 1 -- 172.21.15.105:0/1502869320 learned_addr learned my addr 172.21.15.105:0/1502869320 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:19.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.744+0000 7f4a09ccc640 1 -- 172.21.15.105:0/1502869320 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4a040b58d0 msgr2=0x7f4a040d1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:19.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.744+0000 7f4a09ccc640 1 --2- 172.21.15.105:0/1502869320 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4a040b58d0 0x7f4a040d1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:19.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.744+0000 7f4a09ccc640 1 -- 172.21.15.105:0/1502869320 >> v2:172.21.15.154:3300/0 conn(0x7f4a0415b700 msgr2=0x7f4a040af590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:19.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.744+0000 7f4a09ccc640 1 --2- 172.21.15.105:0/1502869320 >> v2:172.21.15.154:3300/0 conn(0x7f4a0415b700 0x7f4a040af590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:19.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.744+0000 7f4a09ccc640 1 -- 172.21.15.105:0/1502869320 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a040b4b90 con 0x7f4a040b0490 2024-04-15T15:21:19.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.745+0000 7f4a09ccc640 1 --2- 172.21.15.105:0/1502869320 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a040b0490 0x7f4a040b5390 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f4a040b4160 tx=0x7f4a0002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48d4a369dd17cbf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:19.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.745+0000 7f4a094cb640 1 -- 172.21.15.105:0/1502869320 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0002e070 con 0x7f4a040b0490 2024-04-15T15:21:19.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.745+0000 7f4a094cb640 1 -- 172.21.15.105:0/1502869320 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a00037070 con 0x7f4a040b0490 2024-04-15T15:21:19.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.745+0000 7f4a094cb640 1 -- 172.21.15.105:0/1502869320 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a00031a00 con 0x7f4a040b0490 2024-04-15T15:21:19.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.745+0000 7f4a0c731640 1 -- 172.21.15.105:0/1502869320 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a040b0490 msgr2=0x7f4a040b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:19.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.745+0000 7f4a0c731640 1 --2- 172.21.15.105:0/1502869320 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a040b0490 0x7f4a040b5390 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f4a040b4160 tx=0x7f4a0002b880 comp rx=0 tx=0).stop 2024-04-15T15:21:19.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.745+0000 7f4a0c731640 1 -- 172.21.15.105:0/1502869320 shutdown_connections 2024-04-15T15:21:19.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.745+0000 7f4a0c731640 1 --2- 172.21.15.105:0/1502869320 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4a040b58d0 0x7f4a040d1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:19.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.745+0000 7f4a0c731640 1 --2- 172.21.15.105:0/1502869320 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a040b0490 0x7f4a040b5390 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:19.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.745+0000 7f4a0c731640 1 --2- 172.21.15.105:0/1502869320 >> v2:172.21.15.154:3300/0 conn(0x7f4a0415b700 0x7f4a040af590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:19.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.745+0000 7f4a0c731640 1 -- 172.21.15.105:0/1502869320 >> 172.21.15.105:0/1502869320 conn(0x7f4a0409aa40 msgr2=0x7f4a040b6950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:19.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.745+0000 7f4a0c731640 1 -- 172.21.15.105:0/1502869320 shutdown_connections 2024-04-15T15:21:19.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.745+0000 7f4a0c731640 1 -- 172.21.15.105:0/1502869320 wait complete. 2024-04-15T15:21:19.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.747+0000 7f4a0c731640 1 Processor -- start 2024-04-15T15:21:19.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.747+0000 7f4a0c731640 1 -- start start 2024-04-15T15:21:19.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.747+0000 7f4a0c731640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4a040b0490 0x7f4a04147ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:19.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.747+0000 7f4a0c731640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a040b58d0 0x7f4a041559d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:19.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.747+0000 7f4a0c731640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4a0415b700 0x7f4a04155f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:19.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.747+0000 7f4a0c731640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4a040afc50 con 0x7f4a040b58d0 2024-04-15T15:21:19.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.747+0000 7f4a0c731640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4a040afad0 con 0x7f4a0415b700 2024-04-15T15:21:19.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.747+0000 7f4a0c731640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f4a040d19d0 con 0x7f4a040b0490 2024-04-15T15:21:19.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.747+0000 7f4a09ccc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a040b58d0 0x7f4a041559d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:19.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.747+0000 7f4a09ccc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a040b58d0 0x7f4a041559d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33550/0 (socket says 172.21.15.105:33550) 2024-04-15T15:21:19.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.747+0000 7f4a09ccc640 1 -- 172.21.15.105:0/524887642 learned_addr learned my addr 172.21.15.105:0/524887642 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:19.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.748+0000 7f4a0a4cd640 1 --2- 172.21.15.105:0/524887642 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4a040b0490 0x7f4a04147ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:19.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.748+0000 7f4a0acce640 1 --2- 172.21.15.105:0/524887642 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4a0415b700 0x7f4a04155f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:19.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.748+0000 7f4a09ccc640 1 -- 172.21.15.105:0/524887642 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4a040b0490 msgr2=0x7f4a04147ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:19.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.748+0000 7f4a09ccc640 1 --2- 172.21.15.105:0/524887642 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4a040b0490 0x7f4a04147ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:19.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.748+0000 7f4a09ccc640 1 -- 172.21.15.105:0/524887642 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4a0415b700 msgr2=0x7f4a04155f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:19.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.748+0000 7f4a09ccc640 1 --2- 172.21.15.105:0/524887642 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4a0415b700 0x7f4a04155f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:19.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.748+0000 7f4a09ccc640 1 -- 172.21.15.105:0/524887642 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a040ca950 con 0x7f4a040b58d0 2024-04-15T15:21:19.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.748+0000 7f4a09ccc640 1 --2- 172.21.15.105:0/524887642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a040b58d0 0x7f4a041559d0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f4a0002b8c0 tx=0x7f4a000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:19.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.748+0000 7f49f6ffd640 1 -- 172.21.15.105:0/524887642 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a00032880 con 0x7f4a040b58d0 2024-04-15T15:21:19.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.748+0000 7f49f6ffd640 1 -- 172.21.15.105:0/524887642 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a00032a20 con 0x7f4a040b58d0 2024-04-15T15:21:19.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.748+0000 7f4a0c731640 1 -- 172.21.15.105:0/524887642 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a040cac30 con 0x7f4a040b58d0 2024-04-15T15:21:19.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.748+0000 7f49f6ffd640 1 -- 172.21.15.105:0/524887642 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a00032e10 con 0x7f4a040b58d0 2024-04-15T15:21:19.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.748+0000 7f4a0c731640 1 -- 172.21.15.105:0/524887642 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a040cb1f0 con 0x7f4a040b58d0 2024-04-15T15:21:19.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.749+0000 7f4a0c731640 1 -- 172.21.15.105:0/524887642 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a04069150 con 0x7f4a040b58d0 2024-04-15T15:21:19.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.750+0000 7f49f6ffd640 1 -- 172.21.15.105:0/524887642 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f4a0004c060 con 0x7f4a040b58d0 2024-04-15T15:21:19.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.750+0000 7f49f6ffd640 1 --2- 172.21.15.105:0/524887642 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f49e806c980 0x7f49e806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:19.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.750+0000 7f49f6ffd640 1 -- 172.21.15.105:0/524887642 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5795+0+0 (secure 0 0 0) 0x7f4a000ad970 con 0x7f4a040b58d0 2024-04-15T15:21:19.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.750+0000 7f4a0a4cd640 1 --2- 172.21.15.105:0/524887642 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f49e806c980 0x7f49e806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:19.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.750+0000 7f4a0a4cd640 1 --2- 172.21.15.105:0/524887642 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f49e806c980 0x7f49e806ee40 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f49f800b810 tx=0x7f49f8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:19.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.754+0000 7f49f6ffd640 1 -- 172.21.15.105:0/524887642 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4a000afe20 con 0x7f4a040b58d0 2024-04-15T15:21:19.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.965+0000 7f4a0c731640 1 -- 172.21.15.105:0/524887642 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f4a041578c0 con 0x7f4a040b58d0 2024-04-15T15:21:19.972 INFO:tasks.ceph.mon.a.smithi105.stderr:2024-04-15T15:21:19.966+0000 7fd391944640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-15T15:21:19.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.967+0000 7f49f6ffd640 1 -- 172.21.15.105:0/524887642 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v21) ==== 161+0+0 (secure 0 0 0) 0x7f4a00040070 con 0x7f4a040b58d0 2024-04-15T15:21:19.973 INFO:teuthology.orchestra.run.smithi105.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-15T15:21:19.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.970+0000 7f4a0c731640 1 -- 172.21.15.105:0/524887642 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f49e806c980 msgr2=0x7f49e806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:19.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.970+0000 7f4a0c731640 1 --2- 172.21.15.105:0/524887642 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f49e806c980 0x7f49e806ee40 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f49f800b810 tx=0x7f49f8008040 comp rx=0 tx=0).stop 2024-04-15T15:21:19.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.970+0000 7f4a0c731640 1 -- 172.21.15.105:0/524887642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a040b58d0 msgr2=0x7f4a041559d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:19.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.970+0000 7f4a0c731640 1 --2- 172.21.15.105:0/524887642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a040b58d0 0x7f4a041559d0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f4a0002b8c0 tx=0x7f4a000046c0 comp rx=0 tx=0).stop 2024-04-15T15:21:19.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.971+0000 7f4a0c731640 1 -- 172.21.15.105:0/524887642 shutdown_connections 2024-04-15T15:21:19.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.971+0000 7f4a0c731640 1 --2- 172.21.15.105:0/524887642 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f49e806c980 0x7f49e806ee40 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:19.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.971+0000 7f4a0c731640 1 --2- 172.21.15.105:0/524887642 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4a0415b700 0x7f4a04155f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:19.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.971+0000 7f4a0c731640 1 --2- 172.21.15.105:0/524887642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a040b58d0 0x7f4a041559d0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:19.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.971+0000 7f4a0c731640 1 --2- 172.21.15.105:0/524887642 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4a040b0490 0x7f4a04147ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:19.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.971+0000 7f4a0c731640 1 -- 172.21.15.105:0/524887642 >> 172.21.15.105:0/524887642 conn(0x7f4a0409aa40 msgr2=0x7f4a04070160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:19.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.971+0000 7f4a0c731640 1 -- 172.21.15.105:0/524887642 shutdown_connections 2024-04-15T15:21:19.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:19.971+0000 7f4a0c731640 1 -- 172.21.15.105:0/524887642 wait complete. 2024-04-15T15:21:19.977 INFO:tasks.ceph.mds.a.smithi105.stderr: -1349> 2024-04-15T15:21:09.674+0000 7f48b9f2d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-15T15:21:19.984 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-15T15:21:20.003 INFO:tasks.ceph.mds.a.smithi105.stdout:starting mds.a at 2024-04-15T15:21:20.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.089+0000 7fce9b500640 1 Processor -- start 2024-04-15T15:21:20.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.089+0000 7fce9b500640 1 -- start start 2024-04-15T15:21:20.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.090+0000 7fce9b500640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fce9406e460 0x7fce940700f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:20.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.090+0000 7fce9b500640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fce9406e840 0x7fce9409aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:20.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.090+0000 7fce9b500640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce94070840 con 0x7fce9406cad0 2024-04-15T15:21:20.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.090+0000 7fce9b500640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fce94070b20 con 0x7fce9406e460 2024-04-15T15:21:20.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.090+0000 7fce9b500640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fce94070e00 con 0x7fce9406e840 2024-04-15T15:21:20.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.090+0000 7fce9929c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fce9406e460 0x7fce940700f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:20.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.090+0000 7fce9929c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fce9406e460 0x7fce940700f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33570/0 (socket says 172.21.15.105:33570) 2024-04-15T15:21:20.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.090+0000 7fce9929c640 1 -- 172.21.15.105:0/4184001779 learned_addr learned my addr 172.21.15.105:0/4184001779 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:20.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.090+0000 7fce9929c640 1 -- 172.21.15.105:0/4184001779 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fce9406e840 msgr2=0x7fce9409aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:20.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.090+0000 7fce9929c640 1 --2- 172.21.15.105:0/4184001779 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fce9406e840 0x7fce9409aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:20.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.090+0000 7fce9929c640 1 -- 172.21.15.105:0/4184001779 >> v1:172.21.15.154:6789/0 conn(0x7fce9406cad0 legacy=0x7fce940d0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:20.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.090+0000 7fce9929c640 1 -- 172.21.15.105:0/4184001779 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce940710e0 con 0x7fce9406e460 2024-04-15T15:21:20.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.091+0000 7fce9929c640 1 --2- 172.21.15.105:0/4184001779 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce9406e460 0x7fce940700f0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fce90009870 tx=0x7fce9002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e96fb90c2600630d server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:20.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.091+0000 7fce8bfff640 1 -- 172.21.15.105:0/4184001779 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce90035020 con 0x7fce9406e460 2024-04-15T15:21:20.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.091+0000 7fce8bfff640 1 -- 172.21.15.105:0/4184001779 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce90031640 con 0x7fce9406e460 2024-04-15T15:21:20.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.091+0000 7fce8bfff640 1 -- 172.21.15.105:0/4184001779 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9002d040 con 0x7fce9406e460 2024-04-15T15:21:20.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.092+0000 7fce9b500640 1 -- 172.21.15.105:0/4184001779 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce9406e460 msgr2=0x7fce940700f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:20.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.092+0000 7fce9b500640 1 --2- 172.21.15.105:0/4184001779 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce9406e460 0x7fce940700f0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fce90009870 tx=0x7fce9002b8f0 comp rx=0 tx=0).stop 2024-04-15T15:21:20.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.092+0000 7fce9b500640 1 -- 172.21.15.105:0/4184001779 shutdown_connections 2024-04-15T15:21:20.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.092+0000 7fce9b500640 1 --2- 172.21.15.105:0/4184001779 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fce9406e840 0x7fce9409aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:20.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.092+0000 7fce9b500640 1 --2- 172.21.15.105:0/4184001779 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce9406e460 0x7fce940700f0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:20.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.092+0000 7fce9b500640 1 -- 172.21.15.105:0/4184001779 >> 172.21.15.105:0/4184001779 conn(0x7fce940b4030 msgr2=0x7fce940b4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:20.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.092+0000 7fce9b500640 1 -- 172.21.15.105:0/4184001779 shutdown_connections 2024-04-15T15:21:20.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.092+0000 7fce9b500640 1 -- 172.21.15.105:0/4184001779 wait complete. 2024-04-15T15:21:20.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.093+0000 7fce9b500640 1 Processor -- start 2024-04-15T15:21:20.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.093+0000 7fce9b500640 1 -- start start 2024-04-15T15:21:20.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.094+0000 7fce9b500640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce9406cad0 0x7fce9415f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:20.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.094+0000 7fce9b500640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fce9406e460 0x7fce9414d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:20.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.094+0000 7fce9b500640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce9406e840 0x7fce940c8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:20.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.094+0000 7fce9b500640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fce94162670 con 0x7fce9406cad0 2024-04-15T15:21:20.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.094+0000 7fce9b500640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fce941624f0 con 0x7fce9406e840 2024-04-15T15:21:20.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.094+0000 7fce9b500640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fce941627f0 con 0x7fce9406e460 2024-04-15T15:21:20.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.094+0000 7fce9929c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce9406cad0 0x7fce9415f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:20.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.094+0000 7fce9929c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce9406cad0 0x7fce9415f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33574/0 (socket says 172.21.15.105:33574) 2024-04-15T15:21:20.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.094+0000 7fce9929c640 1 -- 172.21.15.105:0/2188438839 learned_addr learned my addr 172.21.15.105:0/2188438839 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:20.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.094+0000 7fce98a9b640 1 --2- 172.21.15.105:0/2188438839 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fce9406e460 0x7fce9414d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:20.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.094+0000 7fce99a9d640 1 --2- 172.21.15.105:0/2188438839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce9406e840 0x7fce940c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:20.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.094+0000 7fce9929c640 1 -- 172.21.15.105:0/2188438839 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fce9406e460 msgr2=0x7fce9414d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:20.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.094+0000 7fce9929c640 1 --2- 172.21.15.105:0/2188438839 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fce9406e460 0x7fce9414d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:20.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.094+0000 7fce9929c640 1 -- 172.21.15.105:0/2188438839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce9406e840 msgr2=0x7fce940c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:20.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.094+0000 7fce9929c640 1 --2- 172.21.15.105:0/2188438839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce9406e840 0x7fce940c8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:20.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.094+0000 7fce9929c640 1 -- 172.21.15.105:0/2188438839 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce9415f960 con 0x7fce9406cad0 2024-04-15T15:21:20.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.094+0000 7fce99a9d640 1 --2- 172.21.15.105:0/2188438839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce9406e840 0x7fce940c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:21:20.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.094+0000 7fce98a9b640 1 --2- 172.21.15.105:0/2188438839 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fce9406e460 0x7fce9414d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:21:20.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.094+0000 7fce9929c640 1 --2- 172.21.15.105:0/2188438839 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce9406cad0 0x7fce9415f280 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fce9002b930 tx=0x7fce900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:20.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.095+0000 7fce89ffb640 1 -- 172.21.15.105:0/2188438839 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9003ac60 con 0x7fce9406cad0 2024-04-15T15:21:20.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.095+0000 7fce89ffb640 1 -- 172.21.15.105:0/2188438839 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce9003ae20 con 0x7fce9406cad0 2024-04-15T15:21:20.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.095+0000 7fce9b500640 1 -- 172.21.15.105:0/2188438839 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce9415fc40 con 0x7fce9406cad0 2024-04-15T15:21:20.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.095+0000 7fce89ffb640 1 -- 172.21.15.105:0/2188438839 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce90044630 con 0x7fce9406cad0 2024-04-15T15:21:20.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.095+0000 7fce9b500640 1 -- 172.21.15.105:0/2188438839 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce94160200 con 0x7fce9406cad0 2024-04-15T15:21:20.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.096+0000 7fce9b500640 1 -- 172.21.15.105:0/2188438839 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce94068ab0 con 0x7fce9406cad0 2024-04-15T15:21:20.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.096+0000 7fce89ffb640 1 -- 172.21.15.105:0/2188438839 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fce9004b060 con 0x7fce9406cad0 2024-04-15T15:21:20.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.097+0000 7fce89ffb640 1 --2- 172.21.15.105:0/2188438839 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fce7806c9b0 0x7fce7806ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:20.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.097+0000 7fce89ffb640 1 -- 172.21.15.105:0/2188438839 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5881+0+0 (secure 0 0 0) 0x7fce900ada60 con 0x7fce9406cad0 2024-04-15T15:21:20.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.097+0000 7fce98a9b640 1 --2- 172.21.15.105:0/2188438839 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fce7806c9b0 0x7fce7806ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:20.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.097+0000 7fce98a9b640 1 --2- 172.21.15.105:0/2188438839 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fce7806c9b0 0x7fce7806ee70 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fce84006fd0 tx=0x7fce84008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:20.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.101+0000 7fce89ffb640 1 -- 172.21.15.105:0/2188438839 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fce9003f070 con 0x7fce9406cad0 2024-04-15T15:21:20.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.318+0000 7fce9b500640 1 -- 172.21.15.105:0/2188438839 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fce940b1680 con 0x7fce9406cad0 2024-04-15T15:21:20.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.968+0000 7fce89ffb640 1 -- 172.21.15.105:0/2188438839 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v22) ==== 104+0+0 (secure 0 0 0) 0x7fce9007bce0 con 0x7fce9406cad0 2024-04-15T15:21:20.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.971+0000 7fce9b500640 1 -- 172.21.15.105:0/2188438839 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fce7806c9b0 msgr2=0x7fce7806ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:20.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.971+0000 7fce9b500640 1 --2- 172.21.15.105:0/2188438839 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fce7806c9b0 0x7fce7806ee70 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fce84006fd0 tx=0x7fce84008040 comp rx=0 tx=0).stop 2024-04-15T15:21:20.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.971+0000 7fce9b500640 1 -- 172.21.15.105:0/2188438839 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce9406cad0 msgr2=0x7fce9415f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:20.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.971+0000 7fce9b500640 1 --2- 172.21.15.105:0/2188438839 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce9406cad0 0x7fce9415f280 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fce9002b930 tx=0x7fce900046c0 comp rx=0 tx=0).stop 2024-04-15T15:21:20.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.971+0000 7fce9b500640 1 -- 172.21.15.105:0/2188438839 shutdown_connections 2024-04-15T15:21:20.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.971+0000 7fce9b500640 1 --2- 172.21.15.105:0/2188438839 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fce9406e460 0x7fce9414d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:20.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.971+0000 7fce9b500640 1 --2- 172.21.15.105:0/2188438839 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fce7806c9b0 0x7fce7806ee70 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:20.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.971+0000 7fce9b500640 1 --2- 172.21.15.105:0/2188438839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce9406e840 0x7fce940c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:20.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.971+0000 7fce9b500640 1 --2- 172.21.15.105:0/2188438839 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce9406cad0 0x7fce9415f280 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:20.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.971+0000 7fce9b500640 1 -- 172.21.15.105:0/2188438839 >> 172.21.15.105:0/2188438839 conn(0x7fce940b4030 msgr2=0x7fce9409aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:20.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.971+0000 7fce9b500640 1 -- 172.21.15.105:0/2188438839 shutdown_connections 2024-04-15T15:21:20.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:20.971+0000 7fce9b500640 1 -- 172.21.15.105:0/2188438839 wait complete. 2024-04-15T15:21:20.986 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-15T15:21:21.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.134+0000 7f2af0d04640 1 Processor -- start 2024-04-15T15:21:21.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.134+0000 7f2af0d04640 1 -- start start 2024-04-15T15:21:21.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.135+0000 7f2af0d04640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2aec13e6f0 0x7f2aec0cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:21.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.135+0000 7f2af0d04640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2aec06f6d0 0x7f2aec15bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:21.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.135+0000 7f2af0d04640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2aec15f280 con 0x7f2aec0d0060 2024-04-15T15:21:21.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.135+0000 7f2af0d04640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f2aec15f560 con 0x7f2aec06f6d0 2024-04-15T15:21:21.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.135+0000 7f2af0d04640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f2aec15f840 con 0x7f2aec13e6f0 2024-04-15T15:21:21.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.135+0000 7f2aea59c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2aec13e6f0 0x7f2aec0cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:21.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.135+0000 7f2aead9d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2aec06f6d0 0x7f2aec15bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:21.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.135+0000 7f2ae1d9b640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f2aec0d0060 0x7f2aec06efc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.105:59248/0 (socket says 172.21.15.105:59248) 2024-04-15T15:21:21.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.135+0000 7f2ae1d9b640 1 -- 172.21.15.105:0/1038657356 learned_addr learned my addr 172.21.15.105:0/1038657356 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:21:21.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.135+0000 7f2ae9d9b640 1 -- 172.21.15.105:0/1038657356 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3301707571 0 0) 0x7f2aec15f280 con 0x7f2aec0d0060 2024-04-15T15:21:21.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.135+0000 7f2ae9d9b640 1 -- 172.21.15.105:0/1038657356 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ad8003520 con 0x7f2aec0d0060 2024-04-15T15:21:21.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.135+0000 7f2aea59c640 1 -- 172.21.15.105:0/1038657356 >> v2:172.21.15.154:3300/0 conn(0x7f2aec06f6d0 msgr2=0x7f2aec15bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:21.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.135+0000 7f2aea59c640 1 --2- 172.21.15.105:0/1038657356 >> v2:172.21.15.154:3300/0 conn(0x7f2aec06f6d0 0x7f2aec15bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:21.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.135+0000 7f2aea59c640 1 -- 172.21.15.105:0/1038657356 >> v1:172.21.15.154:6789/0 conn(0x7f2aec0d0060 legacy=0x7f2aec06efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:21.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.135+0000 7f2aea59c640 1 -- 172.21.15.105:0/1038657356 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aec15fb20 con 0x7f2aec13e6f0 2024-04-15T15:21:21.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.135+0000 7f2aead9d640 1 --2- 172.21.15.105:0/1038657356 >> v2:172.21.15.154:3300/0 conn(0x7f2aec06f6d0 0x7f2aec15bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:21:21.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.136+0000 7f2aea59c640 1 --2- 172.21.15.105:0/1038657356 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2aec13e6f0 0x7f2aec0cfb20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2adc0097a0 tx=0x7f2adc02b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5f7cf11489a4558 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:21.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.136+0000 7f2ae9d9b640 1 -- 172.21.15.105:0/1038657356 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2adc0315a0 con 0x7f2aec13e6f0 2024-04-15T15:21:21.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.136+0000 7f2ae9d9b640 1 -- 172.21.15.105:0/1038657356 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2adc037070 con 0x7f2aec13e6f0 2024-04-15T15:21:21.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.136+0000 7f2ae9d9b640 1 -- 172.21.15.105:0/1038657356 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2adc031940 con 0x7f2aec13e6f0 2024-04-15T15:21:21.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.137+0000 7f2af0d04640 1 -- 172.21.15.105:0/1038657356 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2aec13e6f0 msgr2=0x7f2aec0cfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:21.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.137+0000 7f2af0d04640 1 --2- 172.21.15.105:0/1038657356 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2aec13e6f0 0x7f2aec0cfb20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2adc0097a0 tx=0x7f2adc02b840 comp rx=0 tx=0).stop 2024-04-15T15:21:21.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.137+0000 7f2af0d04640 1 -- 172.21.15.105:0/1038657356 shutdown_connections 2024-04-15T15:21:21.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.137+0000 7f2af0d04640 1 --2- 172.21.15.105:0/1038657356 >> v2:172.21.15.154:3300/0 conn(0x7f2aec06f6d0 0x7f2aec15bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:21.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.137+0000 7f2af0d04640 1 --2- 172.21.15.105:0/1038657356 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2aec13e6f0 0x7f2aec0cfb20 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:21.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.137+0000 7f2af0d04640 1 -- 172.21.15.105:0/1038657356 >> 172.21.15.105:0/1038657356 conn(0x7f2aec067e10 msgr2=0x7f2aec0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:21.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.137+0000 7f2af0d04640 1 -- 172.21.15.105:0/1038657356 shutdown_connections 2024-04-15T15:21:21.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.137+0000 7f2af0d04640 1 -- 172.21.15.105:0/1038657356 wait complete. 2024-04-15T15:21:21.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.138+0000 7f2af0d04640 1 Processor -- start 2024-04-15T15:21:21.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.138+0000 7f2af0d04640 1 -- start start 2024-04-15T15:21:21.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.139+0000 7f2af0d04640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2aec06f6d0 0x7f2aec1594e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:21.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.139+0000 7f2af0d04640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2aec0d0060 0x7f2aec142d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:21.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.139+0000 7f2af0d04640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2aec13e6f0 0x7f2aec154ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:21.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.139+0000 7f2af0d04640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2aec0cf7f0 con 0x7f2aec06f6d0 2024-04-15T15:21:21.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.139+0000 7f2af0d04640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2aec0cf670 con 0x7f2aec0d0060 2024-04-15T15:21:21.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.139+0000 7f2af0d04640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f2aec13ecd0 con 0x7f2aec13e6f0 2024-04-15T15:21:21.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.139+0000 7f2aea59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2aec06f6d0 0x7f2aec1594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:21.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.139+0000 7f2aea59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2aec06f6d0 0x7f2aec1594e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46994/0 (socket says 172.21.15.105:46994) 2024-04-15T15:21:21.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.139+0000 7f2aea59c640 1 -- 172.21.15.105:0/3242940355 learned_addr learned my addr 172.21.15.105:0/3242940355 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:21.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.139+0000 7f2aead9d640 1 --2- 172.21.15.105:0/3242940355 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2aec13e6f0 0x7f2aec154ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:21.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.140+0000 7f2ae1d9b640 1 --2- 172.21.15.105:0/3242940355 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2aec0d0060 0x7f2aec142d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:21.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.140+0000 7f2aea59c640 1 -- 172.21.15.105:0/3242940355 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2aec13e6f0 msgr2=0x7f2aec154ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:21.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.140+0000 7f2aea59c640 1 --2- 172.21.15.105:0/3242940355 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2aec13e6f0 0x7f2aec154ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:21.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.140+0000 7f2aea59c640 1 -- 172.21.15.105:0/3242940355 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2aec0d0060 msgr2=0x7f2aec142d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:21.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.140+0000 7f2aea59c640 1 --2- 172.21.15.105:0/3242940355 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2aec0d0060 0x7f2aec142d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:21.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.140+0000 7f2aea59c640 1 -- 172.21.15.105:0/3242940355 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aec143830 con 0x7f2aec06f6d0 2024-04-15T15:21:21.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.140+0000 7f2aea59c640 1 --2- 172.21.15.105:0/3242940355 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2aec06f6d0 0x7f2aec1594e0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f2adc0098c0 tx=0x7f2adc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:21.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.140+0000 7f2ae37fe640 1 -- 172.21.15.105:0/3242940355 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2adc0317a0 con 0x7f2aec06f6d0 2024-04-15T15:21:21.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.140+0000 7f2ae37fe640 1 -- 172.21.15.105:0/3242940355 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2adc031940 con 0x7f2aec06f6d0 2024-04-15T15:21:21.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.140+0000 7f2af0d04640 1 -- 172.21.15.105:0/3242940355 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2aec143b10 con 0x7f2aec06f6d0 2024-04-15T15:21:21.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.140+0000 7f2ae37fe640 1 -- 172.21.15.105:0/3242940355 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2adc031cf0 con 0x7f2aec06f6d0 2024-04-15T15:21:21.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.140+0000 7f2af0d04640 1 -- 172.21.15.105:0/3242940355 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2aec1440d0 con 0x7f2aec06f6d0 2024-04-15T15:21:21.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.141+0000 7f2af0d04640 1 -- 172.21.15.105:0/3242940355 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2aec068a40 con 0x7f2aec06f6d0 2024-04-15T15:21:21.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.141+0000 7f2ae37fe640 1 -- 172.21.15.105:0/3242940355 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f2adc032430 con 0x7f2aec06f6d0 2024-04-15T15:21:21.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.142+0000 7f2ae37fe640 1 --2- 172.21.15.105:0/3242940355 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f2acc06c980 0x7f2acc06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:21.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.142+0000 7f2ae1d9b640 1 --2- 172.21.15.105:0/3242940355 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f2acc06c980 0x7f2acc06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:21.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.142+0000 7f2ae37fe640 1 -- 172.21.15.105:0/3242940355 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5881+0+0 (secure 0 0 0) 0x7f2adc02f030 con 0x7f2aec06f6d0 2024-04-15T15:21:21.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.142+0000 7f2ae1d9b640 1 --2- 172.21.15.105:0/3242940355 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f2acc06c980 0x7f2acc06ee40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2aec13e3a0 tx=0x7f2ad4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:21.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.146+0000 7f2ae37fe640 1 -- 172.21.15.105:0/3242940355 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2adc04b060 con 0x7f2aec06f6d0 2024-04-15T15:21:21.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.324+0000 7f2af0d04640 1 -- 172.21.15.105:0/3242940355 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2aec0b5720 con 0x7f2aec06f6d0 2024-04-15T15:21:21.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.970+0000 7f2ae37fe640 1 -- 172.21.15.105:0/3242940355 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v37) ==== 181+0+0 (secure 0 0 0) 0x7f2adc031e90 con 0x7f2aec06f6d0 2024-04-15T15:21:21.972 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_metadata' removed 2024-04-15T15:21:21.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.974+0000 7f2af0d04640 1 -- 172.21.15.105:0/3242940355 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f2acc06c980 msgr2=0x7f2acc06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:21.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.974+0000 7f2af0d04640 1 --2- 172.21.15.105:0/3242940355 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f2acc06c980 0x7f2acc06ee40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2aec13e3a0 tx=0x7f2ad4009040 comp rx=0 tx=0).stop 2024-04-15T15:21:21.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.974+0000 7f2af0d04640 1 -- 172.21.15.105:0/3242940355 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2aec06f6d0 msgr2=0x7f2aec1594e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:21.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.974+0000 7f2af0d04640 1 --2- 172.21.15.105:0/3242940355 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2aec06f6d0 0x7f2aec1594e0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f2adc0098c0 tx=0x7f2adc0046c0 comp rx=0 tx=0).stop 2024-04-15T15:21:21.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.974+0000 7f2af0d04640 1 -- 172.21.15.105:0/3242940355 shutdown_connections 2024-04-15T15:21:21.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.974+0000 7f2af0d04640 1 --2- 172.21.15.105:0/3242940355 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2aec13e6f0 0x7f2aec154ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:21.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.974+0000 7f2af0d04640 1 --2- 172.21.15.105:0/3242940355 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f2acc06c980 0x7f2acc06ee40 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:21.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.974+0000 7f2af0d04640 1 --2- 172.21.15.105:0/3242940355 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2aec0d0060 0x7f2aec142d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:21.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.974+0000 7f2af0d04640 1 --2- 172.21.15.105:0/3242940355 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2aec06f6d0 0x7f2aec1594e0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:21.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.974+0000 7f2af0d04640 1 -- 172.21.15.105:0/3242940355 >> 172.21.15.105:0/3242940355 conn(0x7f2aec067e10 msgr2=0x7f2aec06eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:21.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.974+0000 7f2af0d04640 1 -- 172.21.15.105:0/3242940355 shutdown_connections 2024-04-15T15:21:21.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:21.974+0000 7f2af0d04640 1 -- 172.21.15.105:0/3242940355 wait complete. 2024-04-15T15:21:21.990 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-15T15:21:22.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.096+0000 7f69d42c8640 1 Processor -- start 2024-04-15T15:21:22.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.096+0000 7f69d42c8640 1 -- start start 2024-04-15T15:21:22.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.097+0000 7f69d42c8640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f69cc0b3850 0x7f69cc0b12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:22.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.097+0000 7f69d42c8640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69cc0ca8d0 con 0x7f69cc0b1800 2024-04-15T15:21:22.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.097+0000 7f69d42c8640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69cc0cabb0 con 0x7f69cc15e9c0 2024-04-15T15:21:22.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.097+0000 7f69d42c8640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f69cc0b1c30 con 0x7f69cc0b3850 2024-04-15T15:21:22.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.097+0000 7f69d1863640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f69cc0b3850 0x7f69cc0b12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:22.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.097+0000 7f69d2865640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f69cc0b1800 0x7f69cc0cf3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:45176/0 (socket says 172.21.15.105:45176) 2024-04-15T15:21:22.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.097+0000 7f69d2865640 1 -- 172.21.15.105:0/344974523 learned_addr learned my addr 172.21.15.105:0/344974523 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:21:22.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.097+0000 7f69d1863640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f69cc0b3850 0x7f69cc0b12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46998/0 (socket says 172.21.15.105:46998) 2024-04-15T15:21:22.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.097+0000 7f69d1863640 1 -- 172.21.15.105:0/344974523 >> v1:172.21.15.154:6789/0 conn(0x7f69cc15e9c0 legacy=0x7f69cc162ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:22.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.098+0000 7f69d1863640 1 -- 172.21.15.105:0/344974523 >> v1:172.21.15.105:6789/0 conn(0x7f69cc0b1800 legacy=0x7f69cc0cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:22.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.098+0000 7f69d1863640 1 -- 172.21.15.105:0/344974523 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69cc0cae90 con 0x7f69cc0b3850 2024-04-15T15:21:22.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.098+0000 7f69d1863640 1 --2- 172.21.15.105:0/344974523 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69cc0b3850 0x7f69cc0b12c0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f69c0009d90 tx=0x7f69c002bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df2d13769400fad0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:22.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.098+0000 7f69d1062640 1 -- 172.21.15.105:0/344974523 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c0039070 con 0x7f69cc0b3850 2024-04-15T15:21:22.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.098+0000 7f69d1062640 1 -- 172.21.15.105:0/344974523 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69c002cd00 con 0x7f69cc0b3850 2024-04-15T15:21:22.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.098+0000 7f69d1062640 1 -- 172.21.15.105:0/344974523 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c0034660 con 0x7f69cc0b3850 2024-04-15T15:21:22.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.099+0000 7f69d42c8640 1 -- 172.21.15.105:0/344974523 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69cc0b3850 msgr2=0x7f69cc0b12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:22.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.099+0000 7f69d42c8640 1 --2- 172.21.15.105:0/344974523 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69cc0b3850 0x7f69cc0b12c0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f69c0009d90 tx=0x7f69c002bf50 comp rx=0 tx=0).stop 2024-04-15T15:21:22.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.099+0000 7f69d42c8640 1 -- 172.21.15.105:0/344974523 shutdown_connections 2024-04-15T15:21:22.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.099+0000 7f69d42c8640 1 --2- 172.21.15.105:0/344974523 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69cc0b3850 0x7f69cc0b12c0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:22.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.099+0000 7f69d42c8640 1 -- 172.21.15.105:0/344974523 >> 172.21.15.105:0/344974523 conn(0x7f69cc09aa40 msgr2=0x7f69cc0b3140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:22.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.099+0000 7f69d42c8640 1 -- 172.21.15.105:0/344974523 shutdown_connections 2024-04-15T15:21:22.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.099+0000 7f69d42c8640 1 -- 172.21.15.105:0/344974523 wait complete. 2024-04-15T15:21:22.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.100+0000 7f69d42c8640 1 Processor -- start 2024-04-15T15:21:22.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.100+0000 7f69d42c8640 1 -- start start 2024-04-15T15:21:22.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.101+0000 7f69d42c8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f69cc0b1800 0x7f69cc1f4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:22.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.101+0000 7f69d42c8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69cc0b3850 0x7f69cc0ce300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:22.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.101+0000 7f69d42c8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69cc15e9c0 0x7f69cc0b63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:22.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.101+0000 7f69d42c8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f69cc15eda0 con 0x7f69cc15e9c0 2024-04-15T15:21:22.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.101+0000 7f69d42c8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f69cc0cb450 con 0x7f69cc0b3850 2024-04-15T15:21:22.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.101+0000 7f69d42c8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f69cc0cfae0 con 0x7f69cc0b1800 2024-04-15T15:21:22.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.101+0000 7f69d2865640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69cc15e9c0 0x7f69cc0b63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:22.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.101+0000 7f69d2865640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69cc15e9c0 0x7f69cc0b63b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47006/0 (socket says 172.21.15.105:47006) 2024-04-15T15:21:22.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.101+0000 7f69d2865640 1 -- 172.21.15.105:0/3961865921 learned_addr learned my addr 172.21.15.105:0/3961865921 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:22.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.101+0000 7f69d1863640 1 --2- 172.21.15.105:0/3961865921 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69cc0b3850 0x7f69cc0ce300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:22.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.101+0000 7f69d2064640 1 --2- 172.21.15.105:0/3961865921 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f69cc0b1800 0x7f69cc1f4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:22.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.101+0000 7f69d2865640 1 -- 172.21.15.105:0/3961865921 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f69cc0b1800 msgr2=0x7f69cc1f4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:22.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.101+0000 7f69d2865640 1 --2- 172.21.15.105:0/3961865921 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f69cc0b1800 0x7f69cc1f4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:22.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.101+0000 7f69d2865640 1 -- 172.21.15.105:0/3961865921 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69cc0b3850 msgr2=0x7f69cc0ce300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:22.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.101+0000 7f69d2865640 1 --2- 172.21.15.105:0/3961865921 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69cc0b3850 0x7f69cc0ce300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:22.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.101+0000 7f69d2865640 1 -- 172.21.15.105:0/3961865921 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69cc0b51b0 con 0x7f69cc15e9c0 2024-04-15T15:21:22.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.101+0000 7f69d2064640 1 --2- 172.21.15.105:0/3961865921 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f69cc0b1800 0x7f69cc1f4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:21:22.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.102+0000 7f69d2865640 1 --2- 172.21.15.105:0/3961865921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69cc15e9c0 0x7f69cc0b63b0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f69bc00c420 tx=0x7f69bc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:22.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.102+0000 7f69caffd640 1 -- 172.21.15.105:0/3961865921 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69bc017070 con 0x7f69cc15e9c0 2024-04-15T15:21:22.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.102+0000 7f69caffd640 1 -- 172.21.15.105:0/3961865921 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69bc005ce0 con 0x7f69cc15e9c0 2024-04-15T15:21:22.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.102+0000 7f69d42c8640 1 -- 172.21.15.105:0/3961865921 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69cc0b5490 con 0x7f69cc15e9c0 2024-04-15T15:21:22.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.102+0000 7f69caffd640 1 -- 172.21.15.105:0/3961865921 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69bc0072d0 con 0x7f69cc15e9c0 2024-04-15T15:21:22.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.102+0000 7f69d42c8640 1 -- 172.21.15.105:0/3961865921 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69cc0b5a50 con 0x7f69cc15e9c0 2024-04-15T15:21:22.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.103+0000 7f69d42c8640 1 -- 172.21.15.105:0/3961865921 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69cc068a40 con 0x7f69cc15e9c0 2024-04-15T15:21:22.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.103+0000 7f69caffd640 1 -- 172.21.15.105:0/3961865921 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f69bc01a070 con 0x7f69cc15e9c0 2024-04-15T15:21:22.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.103+0000 7f69caffd640 1 --2- 172.21.15.105:0/3961865921 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f69b806c980 0x7f69b806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:22.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.104+0000 7f69caffd640 1 -- 172.21.15.105:0/3961865921 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5443+0+0 (secure 0 0 0) 0x7f69bc08acf0 con 0x7f69cc15e9c0 2024-04-15T15:21:22.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.104+0000 7f69d2064640 1 --2- 172.21.15.105:0/3961865921 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f69b806c980 0x7f69b806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:22.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.104+0000 7f69d2064640 1 --2- 172.21.15.105:0/3961865921 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f69b806c980 0x7f69b806ee40 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f69c40015e0 tx=0x7f69c4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:22.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.108+0000 7f69caffd640 1 -- 172.21.15.105:0/3961865921 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f69bc058c70 con 0x7f69cc15e9c0 2024-04-15T15:21:22.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.286+0000 7f69d42c8640 1 -- 172.21.15.105:0/3961865921 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f69cc0b44e0 con 0x7f69cc15e9c0 2024-04-15T15:21:22.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.977+0000 7f69caffd640 1 -- 172.21.15.105:0/3961865921 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v38) ==== 169+0+0 (secure 0 0 0) 0x7f69bc05cf70 con 0x7f69cc15e9c0 2024-04-15T15:21:22.979 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_data' removed 2024-04-15T15:21:22.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.979+0000 7f69d42c8640 1 -- 172.21.15.105:0/3961865921 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f69b806c980 msgr2=0x7f69b806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:22.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.979+0000 7f69d42c8640 1 --2- 172.21.15.105:0/3961865921 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f69b806c980 0x7f69b806ee40 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f69c40015e0 tx=0x7f69c4009040 comp rx=0 tx=0).stop 2024-04-15T15:21:22.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.979+0000 7f69d42c8640 1 -- 172.21.15.105:0/3961865921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69cc15e9c0 msgr2=0x7f69cc0b63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:22.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.979+0000 7f69d42c8640 1 --2- 172.21.15.105:0/3961865921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69cc15e9c0 0x7f69cc0b63b0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f69bc00c420 tx=0x7f69bc00c8e0 comp rx=0 tx=0).stop 2024-04-15T15:21:22.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.979+0000 7f69d42c8640 1 -- 172.21.15.105:0/3961865921 shutdown_connections 2024-04-15T15:21:22.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.979+0000 7f69d42c8640 1 --2- 172.21.15.105:0/3961865921 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f69b806c980 0x7f69b806ee40 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:22.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.979+0000 7f69d42c8640 1 --2- 172.21.15.105:0/3961865921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69cc15e9c0 0x7f69cc0b63b0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:22.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.979+0000 7f69d42c8640 1 --2- 172.21.15.105:0/3961865921 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69cc0b3850 0x7f69cc0ce300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:22.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.979+0000 7f69d42c8640 1 --2- 172.21.15.105:0/3961865921 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f69cc0b1800 0x7f69cc1f4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:22.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.979+0000 7f69d42c8640 1 -- 172.21.15.105:0/3961865921 >> 172.21.15.105:0/3961865921 conn(0x7f69cc09aa40 msgr2=0x7f69cc158a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:22.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.979+0000 7f69d42c8640 1 -- 172.21.15.105:0/3961865921 shutdown_connections 2024-04-15T15:21:22.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:22.979+0000 7f69d42c8640 1 -- 172.21.15.105:0/3961865921 wait complete. 2024-04-15T15:21:22.997 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-15T15:21:23.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.101+0000 7f3655f74640 1 Processor -- start 2024-04-15T15:21:23.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.101+0000 7f3655f74640 1 -- start start 2024-04-15T15:21:23.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.102+0000 7f3655f74640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f365015e8d0 0x7f36500d2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:23.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.102+0000 7f3655f74640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f36500ba480 0x7f36500d1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:23.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.102+0000 7f3655f74640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36500d3400 con 0x7f36500d1650 2024-04-15T15:21:23.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.102+0000 7f3655f74640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f36500d1ac0 con 0x7f36500ba480 2024-04-15T15:21:23.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.102+0000 7f3655f74640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f36500d36e0 con 0x7f365015e8d0 2024-04-15T15:21:23.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.102+0000 7f364f7fe640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f365015e8d0 0x7f36500d2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:23.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.102+0000 7f3647fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f36500ba480 0x7f36500d1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:23.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.102+0000 7f364ffff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f36500d1650 0x7f36500b61d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.105:59262/0 (socket says 172.21.15.105:59262) 2024-04-15T15:21:23.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.102+0000 7f364ffff640 1 -- 172.21.15.105:0/4169864940 learned_addr learned my addr 172.21.15.105:0/4169864940 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:21:23.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.102+0000 7f364f7fe640 1 -- 172.21.15.105:0/4169864940 >> v2:172.21.15.154:3300/0 conn(0x7f36500ba480 msgr2=0x7f36500d1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:23.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.102+0000 7f364f7fe640 1 --2- 172.21.15.105:0/4169864940 >> v2:172.21.15.154:3300/0 conn(0x7f36500ba480 0x7f36500d1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:23.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.102+0000 7f364effd640 1 -- 172.21.15.105:0/4169864940 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1326952521 0 0) 0x7f36500d3400 con 0x7f36500d1650 2024-04-15T15:21:23.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.102+0000 7f364f7fe640 1 -- 172.21.15.105:0/4169864940 >> v1:172.21.15.154:6789/0 conn(0x7f36500d1650 legacy=0x7f36500b61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:23.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.102+0000 7f364f7fe640 1 -- 172.21.15.105:0/4169864940 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36500d39c0 con 0x7f365015e8d0 2024-04-15T15:21:23.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.103+0000 7f364f7fe640 1 --2- 172.21.15.105:0/4169864940 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f365015e8d0 0x7f36500d2ab0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f36400097a0 tx=0x7f364002b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ce88824b8e46421 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:23.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.103+0000 7f364effd640 1 -- 172.21.15.105:0/4169864940 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3640034070 con 0x7f365015e8d0 2024-04-15T15:21:23.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.104+0000 7f364effd640 1 -- 172.21.15.105:0/4169864940 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3640030610 con 0x7f365015e8d0 2024-04-15T15:21:23.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.104+0000 7f364effd640 1 -- 172.21.15.105:0/4169864940 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36400309e0 con 0x7f365015e8d0 2024-04-15T15:21:23.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.104+0000 7f3655f74640 1 -- 172.21.15.105:0/4169864940 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f365015e8d0 msgr2=0x7f36500d2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:23.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.104+0000 7f3655f74640 1 --2- 172.21.15.105:0/4169864940 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f365015e8d0 0x7f36500d2ab0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f36400097a0 tx=0x7f364002b960 comp rx=0 tx=0).stop 2024-04-15T15:21:23.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.104+0000 7f3655f74640 1 -- 172.21.15.105:0/4169864940 shutdown_connections 2024-04-15T15:21:23.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.104+0000 7f3655f74640 1 --2- 172.21.15.105:0/4169864940 >> v2:172.21.15.154:3300/0 conn(0x7f36500ba480 0x7f36500d1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:23.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.104+0000 7f3655f74640 1 --2- 172.21.15.105:0/4169864940 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f365015e8d0 0x7f36500d2ab0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:23.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.104+0000 7f3655f74640 1 -- 172.21.15.105:0/4169864940 >> 172.21.15.105:0/4169864940 conn(0x7f365006d460 msgr2=0x7f3650067720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:23.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.104+0000 7f3655f74640 1 -- 172.21.15.105:0/4169864940 shutdown_connections 2024-04-15T15:21:23.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.104+0000 7f3655f74640 1 -- 172.21.15.105:0/4169864940 wait complete. 2024-04-15T15:21:23.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.105+0000 7f3655f74640 1 Processor -- start 2024-04-15T15:21:23.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.106+0000 7f3655f74640 1 -- start start 2024-04-15T15:21:23.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.106+0000 7f3655f74640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36500ba480 0x7f3650157d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:23.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.106+0000 7f3655f74640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36500d1650 0x7f36500ca240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:23.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.106+0000 7f3655f74640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f365015e8d0 0x7f3650142ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:23.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.106+0000 7f3655f74640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f36500d44a0 con 0x7f36500d1650 2024-04-15T15:21:23.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.106+0000 7f3655f74640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f36500d3fc0 con 0x7f36500ba480 2024-04-15T15:21:23.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.106+0000 7f3655f74640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f36500b8890 con 0x7f365015e8d0 2024-04-15T15:21:23.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.106+0000 7f3647fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36500d1650 0x7f36500ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:23.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.106+0000 7f3647fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36500d1650 0x7f36500ca240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47022/0 (socket says 172.21.15.105:47022) 2024-04-15T15:21:23.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.106+0000 7f3647fff640 1 -- 172.21.15.105:0/401180768 learned_addr learned my addr 172.21.15.105:0/401180768 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:23.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.106+0000 7f364ffff640 1 --2- 172.21.15.105:0/401180768 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f365015e8d0 0x7f3650142ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:23.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.106+0000 7f364f7fe640 1 --2- 172.21.15.105:0/401180768 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36500ba480 0x7f3650157d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:23.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.107+0000 7f3647fff640 1 -- 172.21.15.105:0/401180768 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f365015e8d0 msgr2=0x7f3650142ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:23.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.107+0000 7f3647fff640 1 --2- 172.21.15.105:0/401180768 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f365015e8d0 0x7f3650142ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:23.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.107+0000 7f3647fff640 1 -- 172.21.15.105:0/401180768 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36500ba480 msgr2=0x7f3650157d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:23.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.107+0000 7f3647fff640 1 --2- 172.21.15.105:0/401180768 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36500ba480 0x7f3650157d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:23.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.107+0000 7f3647fff640 1 -- 172.21.15.105:0/401180768 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36501422f0 con 0x7f36500d1650 2024-04-15T15:21:23.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.107+0000 7f3647fff640 1 --2- 172.21.15.105:0/401180768 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36500d1650 0x7f36500ca240 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f363800e840 tx=0x7f363800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:23.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.107+0000 7f364cff9640 1 -- 172.21.15.105:0/401180768 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3638017070 con 0x7f36500d1650 2024-04-15T15:21:23.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.107+0000 7f364cff9640 1 -- 172.21.15.105:0/401180768 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f363800c940 con 0x7f36500d1650 2024-04-15T15:21:23.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.107+0000 7f364cff9640 1 -- 172.21.15.105:0/401180768 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f363800cd10 con 0x7f36500d1650 2024-04-15T15:21:23.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.107+0000 7f3655f74640 1 -- 172.21.15.105:0/401180768 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36501582a0 con 0x7f36500d1650 2024-04-15T15:21:23.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.107+0000 7f3655f74640 1 -- 172.21.15.105:0/401180768 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3650142590 con 0x7f36500d1650 2024-04-15T15:21:23.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.108+0000 7f3655f74640 1 -- 172.21.15.105:0/401180768 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3650068ab0 con 0x7f36500d1650 2024-04-15T15:21:23.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.108+0000 7f364cff9640 1 -- 172.21.15.105:0/401180768 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f363801a070 con 0x7f36500d1650 2024-04-15T15:21:23.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.109+0000 7f364cff9640 1 --2- 172.21.15.105:0/401180768 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f363006c980 0x7f363006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:23.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.109+0000 7f364f7fe640 1 --2- 172.21.15.105:0/401180768 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f363006c980 0x7f363006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:23.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.109+0000 7f364cff9640 1 -- 172.21.15.105:0/401180768 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5029+0+0 (secure 0 0 0) 0x7f363808b6c0 con 0x7f36500d1650 2024-04-15T15:21:23.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.110+0000 7f364f7fe640 1 --2- 172.21.15.105:0/401180768 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f363006c980 0x7f363006ee40 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f3640036000 tx=0x7f364003b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:23.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.114+0000 7f364cff9640 1 -- 172.21.15.105:0/401180768 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3638059750 con 0x7f36500d1650 2024-04-15T15:21:23.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.291+0000 7f3655f74640 1 -- 172.21.15.105:0/401180768 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f36500d2dd0 con 0x7f36500d1650 2024-04-15T15:21:23.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.294+0000 7f364cff9640 1 -- 172.21.15.105:0/401180768 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) ==== 104+0+0 (secure 0 0 0) 0x7f363805da50 con 0x7f36500d1650 2024-04-15T15:21:23.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.297+0000 7f3655f74640 1 -- 172.21.15.105:0/401180768 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f363006c980 msgr2=0x7f363006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:23.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.297+0000 7f3655f74640 1 --2- 172.21.15.105:0/401180768 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f363006c980 0x7f363006ee40 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f3640036000 tx=0x7f364003b040 comp rx=0 tx=0).stop 2024-04-15T15:21:23.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.297+0000 7f3655f74640 1 -- 172.21.15.105:0/401180768 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36500d1650 msgr2=0x7f36500ca240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:23.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.297+0000 7f3655f74640 1 --2- 172.21.15.105:0/401180768 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36500d1650 0x7f36500ca240 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f363800e840 tx=0x7f363800ed00 comp rx=0 tx=0).stop 2024-04-15T15:21:23.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.297+0000 7f3655f74640 1 -- 172.21.15.105:0/401180768 shutdown_connections 2024-04-15T15:21:23.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.297+0000 7f3655f74640 1 --2- 172.21.15.105:0/401180768 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f365015e8d0 0x7f3650142ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:23.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.297+0000 7f3655f74640 1 --2- 172.21.15.105:0/401180768 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f363006c980 0x7f363006ee40 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:23.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.297+0000 7f3655f74640 1 --2- 172.21.15.105:0/401180768 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36500d1650 0x7f36500ca240 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:23.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.297+0000 7f3655f74640 1 --2- 172.21.15.105:0/401180768 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36500ba480 0x7f3650157d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:23.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.297+0000 7f3655f74640 1 -- 172.21.15.105:0/401180768 >> 172.21.15.105:0/401180768 conn(0x7f365006d460 msgr2=0x7f365015dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:23.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.297+0000 7f3655f74640 1 -- 172.21.15.105:0/401180768 shutdown_connections 2024-04-15T15:21:23.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.297+0000 7f3655f74640 1 -- 172.21.15.105:0/401180768 wait complete. 2024-04-15T15:21:23.310 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_cache_quiesce_splitauth 2024-04-15T15:21:23.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.412+0000 7ffb2a87b640 1 Processor -- start 2024-04-15T15:21:23.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.412+0000 7ffb2a87b640 1 -- start start 2024-04-15T15:21:23.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.413+0000 7ffb2a87b640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ffb24072570 0x7ffb2406de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:23.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.413+0000 7ffb2a87b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ffb2406e3c0 0x7ffb2415d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:23.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.413+0000 7ffb2a87b640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb2406ebb0 con 0x7ffb24145f00 2024-04-15T15:21:23.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.413+0000 7ffb2a87b640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7ffb241462e0 con 0x7ffb2406e3c0 2024-04-15T15:21:23.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.413+0000 7ffb2a87b640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7ffb2406ee90 con 0x7ffb24072570 2024-04-15T15:21:23.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.413+0000 7ffb23fff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7ffb24145f00 0x7ffb24071e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:45178/0 (socket says 172.21.15.105:45178) 2024-04-15T15:21:23.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.413+0000 7ffb23fff640 1 -- 172.21.15.105:0/713222001 learned_addr learned my addr 172.21.15.105:0/713222001 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:21:23.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.413+0000 7ffb28e18640 1 --2- 172.21.15.105:0/713222001 >> v2:172.21.15.105:3300/0 conn(0x7ffb2406e3c0 0x7ffb2415d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:23.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.413+0000 7ffb237fe640 1 -- 172.21.15.105:0/713222001 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1486933497 0 0) 0x7ffb2406ebb0 con 0x7ffb24145f00 2024-04-15T15:21:23.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.414+0000 7ffb237fe640 1 -- 172.21.15.105:0/713222001 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb10003520 con 0x7ffb24145f00 2024-04-15T15:21:23.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.414+0000 7ffb28e18640 1 -- 172.21.15.105:0/713222001 >> v2:172.21.15.154:3300/0 conn(0x7ffb24072570 msgr2=0x7ffb2406de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:23.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.414+0000 7ffb1bfff640 1 --2- 172.21.15.105:0/713222001 >> v2:172.21.15.154:3300/0 conn(0x7ffb24072570 0x7ffb2406de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:23.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.414+0000 7ffb28e18640 1 --2- 172.21.15.105:0/713222001 >> v2:172.21.15.154:3300/0 conn(0x7ffb24072570 0x7ffb2406de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:23.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.414+0000 7ffb28e18640 1 -- 172.21.15.105:0/713222001 >> v1:172.21.15.105:6789/0 conn(0x7ffb24145f00 legacy=0x7ffb24071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:23.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.414+0000 7ffb28e18640 1 -- 172.21.15.105:0/713222001 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb2415cde0 con 0x7ffb2406e3c0 2024-04-15T15:21:23.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.414+0000 7ffb1bfff640 1 --2- 172.21.15.105:0/713222001 >> v2:172.21.15.154:3300/0 conn(0x7ffb24072570 0x7ffb2406de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:21:23.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.414+0000 7ffb28e18640 1 --2- 172.21.15.105:0/713222001 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb2406e3c0 0x7ffb2415d5e0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7ffb240a94e0 tx=0x7ffb1c02bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a44412a6513b0bab server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:23.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.414+0000 7ffb237fe640 1 -- 172.21.15.105:0/713222001 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb1c039070 con 0x7ffb2406e3c0 2024-04-15T15:21:23.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.414+0000 7ffb237fe640 1 -- 172.21.15.105:0/713222001 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb1c02cb60 con 0x7ffb2406e3c0 2024-04-15T15:21:23.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.415+0000 7ffb237fe640 1 -- 172.21.15.105:0/713222001 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb1c034440 con 0x7ffb2406e3c0 2024-04-15T15:21:23.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.415+0000 7ffb2a87b640 1 -- 172.21.15.105:0/713222001 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb2406e3c0 msgr2=0x7ffb2415d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:23.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.415+0000 7ffb2a87b640 1 --2- 172.21.15.105:0/713222001 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb2406e3c0 0x7ffb2415d5e0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7ffb240a94e0 tx=0x7ffb1c02bdb0 comp rx=0 tx=0).stop 2024-04-15T15:21:23.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.415+0000 7ffb2a87b640 1 -- 172.21.15.105:0/713222001 shutdown_connections 2024-04-15T15:21:23.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.415+0000 7ffb2a87b640 1 --2- 172.21.15.105:0/713222001 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb2406e3c0 0x7ffb2415d5e0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:23.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.415+0000 7ffb2a87b640 1 --2- 172.21.15.105:0/713222001 >> v2:172.21.15.154:3300/0 conn(0x7ffb24072570 0x7ffb2406de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:23.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.415+0000 7ffb2a87b640 1 -- 172.21.15.105:0/713222001 >> 172.21.15.105:0/713222001 conn(0x7ffb24067d50 msgr2=0x7ffb2406b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:23.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.415+0000 7ffb2a87b640 1 -- 172.21.15.105:0/713222001 shutdown_connections 2024-04-15T15:21:23.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.415+0000 7ffb2a87b640 1 -- 172.21.15.105:0/713222001 wait complete. 2024-04-15T15:21:23.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.416+0000 7ffb2a87b640 1 Processor -- start 2024-04-15T15:21:23.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.416+0000 7ffb2a87b640 1 -- start start 2024-04-15T15:21:23.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.417+0000 7ffb2a87b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb2406e3c0 0x7ffb2415be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:23.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.417+0000 7ffb2a87b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb24072570 0x7ffb240cab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:23.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.417+0000 7ffb2a87b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffb24145f00 0x7ffb241429a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:23.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.417+0000 7ffb2a87b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ffb2415db20 con 0x7ffb24072570 2024-04-15T15:21:23.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.417+0000 7ffb2a87b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ffb2406e830 con 0x7ffb2406e3c0 2024-04-15T15:21:23.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.417+0000 7ffb2a87b640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7ffb2415dca0 con 0x7ffb24145f00 2024-04-15T15:21:23.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.417+0000 7ffb1bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb24072570 0x7ffb240cab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:23.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.417+0000 7ffb1bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb24072570 0x7ffb240cab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47044/0 (socket says 172.21.15.105:47044) 2024-04-15T15:21:23.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.417+0000 7ffb1bfff640 1 -- 172.21.15.105:0/705575892 learned_addr learned my addr 172.21.15.105:0/705575892 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:23.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.417+0000 7ffb28e18640 1 --2- 172.21.15.105:0/705575892 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffb24145f00 0x7ffb241429a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:23.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.417+0000 7ffb23fff640 1 --2- 172.21.15.105:0/705575892 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb2406e3c0 0x7ffb2415be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:23.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.417+0000 7ffb1bfff640 1 -- 172.21.15.105:0/705575892 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffb24145f00 msgr2=0x7ffb241429a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:23.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.417+0000 7ffb1bfff640 1 --2- 172.21.15.105:0/705575892 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffb24145f00 0x7ffb241429a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:23.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.417+0000 7ffb1bfff640 1 -- 172.21.15.105:0/705575892 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb2406e3c0 msgr2=0x7ffb2415be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:23.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.418+0000 7ffb1bfff640 1 --2- 172.21.15.105:0/705575892 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb2406e3c0 0x7ffb2415be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:23.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.418+0000 7ffb1bfff640 1 -- 172.21.15.105:0/705575892 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb240cb560 con 0x7ffb24072570 2024-04-15T15:21:23.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.418+0000 7ffb28e18640 1 --2- 172.21.15.105:0/705575892 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffb24145f00 0x7ffb241429a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:21:23.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.418+0000 7ffb1bfff640 1 --2- 172.21.15.105:0/705575892 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb24072570 0x7ffb240cab10 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c00c820 tx=0x7ffb0c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:23.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.418+0000 7ffb217fa640 1 -- 172.21.15.105:0/705575892 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c015070 con 0x7ffb24072570 2024-04-15T15:21:23.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.418+0000 7ffb217fa640 1 -- 172.21.15.105:0/705575892 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb0c007a30 con 0x7ffb24072570 2024-04-15T15:21:23.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.418+0000 7ffb2a87b640 1 -- 172.21.15.105:0/705575892 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb240cb840 con 0x7ffb24072570 2024-04-15T15:21:23.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.418+0000 7ffb217fa640 1 -- 172.21.15.105:0/705575892 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c007e00 con 0x7ffb24072570 2024-04-15T15:21:23.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.418+0000 7ffb2a87b640 1 -- 172.21.15.105:0/705575892 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb240cbe00 con 0x7ffb24072570 2024-04-15T15:21:23.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.419+0000 7ffb2a87b640 1 -- 172.21.15.105:0/705575892 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb24068ab0 con 0x7ffb24072570 2024-04-15T15:21:23.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.419+0000 7ffb217fa640 1 -- 172.21.15.105:0/705575892 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7ffb0c018070 con 0x7ffb24072570 2024-04-15T15:21:23.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.420+0000 7ffb217fa640 1 --2- 172.21.15.105:0/705575892 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ffb0406c980 0x7ffb0406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:23.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.420+0000 7ffb23fff640 1 --2- 172.21.15.105:0/705575892 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ffb0406c980 0x7ffb0406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:23.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.420+0000 7ffb217fa640 1 -- 172.21.15.105:0/705575892 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5029+0+0 (secure 0 0 0) 0x7ffb0c08b930 con 0x7ffb24072570 2024-04-15T15:21:23.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.420+0000 7ffb23fff640 1 --2- 172.21.15.105:0/705575892 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ffb0406c980 0x7ffb0406ee40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ffb14005880 tx=0x7ffb14009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:23.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.424+0000 7ffb217fa640 1 -- 172.21.15.105:0/705575892 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ffb0c059f90 con 0x7ffb24072570 2024-04-15T15:21:23.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.602+0000 7ffb2a87b640 1 -- 172.21.15.105:0/705575892 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_cache_quiesce_splitauth"} v 0) -- 0x7ffb240693a0 con 0x7ffb24072570 2024-04-15T15:21:23.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.607+0000 7ffb217fa640 1 -- 172.21.15.105:0/705575892 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_cache_quiesce_splitauth"}]=0 v14) ==== 110+0+0 (secure 0 0 0) 0x7ffb0c059890 con 0x7ffb24072570 2024-04-15T15:21:23.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.609+0000 7ffb2a87b640 1 -- 172.21.15.105:0/705575892 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ffb0406c980 msgr2=0x7ffb0406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:23.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.609+0000 7ffb2a87b640 1 --2- 172.21.15.105:0/705575892 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ffb0406c980 0x7ffb0406ee40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ffb14005880 tx=0x7ffb14009040 comp rx=0 tx=0).stop 2024-04-15T15:21:23.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.609+0000 7ffb2a87b640 1 -- 172.21.15.105:0/705575892 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb24072570 msgr2=0x7ffb240cab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:23.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.609+0000 7ffb2a87b640 1 --2- 172.21.15.105:0/705575892 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb24072570 0x7ffb240cab10 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c00c820 tx=0x7ffb0c00cce0 comp rx=0 tx=0).stop 2024-04-15T15:21:23.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.609+0000 7ffb2a87b640 1 -- 172.21.15.105:0/705575892 shutdown_connections 2024-04-15T15:21:23.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.609+0000 7ffb2a87b640 1 --2- 172.21.15.105:0/705575892 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffb24145f00 0x7ffb241429a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:23.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.609+0000 7ffb2a87b640 1 --2- 172.21.15.105:0/705575892 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7ffb0406c980 0x7ffb0406ee40 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:23.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.609+0000 7ffb2a87b640 1 --2- 172.21.15.105:0/705575892 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb24072570 0x7ffb240cab10 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:23.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.609+0000 7ffb2a87b640 1 --2- 172.21.15.105:0/705575892 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb2406e3c0 0x7ffb2415be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:23.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.609+0000 7ffb2a87b640 1 -- 172.21.15.105:0/705575892 >> 172.21.15.105:0/705575892 conn(0x7ffb24067d50 msgr2=0x7ffb240b2910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:23.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.609+0000 7ffb2a87b640 1 -- 172.21.15.105:0/705575892 shutdown_connections 2024-04-15T15:21:23.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.609+0000 7ffb2a87b640 1 -- 172.21.15.105:0/705575892 wait complete. 2024-04-15T15:21:23.622 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds debug_mds 2024-04-15T15:21:23.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.726+0000 7f4c9ec1b640 1 Processor -- start 2024-04-15T15:21:23.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.726+0000 7f4c9ec1b640 1 -- start start 2024-04-15T15:21:23.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.727+0000 7f4c9ec1b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4c9809aa40 0x7f4c9815d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:23.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.727+0000 7f4c9ec1b640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4c9809b8f0 0x7f4c980cd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:23.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.727+0000 7f4c9ec1b640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c9815caf0 con 0x7f4c980d12c0 2024-04-15T15:21:23.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.727+0000 7f4c9ec1b640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f4c980d0e30 con 0x7f4c9809aa40 2024-04-15T15:21:23.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.727+0000 7f4c9ec1b640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f4c9815dbe0 con 0x7f4c9809b8f0 2024-04-15T15:21:23.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.727+0000 7f4c9c9b7640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4c9809aa40 0x7f4c9815d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:23.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.727+0000 7f4c97fff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f4c980d12c0 0x7f4c9809ae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:45194/0 (socket says 172.21.15.105:45194) 2024-04-15T15:21:23.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.727+0000 7f4c97fff640 1 -- 172.21.15.105:0/699557338 learned_addr learned my addr 172.21.15.105:0/699557338 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:21:23.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.728+0000 7f4c9c9b7640 1 -- 172.21.15.105:0/699557338 >> v2:172.21.15.154:3300/0 conn(0x7f4c9809b8f0 msgr2=0x7f4c980cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:23.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.728+0000 7f4c9c9b7640 1 --2- 172.21.15.105:0/699557338 >> v2:172.21.15.154:3300/0 conn(0x7f4c9809b8f0 0x7f4c980cd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:23.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.728+0000 7f4c9c9b7640 1 -- 172.21.15.105:0/699557338 >> v1:172.21.15.105:6789/0 conn(0x7f4c980d12c0 legacy=0x7f4c9809ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:23.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.728+0000 7f4c977fe640 1 -- 172.21.15.105:0/699557338 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2449239741 0 0) 0x7f4c9815caf0 con 0x7f4c980d12c0 2024-04-15T15:21:23.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.728+0000 7f4c9c9b7640 1 -- 172.21.15.105:0/699557338 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c980d0220 con 0x7f4c9809aa40 2024-04-15T15:21:23.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.728+0000 7f4c9c9b7640 1 --2- 172.21.15.105:0/699557338 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c9809aa40 0x7f4c9815d6a0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c0097a0 tx=0x7f4c8c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73404e23b894c0df server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:23.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.729+0000 7f4c977fe640 1 -- 172.21.15.105:0/699557338 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c0315a0 con 0x7f4c9809aa40 2024-04-15T15:21:23.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.729+0000 7f4c977fe640 1 -- 172.21.15.105:0/699557338 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c8c037070 con 0x7f4c9809aa40 2024-04-15T15:21:23.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.729+0000 7f4c977fe640 1 -- 172.21.15.105:0/699557338 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c031960 con 0x7f4c9809aa40 2024-04-15T15:21:23.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.729+0000 7f4c9ec1b640 1 -- 172.21.15.105:0/699557338 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c9809aa40 msgr2=0x7f4c9815d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:23.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.729+0000 7f4c9ec1b640 1 --2- 172.21.15.105:0/699557338 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c9809aa40 0x7f4c9815d6a0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c0097a0 tx=0x7f4c8c02b840 comp rx=0 tx=0).stop 2024-04-15T15:21:23.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.729+0000 7f4c9ec1b640 1 -- 172.21.15.105:0/699557338 shutdown_connections 2024-04-15T15:21:23.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.729+0000 7f4c9ec1b640 1 --2- 172.21.15.105:0/699557338 >> v2:172.21.15.154:3300/0 conn(0x7f4c9809b8f0 0x7f4c980cd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:23.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.729+0000 7f4c9ec1b640 1 --2- 172.21.15.105:0/699557338 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c9809aa40 0x7f4c9815d6a0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:23.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.729+0000 7f4c9ec1b640 1 -- 172.21.15.105:0/699557338 >> 172.21.15.105:0/699557338 conn(0x7f4c980b4030 msgr2=0x7f4c980b4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:23.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.729+0000 7f4c9ec1b640 1 -- 172.21.15.105:0/699557338 shutdown_connections 2024-04-15T15:21:23.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.729+0000 7f4c9ec1b640 1 -- 172.21.15.105:0/699557338 wait complete. 2024-04-15T15:21:23.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.730+0000 7f4c9ec1b640 1 Processor -- start 2024-04-15T15:21:23.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.730+0000 7f4c9ec1b640 1 -- start start 2024-04-15T15:21:23.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.731+0000 7f4c9ec1b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4c9809aa40 0x7f4c980b6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:23.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.731+0000 7f4c9ec1b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c9809b8f0 0x7f4c980b5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:23.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.731+0000 7f4c9ec1b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c980d12c0 0x7f4c981607d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:23.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.731+0000 7f4c9ec1b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4c980d0ac0 con 0x7f4c980d12c0 2024-04-15T15:21:23.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.731+0000 7f4c9ec1b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4c980d07e0 con 0x7f4c9809b8f0 2024-04-15T15:21:23.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.731+0000 7f4c9ec1b640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f4c980cd620 con 0x7f4c9809aa40 2024-04-15T15:21:23.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.731+0000 7f4c9d1b8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c980d12c0 0x7f4c981607d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:23.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.731+0000 7f4c9d1b8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c980d12c0 0x7f4c981607d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47064/0 (socket says 172.21.15.105:47064) 2024-04-15T15:21:23.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.731+0000 7f4c9d1b8640 1 -- 172.21.15.105:0/563025368 learned_addr learned my addr 172.21.15.105:0/563025368 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:23.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.732+0000 7f4c97fff640 1 --2- 172.21.15.105:0/563025368 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c9809b8f0 0x7f4c980b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:23.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.732+0000 7f4c9c9b7640 1 --2- 172.21.15.105:0/563025368 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4c9809aa40 0x7f4c980b6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:23.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.732+0000 7f4c9d1b8640 1 -- 172.21.15.105:0/563025368 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4c9809aa40 msgr2=0x7f4c980b6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:23.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.732+0000 7f4c9d1b8640 1 --2- 172.21.15.105:0/563025368 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4c9809aa40 0x7f4c980b6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:23.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.732+0000 7f4c9d1b8640 1 -- 172.21.15.105:0/563025368 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c9809b8f0 msgr2=0x7f4c980b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:23.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.732+0000 7f4c9d1b8640 1 --2- 172.21.15.105:0/563025368 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c9809b8f0 0x7f4c980b5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:23.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.732+0000 7f4c9d1b8640 1 -- 172.21.15.105:0/563025368 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c980b66e0 con 0x7f4c980d12c0 2024-04-15T15:21:23.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.732+0000 7f4c9d1b8640 1 --2- 172.21.15.105:0/563025368 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c980d12c0 0x7f4c981607d0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f4c9000b970 tx=0x7f4c9000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:23.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.732+0000 7f4c957fa640 1 -- 172.21.15.105:0/563025368 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c90019070 con 0x7f4c980d12c0 2024-04-15T15:21:23.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.732+0000 7f4c957fa640 1 -- 172.21.15.105:0/563025368 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c9000cb60 con 0x7f4c980d12c0 2024-04-15T15:21:23.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.732+0000 7f4c9ec1b640 1 -- 172.21.15.105:0/563025368 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c980b69c0 con 0x7f4c980d12c0 2024-04-15T15:21:23.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.732+0000 7f4c957fa640 1 -- 172.21.15.105:0/563025368 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c90014440 con 0x7f4c980d12c0 2024-04-15T15:21:23.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.732+0000 7f4c9ec1b640 1 -- 172.21.15.105:0/563025368 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c981e75c0 con 0x7f4c980d12c0 2024-04-15T15:21:23.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.733+0000 7f4c9ec1b640 1 -- 172.21.15.105:0/563025368 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c981e77c0 con 0x7f4c980d12c0 2024-04-15T15:21:23.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.733+0000 7f4c957fa640 1 -- 172.21.15.105:0/563025368 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f4c9001c070 con 0x7f4c980d12c0 2024-04-15T15:21:23.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.734+0000 7f4c957fa640 1 --2- 172.21.15.105:0/563025368 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f4c7c06c980 0x7f4c7c06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:23.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.734+0000 7f4c957fa640 1 -- 172.21.15.105:0/563025368 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5029+0+0 (secure 0 0 0) 0x7f4c9008b580 con 0x7f4c980d12c0 2024-04-15T15:21:23.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.734+0000 7f4c9c9b7640 1 --2- 172.21.15.105:0/563025368 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f4c7c06c980 0x7f4c7c06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:23.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.734+0000 7f4c9c9b7640 1 --2- 172.21.15.105:0/563025368 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f4c7c06c980 0x7f4c7c06ee40 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c036000 tx=0x7f4c8c03d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:23.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.738+0000 7f4c957fa640 1 -- 172.21.15.105:0/563025368 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4c981e77c0 con 0x7f4c980d12c0 2024-04-15T15:21:23.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.918+0000 7f4c9ec1b640 1 -- 172.21.15.105:0/563025368 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "debug_mds"} v 0) -- 0x7f4c981e70d0 con 0x7f4c980d12c0 2024-04-15T15:21:23.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.921+0000 7f4c957fa640 1 -- 172.21.15.105:0/563025368 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "debug_mds"}]=0 v15) ==== 92+0+0 (secure 0 0 0) 0x7f4c90059cf0 con 0x7f4c980d12c0 2024-04-15T15:21:23.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.923+0000 7f4c9ec1b640 1 -- 172.21.15.105:0/563025368 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f4c7c06c980 msgr2=0x7f4c7c06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:23.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.923+0000 7f4c9ec1b640 1 --2- 172.21.15.105:0/563025368 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f4c7c06c980 0x7f4c7c06ee40 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c036000 tx=0x7f4c8c03d040 comp rx=0 tx=0).stop 2024-04-15T15:21:23.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.923+0000 7f4c9ec1b640 1 -- 172.21.15.105:0/563025368 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c980d12c0 msgr2=0x7f4c981607d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:23.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.923+0000 7f4c9ec1b640 1 --2- 172.21.15.105:0/563025368 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c980d12c0 0x7f4c981607d0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f4c9000b970 tx=0x7f4c9000be30 comp rx=0 tx=0).stop 2024-04-15T15:21:23.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.923+0000 7f4c9ec1b640 1 -- 172.21.15.105:0/563025368 shutdown_connections 2024-04-15T15:21:23.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.923+0000 7f4c9ec1b640 1 --2- 172.21.15.105:0/563025368 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f4c7c06c980 0x7f4c7c06ee40 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:23.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.923+0000 7f4c9ec1b640 1 --2- 172.21.15.105:0/563025368 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c980d12c0 0x7f4c981607d0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:23.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.923+0000 7f4c9ec1b640 1 --2- 172.21.15.105:0/563025368 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c9809b8f0 0x7f4c980b5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:23.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.923+0000 7f4c9ec1b640 1 --2- 172.21.15.105:0/563025368 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4c9809aa40 0x7f4c980b6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:23.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.923+0000 7f4c9ec1b640 1 -- 172.21.15.105:0/563025368 >> 172.21.15.105:0/563025368 conn(0x7f4c980b4030 msgr2=0x7f4c98071060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:23.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.923+0000 7f4c9ec1b640 1 -- 172.21.15.105:0/563025368 shutdown_connections 2024-04-15T15:21:23.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:23.923+0000 7f4c9ec1b640 1 -- 172.21.15.105:0/563025368 wait complete. 2024-04-15T15:21:23.936 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_quiesce.TestQuiesce.test_quiesce_blocked' 2024-04-15T15:21:24.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.039+0000 7f9c992d9640 1 Processor -- start 2024-04-15T15:21:24.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.039+0000 7f9c992d9640 1 -- start start 2024-04-15T15:21:24.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.040+0000 7f9c992d9640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9c9415e8d0 0x7f9c940d2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:24.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.040+0000 7f9c992d9640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c940d3400 con 0x7f9c940ba480 2024-04-15T15:21:24.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.040+0000 7f9c992d9640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c940d36e0 con 0x7f9c940d1820 2024-04-15T15:21:24.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.040+0000 7f9c992d9640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f9c940d39c0 con 0x7f9c9415e8d0 2024-04-15T15:21:24.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.040+0000 7f9c92ffd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9c9415e8d0 0x7f9c940d2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:24.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.040+0000 7f9c927fc640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f9c940ba480 0x7f9c940d1110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:45200/0 (socket says 172.21.15.105:45200) 2024-04-15T15:21:24.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.040+0000 7f9c92ffd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9c9415e8d0 0x7f9c940d2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47076/0 (socket says 172.21.15.105:47076) 2024-04-15T15:21:24.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.040+0000 7f9c927fc640 1 -- 172.21.15.105:0/3552591613 learned_addr learned my addr 172.21.15.105:0/3552591613 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:21:24.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.040+0000 7f9c92ffd640 1 -- 172.21.15.105:0/3552591613 >> v1:172.21.15.154:6789/0 conn(0x7f9c940d1820 legacy=0x7f9c940b7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:24.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.040+0000 7f9c92ffd640 1 -- 172.21.15.105:0/3552591613 >> v1:172.21.15.105:6789/0 conn(0x7f9c940ba480 legacy=0x7f9c940d1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:24.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.040+0000 7f9c92ffd640 1 -- 172.21.15.105:0/3552591613 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c940d3ca0 con 0x7f9c9415e8d0 2024-04-15T15:21:24.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.041+0000 7f9c92ffd640 1 --2- 172.21.15.105:0/3552591613 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c9415e8d0 0x7f9c940d2ab0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f9c84009dc0 tx=0x7f9c8402bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a86ada875cb8f9f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:24.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.041+0000 7f9c91ffb640 1 -- 172.21.15.105:0/3552591613 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c84039070 con 0x7f9c9415e8d0 2024-04-15T15:21:24.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.041+0000 7f9c91ffb640 1 -- 172.21.15.105:0/3552591613 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c8402ccd0 con 0x7f9c9415e8d0 2024-04-15T15:21:24.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.041+0000 7f9c91ffb640 1 -- 172.21.15.105:0/3552591613 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c84034660 con 0x7f9c9415e8d0 2024-04-15T15:21:24.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.042+0000 7f9c992d9640 1 -- 172.21.15.105:0/3552591613 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c9415e8d0 msgr2=0x7f9c940d2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:24.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.042+0000 7f9c992d9640 1 --2- 172.21.15.105:0/3552591613 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c9415e8d0 0x7f9c940d2ab0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f9c84009dc0 tx=0x7f9c8402bf20 comp rx=0 tx=0).stop 2024-04-15T15:21:24.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.042+0000 7f9c992d9640 1 -- 172.21.15.105:0/3552591613 shutdown_connections 2024-04-15T15:21:24.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.042+0000 7f9c992d9640 1 --2- 172.21.15.105:0/3552591613 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c9415e8d0 0x7f9c940d2ab0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:24.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.042+0000 7f9c992d9640 1 -- 172.21.15.105:0/3552591613 >> 172.21.15.105:0/3552591613 conn(0x7f9c9406d460 msgr2=0x7f9c94067720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:24.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.042+0000 7f9c992d9640 1 -- 172.21.15.105:0/3552591613 shutdown_connections 2024-04-15T15:21:24.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.042+0000 7f9c992d9640 1 -- 172.21.15.105:0/3552591613 wait complete. 2024-04-15T15:21:24.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.043+0000 7f9c992d9640 1 Processor -- start 2024-04-15T15:21:24.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.043+0000 7f9c992d9640 1 -- start start 2024-04-15T15:21:24.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.044+0000 7f9c992d9640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9c940ba480 0x7f9c941ea250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:24.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.044+0000 7f9c992d9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c940d1820 0x7f9c941ea790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:24.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.044+0000 7f9c992d9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9c9415e8d0 0x7f9c940cea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:24.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.044+0000 7f9c992d9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9c940d2ff0 con 0x7f9c940d1820 2024-04-15T15:21:24.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.044+0000 7f9c992d9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9c940d44e0 con 0x7f9c9415e8d0 2024-04-15T15:21:24.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.044+0000 7f9c992d9640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f9c940d3170 con 0x7f9c940ba480 2024-04-15T15:21:24.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.044+0000 7f9c927fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c940d1820 0x7f9c941ea790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:24.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.044+0000 7f9c927fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c940d1820 0x7f9c941ea790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47092/0 (socket says 172.21.15.105:47092) 2024-04-15T15:21:24.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.044+0000 7f9c927fc640 1 -- 172.21.15.105:0/3939786090 learned_addr learned my addr 172.21.15.105:0/3939786090 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:24.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.044+0000 7f9c92ffd640 1 --2- 172.21.15.105:0/3939786090 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9c940ba480 0x7f9c941ea250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:24.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.044+0000 7f9c937fe640 1 --2- 172.21.15.105:0/3939786090 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9c9415e8d0 0x7f9c940cea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:24.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.044+0000 7f9c927fc640 1 -- 172.21.15.105:0/3939786090 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9c940ba480 msgr2=0x7f9c941ea250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:24.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.044+0000 7f9c927fc640 1 --2- 172.21.15.105:0/3939786090 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9c940ba480 0x7f9c941ea250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:24.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.044+0000 7f9c927fc640 1 -- 172.21.15.105:0/3939786090 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9c9415e8d0 msgr2=0x7f9c940cea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:24.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.044+0000 7f9c927fc640 1 --2- 172.21.15.105:0/3939786090 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9c9415e8d0 0x7f9c940cea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:24.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.044+0000 7f9c927fc640 1 -- 172.21.15.105:0/3939786090 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c94149cb0 con 0x7f9c940d1820 2024-04-15T15:21:24.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.045+0000 7f9c927fc640 1 --2- 172.21.15.105:0/3939786090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c940d1820 0x7f9c941ea790 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f9c8800c4a0 tx=0x7f9c8800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:24.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.045+0000 7f9c7b7fe640 1 -- 172.21.15.105:0/3939786090 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c88017070 con 0x7f9c940d1820 2024-04-15T15:21:24.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.045+0000 7f9c7b7fe640 1 -- 172.21.15.105:0/3939786090 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c88005ce0 con 0x7f9c940d1820 2024-04-15T15:21:24.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.045+0000 7f9c992d9640 1 -- 172.21.15.105:0/3939786090 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c94149f90 con 0x7f9c940d1820 2024-04-15T15:21:24.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.045+0000 7f9c7b7fe640 1 -- 172.21.15.105:0/3939786090 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c88007270 con 0x7f9c940d1820 2024-04-15T15:21:24.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.045+0000 7f9c992d9640 1 -- 172.21.15.105:0/3939786090 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c941e8230 con 0x7f9c940d1820 2024-04-15T15:21:24.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.046+0000 7f9c992d9640 1 -- 172.21.15.105:0/3939786090 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c94068a40 con 0x7f9c940d1820 2024-04-15T15:21:24.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.046+0000 7f9c7b7fe640 1 -- 172.21.15.105:0/3939786090 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f9c8801a070 con 0x7f9c940d1820 2024-04-15T15:21:24.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.047+0000 7f9c7b7fe640 1 --2- 172.21.15.105:0/3939786090 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f9c8006c9b0 0x7f9c8006ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:24.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.047+0000 7f9c7b7fe640 1 -- 172.21.15.105:0/3939786090 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5029+0+0 (secure 0 0 0) 0x7f9c8808aab0 con 0x7f9c940d1820 2024-04-15T15:21:24.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.047+0000 7f9c92ffd640 1 --2- 172.21.15.105:0/3939786090 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f9c8006c9b0 0x7f9c8006ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:24.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.047+0000 7f9c92ffd640 1 --2- 172.21.15.105:0/3939786090 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f9c8006c9b0 0x7f9c8006ee70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9c84036000 tx=0x7f9c84030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:24.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.051+0000 7f9c7b7fe640 1 -- 172.21.15.105:0/3939786090 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9c88058b50 con 0x7f9c940d1820 2024-04-15T15:21:24.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.291+0000 7f9c992d9640 1 -- 172.21.15.105:0/3939786090 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_quiesce.TestQuiesce.test_quiesce_blocked"]} v 0) -- 0x7f9c940ba110 con 0x7f9c940d1820 2024-04-15T15:21:24.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.981+0000 7f9c7b7fe640 1 -- 172.21.15.105:0/3939786090 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_quiesce.TestQuiesce.test_quiesce_blocked"]}]=0 v129) ==== 137+0+0 (secure 0 0 0) 0x7f9c8805ce50 con 0x7f9c940d1820 2024-04-15T15:21:24.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.984+0000 7f9c992d9640 1 -- 172.21.15.105:0/3939786090 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f9c8006c9b0 msgr2=0x7f9c8006ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:24.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.984+0000 7f9c992d9640 1 --2- 172.21.15.105:0/3939786090 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f9c8006c9b0 0x7f9c8006ee70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9c84036000 tx=0x7f9c84030000 comp rx=0 tx=0).stop 2024-04-15T15:21:24.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.984+0000 7f9c992d9640 1 -- 172.21.15.105:0/3939786090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c940d1820 msgr2=0x7f9c941ea790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:24.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.984+0000 7f9c992d9640 1 --2- 172.21.15.105:0/3939786090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c940d1820 0x7f9c941ea790 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f9c8800c4a0 tx=0x7f9c8800c960 comp rx=0 tx=0).stop 2024-04-15T15:21:24.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.984+0000 7f9c992d9640 1 -- 172.21.15.105:0/3939786090 shutdown_connections 2024-04-15T15:21:24.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.984+0000 7f9c992d9640 1 --2- 172.21.15.105:0/3939786090 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f9c8006c9b0 0x7f9c8006ee70 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:24.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.984+0000 7f9c992d9640 1 --2- 172.21.15.105:0/3939786090 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9c9415e8d0 0x7f9c940cea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:24.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.984+0000 7f9c992d9640 1 --2- 172.21.15.105:0/3939786090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c940d1820 0x7f9c941ea790 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:24.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.984+0000 7f9c992d9640 1 --2- 172.21.15.105:0/3939786090 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9c940ba480 0x7f9c941ea250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:24.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.984+0000 7f9c992d9640 1 -- 172.21.15.105:0/3939786090 >> 172.21.15.105:0/3939786090 conn(0x7f9c9406d460 msgr2=0x7f9c940b3250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:24.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.984+0000 7f9c992d9640 1 -- 172.21.15.105:0/3939786090 shutdown_connections 2024-04-15T15:21:24.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:24.984+0000 7f9c992d9640 1 -- 172.21.15.105:0/3939786090 wait complete. 2024-04-15T15:21:25.000 INFO:tasks.cephfs_test_runner:That a file with ceph.quiesce.block is not quiesced. ... FAIL 2024-04-15T15:21:25.000 INFO:tasks.cephfs_test_runner: 2024-04-15T15:21:25.001 INFO:tasks.cephfs_test_runner:====================================================================== 2024-04-15T15:21:25.001 INFO:tasks.cephfs_test_runner:FAIL: test_quiesce_blocked (tasks.cephfs.test_quiesce.TestQuiesce) 2024-04-15T15:21:25.001 INFO:tasks.cephfs_test_runner:That a file with ceph.quiesce.block is not quiesced. 2024-04-15T15:21:25.001 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-15T15:21:25.001 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-04-15T15:21:25.001 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/qa/tasks/cephfs/test_quiesce.py", line 541, in test_quiesce_blocked 2024-04-15T15:21:25.001 INFO:tasks.cephfs_test_runner: self._verify_quiesce(root=self.subvolume) 2024-04-15T15:21:25.001 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/qa/tasks/cephfs/test_quiesce.py", line 138, in _verify_quiesce 2024-04-15T15:21:25.001 INFO:tasks.cephfs_test_runner: return self._verify_quiesce_wrapped(rank, status, root, root_inode, ops, cache, splitauth) 2024-04-15T15:21:25.001 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/qa/tasks/cephfs/test_quiesce.py", line 272, in _verify_quiesce_wrapped 2024-04-15T15:21:25.001 INFO:tasks.cephfs_test_runner: self.assertFalse(lock_type.startswith("i")) 2024-04-15T15:21:25.001 INFO:tasks.cephfs_test_runner:AssertionError: True is not false 2024-04-15T15:21:25.001 INFO:tasks.cephfs_test_runner: 2024-04-15T15:21:25.001 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-15T15:21:25.001 INFO:tasks.cephfs_test_runner:Ran 1 test in 92.556s 2024-04-15T15:21:25.002 INFO:tasks.cephfs_test_runner: 2024-04-15T15:21:25.002 INFO:tasks.cephfs_test_runner:FAILED (failures=1) 2024-04-15T15:21:25.002 INFO:tasks.cephfs_test_runner: 2024-04-15T15:21:25.002 INFO:tasks.cephfs_test_runner:====================================================================== 2024-04-15T15:21:25.002 INFO:tasks.cephfs_test_runner:FAIL: test_quiesce_blocked (tasks.cephfs.test_quiesce.TestQuiesce) 2024-04-15T15:21:25.002 INFO:tasks.cephfs_test_runner:That a file with ceph.quiesce.block is not quiesced. 2024-04-15T15:21:25.002 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-15T15:21:25.002 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-04-15T15:21:25.002 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/qa/tasks/cephfs/test_quiesce.py", line 541, in test_quiesce_blocked 2024-04-15T15:21:25.002 INFO:tasks.cephfs_test_runner: self._verify_quiesce(root=self.subvolume) 2024-04-15T15:21:25.002 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/qa/tasks/cephfs/test_quiesce.py", line 138, in _verify_quiesce 2024-04-15T15:21:25.002 INFO:tasks.cephfs_test_runner: return self._verify_quiesce_wrapped(rank, status, root, root_inode, ops, cache, splitauth) 2024-04-15T15:21:25.002 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/qa/tasks/cephfs/test_quiesce.py", line 272, in _verify_quiesce_wrapped 2024-04-15T15:21:25.002 INFO:tasks.cephfs_test_runner: self.assertFalse(lock_type.startswith("i")) 2024-04-15T15:21:25.002 INFO:tasks.cephfs_test_runner:AssertionError: True is not false 2024-04-15T15:21:25.003 INFO:tasks.cephfs_test_runner: 2024-04-15T15:21:25.003 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_quiesce_blocked (tasks.cephfs.test_quiesce.TestQuiesce) 2024-04-15T15:21:25.239 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=a5d90336d7714916919c8bcf287d1c5f Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_quiesce_blocked (tasks.cephfs.test_quiesce.TestQuiesce) 2024-04-15T15:21:25.240 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-15T15:21:25.251 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-15T15:21:25.258 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-15T15:21:25.259 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-04-15T15:21:25.259 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:21:25.281 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056369,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1468 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248840k,nr_inodes=812210,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-15T15:21:25.281 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-04-15T15:21:25.282 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:21:25.341 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056369,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1468 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248840k,nr_inodes=812210,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-15T15:21:25.341 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-15T15:21:25.341 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:21:25.361 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056372,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=679 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-15T15:21:25.361 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-15T15:21:25.361 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:21:25.420 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056372,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=679 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-15T15:21:25.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:21:25.420 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2024-04-15T15:21:25.439 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-15T15:21:25.439 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-15T15:21:25.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:21:25.440 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-15T15:21:25.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:21:25.513 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-15T15:21:25.564 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-15T15:21:25.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:21:25.585 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link delete ceph-brx 2024-04-15T15:21:25.627 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-15T15:21:25.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:21:25.706 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list 2024-04-15T15:21:25.728 INFO:teuthology.orchestra.run.smithi154.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-15T15:21:25.728 INFO:teuthology.orchestra.run.smithi154.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-15T15:21:25.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:21:25.729 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-15T15:21:25.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:21:25.801 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-15T15:21:25.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:21:25.873 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link delete ceph-brx 2024-04-15T15:21:26.021 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-15T15:21:26.048 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-15T15:21:26.049 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-15T15:21:26.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.155+0000 7fe1b178d640 1 Processor -- start 2024-04-15T15:21:26.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.155+0000 7fe1b178d640 1 -- start start 2024-04-15T15:21:26.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.156+0000 7fe1b178d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe1ac15eb60 0x7fe1ac0d11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:26.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.156+0000 7fe1b178d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe1ac162d40 0x7fe1ac0d3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:26.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.156+0000 7fe1b178d640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1ac15ef40 con 0x7fe1ac0d1710 2024-04-15T15:21:26.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.156+0000 7fe1b178d640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fe1ac0d1b60 con 0x7fe1ac15eb60 2024-04-15T15:21:26.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.156+0000 7fe1b178d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fe1ac163540 con 0x7fe1ac162d40 2024-04-15T15:21:26.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.156+0000 7fe1aaffd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe1ac15eb60 0x7fe1ac0d11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:26.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.156+0000 7fe1aa7fc640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fe1ac0d1710 0x7fe1ac162630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:45214/0 (socket says 172.21.15.105:45214) 2024-04-15T15:21:26.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.156+0000 7fe1aa7fc640 1 -- 172.21.15.105:0/3713663455 learned_addr learned my addr 172.21.15.105:0/3713663455 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:21:26.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.157+0000 7fe1ab7fe640 1 --2- 172.21.15.105:0/3713663455 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe1ac162d40 0x7fe1ac0d3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:26.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.157+0000 7fe1aaffd640 1 -- 172.21.15.105:0/3713663455 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe1ac162d40 msgr2=0x7fe1ac0d3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:26.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.157+0000 7fe1aaffd640 1 --2- 172.21.15.105:0/3713663455 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe1ac162d40 0x7fe1ac0d3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:26.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.157+0000 7fe1aaffd640 1 -- 172.21.15.105:0/3713663455 >> v1:172.21.15.105:6789/0 conn(0x7fe1ac0d1710 legacy=0x7fe1ac162630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:26.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.157+0000 7fe1aaffd640 1 -- 172.21.15.105:0/3713663455 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1ac163260 con 0x7fe1ac15eb60 2024-04-15T15:21:26.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.157+0000 7fe1a9ffb640 1 -- 172.21.15.105:0/3713663455 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2407288504 0 0) 0x7fe1ac15ef40 con 0x7fe1ac0d1710 2024-04-15T15:21:26.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.157+0000 7fe1aaffd640 1 --2- 172.21.15.105:0/3713663455 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1ac15eb60 0x7fe1ac0d11d0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fe19c009d90 tx=0x7fe19c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6136e4c2b0ef53af server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:26.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.158+0000 7fe1a9ffb640 1 -- 172.21.15.105:0/3713663455 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe19c039070 con 0x7fe1ac15eb60 2024-04-15T15:21:26.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.158+0000 7fe1a9ffb640 1 -- 172.21.15.105:0/3713663455 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe19c02cc50 con 0x7fe1ac15eb60 2024-04-15T15:21:26.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.158+0000 7fe1a9ffb640 1 -- 172.21.15.105:0/3713663455 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe19c034440 con 0x7fe1ac15eb60 2024-04-15T15:21:26.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.158+0000 7fe1b178d640 1 -- 172.21.15.105:0/3713663455 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1ac15eb60 msgr2=0x7fe1ac0d11d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:26.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.158+0000 7fe1b178d640 1 --2- 172.21.15.105:0/3713663455 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1ac15eb60 0x7fe1ac0d11d0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fe19c009d90 tx=0x7fe19c02bea0 comp rx=0 tx=0).stop 2024-04-15T15:21:26.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.158+0000 7fe1b178d640 1 -- 172.21.15.105:0/3713663455 shutdown_connections 2024-04-15T15:21:26.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.158+0000 7fe1b178d640 1 --2- 172.21.15.105:0/3713663455 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe1ac162d40 0x7fe1ac0d3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:26.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.158+0000 7fe1b178d640 1 --2- 172.21.15.105:0/3713663455 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1ac15eb60 0x7fe1ac0d11d0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:26.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.158+0000 7fe1b178d640 1 -- 172.21.15.105:0/3713663455 >> 172.21.15.105:0/3713663455 conn(0x7fe1ac067720 msgr2=0x7fe1ac067b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:26.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.158+0000 7fe1b178d640 1 -- 172.21.15.105:0/3713663455 shutdown_connections 2024-04-15T15:21:26.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.158+0000 7fe1b178d640 1 -- 172.21.15.105:0/3713663455 wait complete. 2024-04-15T15:21:26.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.160+0000 7fe1b178d640 1 Processor -- start 2024-04-15T15:21:26.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.160+0000 7fe1b178d640 1 -- start start 2024-04-15T15:21:26.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.160+0000 7fe1b178d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1ac0d1710 0x7fe1ac1f8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:26.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.160+0000 7fe1b178d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe1ac15eb60 0x7fe1ac1f5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:26.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.160+0000 7fe1b178d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe1ac162d40 0x7fe1ac0b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:26.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.160+0000 7fe1b178d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ac0d2c50 con 0x7fe1ac0d1710 2024-04-15T15:21:26.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.160+0000 7fe1b178d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ac0d2fc0 con 0x7fe1ac162d40 2024-04-15T15:21:26.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.160+0000 7fe1b178d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fe1ac0b78f0 con 0x7fe1ac15eb60 2024-04-15T15:21:26.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.160+0000 7fe1aaffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1ac0d1710 0x7fe1ac1f8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:26.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.160+0000 7fe1aaffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1ac0d1710 0x7fe1ac1f8650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47120/0 (socket says 172.21.15.105:47120) 2024-04-15T15:21:26.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.160+0000 7fe1aaffd640 1 -- 172.21.15.105:0/3995867428 learned_addr learned my addr 172.21.15.105:0/3995867428 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:26.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.160+0000 7fe1aa7fc640 1 --2- 172.21.15.105:0/3995867428 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe1ac15eb60 0x7fe1ac1f5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:26.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.161+0000 7fe1ab7fe640 1 --2- 172.21.15.105:0/3995867428 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe1ac162d40 0x7fe1ac0b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:26.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.161+0000 7fe1aaffd640 1 -- 172.21.15.105:0/3995867428 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe1ac15eb60 msgr2=0x7fe1ac1f5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:26.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.161+0000 7fe1aaffd640 1 --2- 172.21.15.105:0/3995867428 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe1ac15eb60 0x7fe1ac1f5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:26.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.161+0000 7fe1aaffd640 1 -- 172.21.15.105:0/3995867428 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe1ac162d40 msgr2=0x7fe1ac0b5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:26.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.161+0000 7fe1aaffd640 1 --2- 172.21.15.105:0/3995867428 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe1ac162d40 0x7fe1ac0b5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:26.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.161+0000 7fe1aaffd640 1 -- 172.21.15.105:0/3995867428 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1ac0b5540 con 0x7fe1ac0d1710 2024-04-15T15:21:26.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.161+0000 7fe1ab7fe640 1 --2- 172.21.15.105:0/3995867428 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe1ac162d40 0x7fe1ac0b5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:21:26.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.161+0000 7fe1aaffd640 1 --2- 172.21.15.105:0/3995867428 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1ac0d1710 0x7fe1ac1f8650 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fe19c009d90 tx=0x7fe19c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:26.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.161+0000 7fe1937fe640 1 -- 172.21.15.105:0/3995867428 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe19c039050 con 0x7fe1ac0d1710 2024-04-15T15:21:26.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.161+0000 7fe1937fe640 1 -- 172.21.15.105:0/3995867428 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe19c045070 con 0x7fe1ac0d1710 2024-04-15T15:21:26.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.161+0000 7fe1b178d640 1 -- 172.21.15.105:0/3995867428 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1ac0b5ac0 con 0x7fe1ac0d1710 2024-04-15T15:21:26.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.161+0000 7fe1937fe640 1 -- 172.21.15.105:0/3995867428 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe19c03e610 con 0x7fe1ac0d1710 2024-04-15T15:21:26.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.161+0000 7fe1b178d640 1 -- 172.21.15.105:0/3995867428 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1ac1f7b70 con 0x7fe1ac0d1710 2024-04-15T15:21:26.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.162+0000 7fe1b178d640 1 -- 172.21.15.105:0/3995867428 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1ac068a40 con 0x7fe1ac0d1710 2024-04-15T15:21:26.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.163+0000 7fe1937fe640 1 -- 172.21.15.105:0/3995867428 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fe19c03e7b0 con 0x7fe1ac0d1710 2024-04-15T15:21:26.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.163+0000 7fe1937fe640 1 --2- 172.21.15.105:0/3995867428 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fe19806c980 0x7fe19806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:26.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.163+0000 7fe1aa7fc640 1 --2- 172.21.15.105:0/3995867428 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fe19806c980 0x7fe19806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:26.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.163+0000 7fe1937fe640 1 -- 172.21.15.105:0/3995867428 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5029+0+0 (secure 0 0 0) 0x7fe19c0ae3c0 con 0x7fe1ac0d1710 2024-04-15T15:21:26.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.164+0000 7fe1aa7fc640 1 --2- 172.21.15.105:0/3995867428 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fe19806c980 0x7fe19806ee40 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0007f70 tx=0x7fe1a0007f00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:26.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.168+0000 7fe1937fe640 1 -- 172.21.15.105:0/3995867428 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe19c07caa0 con 0x7fe1ac0d1710 2024-04-15T15:21:26.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.339+0000 7fe1b178d640 1 -- 172.21.15.105:0/3995867428 --> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe1ac0c9810 con 0x7fe19806c980 2024-04-15T15:21:26.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.340+0000 7fe1937fe640 1 -- 172.21.15.105:0/3995867428 <== mgr.4107 v2:172.21.15.105:6832/3506549322 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13482 (secure 0 0 0) 0x7fe1ac0c9810 con 0x7fe19806c980 2024-04-15T15:21:26.342 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:21:26.342 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-04-15T15:21:26.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.342+0000 7fe1b178d640 1 -- 172.21.15.105:0/3995867428 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fe19806c980 msgr2=0x7fe19806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:26.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.342+0000 7fe1b178d640 1 --2- 172.21.15.105:0/3995867428 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fe19806c980 0x7fe19806ee40 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0007f70 tx=0x7fe1a0007f00 comp rx=0 tx=0).stop 2024-04-15T15:21:26.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.342+0000 7fe1b178d640 1 -- 172.21.15.105:0/3995867428 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1ac0d1710 msgr2=0x7fe1ac1f8650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:26.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.342+0000 7fe1b178d640 1 --2- 172.21.15.105:0/3995867428 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1ac0d1710 0x7fe1ac1f8650 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fe19c009d90 tx=0x7fe19c004a40 comp rx=0 tx=0).stop 2024-04-15T15:21:26.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.342+0000 7fe1b178d640 1 -- 172.21.15.105:0/3995867428 shutdown_connections 2024-04-15T15:21:26.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.342+0000 7fe1b178d640 1 --2- 172.21.15.105:0/3995867428 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe1ac15eb60 0x7fe1ac1f5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:26.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.342+0000 7fe1b178d640 1 --2- 172.21.15.105:0/3995867428 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fe19806c980 0x7fe19806ee40 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:26.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.342+0000 7fe1b178d640 1 --2- 172.21.15.105:0/3995867428 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe1ac162d40 0x7fe1ac0b5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:26.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.342+0000 7fe1b178d640 1 --2- 172.21.15.105:0/3995867428 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1ac0d1710 0x7fe1ac1f8650 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:26.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.342+0000 7fe1b178d640 1 -- 172.21.15.105:0/3995867428 >> 172.21.15.105:0/3995867428 conn(0x7fe1ac067720 msgr2=0x7fe1ac0c82d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:26.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.342+0000 7fe1b178d640 1 -- 172.21.15.105:0/3995867428 shutdown_connections 2024-04-15T15:21:26.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.342+0000 7fe1b178d640 1 -- 172.21.15.105:0/3995867428 wait complete. 2024-04-15T15:21:26.354 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":97,"stamp":"2024-04-15T15:21:25.757016+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":162,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":5,"kb":754974720,"kb_used":220716,"kb_used_data":4540,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754004,"statfs":{"total":773094113280,"available":772868100096,"internally_reserved":0,"allocated":4648960,"data_stored":2559902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-62722,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-79,"num_read_kb":-130,"num_write":-93,"num_write_kb":-97,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001120"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":204,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-15T15:21:22.138790+0000","last_change":"2024-04-15T15:19:14.878269+0000","last_active":"2024-04-15T15:21:22.138790+0000","last_peered":"2024-04-15T15:21:22.138790+0000","last_clean":"2024-04-15T15:21:22.138790+0000","last_became_active":"2024-04-15T15:19:14.877367+0000","last_became_peered":"2024-04-15T15:19:14.877367+0000","last_unstale":"2024-04-15T15:21:22.138790+0000","last_undegraded":"2024-04-15T15:21:22.138790+0000","last_fullsized":"2024-04-15T15:21:22.138790+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:13.827511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:13.827511+0000","last_clean_scrub_stamp":"2024-04-15T15:19:13.827511+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:32:55.902561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574877,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1019904,"data_stored":760628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574879,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":170260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574878,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":170260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574877,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":170260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574878,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27456,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":442368,"data_stored":175911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574879,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27456,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":442368,"data_stored":175911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574878,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":430080,"data_stored":170393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574878,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343804,"statfs":{"total":96636764160,"available":96608055296,"internally_reserved":0,"allocated":1036288,"data_stored":766279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-15T15:21:26.355 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-15T15:21:26.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.456+0000 7f21b402a640 1 Processor -- start 2024-04-15T15:21:26.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.456+0000 7f21b402a640 1 -- start start 2024-04-15T15:21:26.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.457+0000 7f21b402a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f21ac0d18e0 0x7f21ac162630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:26.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.457+0000 7f21b402a640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f21ac162b70 0x7f21ac0d3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:26.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.457+0000 7f21b402a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21ac15ef40 con 0x7f21ac15eb60 2024-04-15T15:21:26.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.457+0000 7f21b402a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f21ac163540 con 0x7f21ac0d18e0 2024-04-15T15:21:26.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.457+0000 7f21b402a640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f21ac0d2fc0 con 0x7f21ac162b70 2024-04-15T15:21:26.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.457+0000 7f21b15c5640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f21ac0d18e0 0x7f21ac162630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:26.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.457+0000 7f21b1dc6640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f21ac15eb60 0x7f21ac0d11d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:45220/0 (socket says 172.21.15.105:45220) 2024-04-15T15:21:26.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.457+0000 7f21b1dc6640 1 -- 172.21.15.105:0/524713973 learned_addr learned my addr 172.21.15.105:0/524713973 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:21:26.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.458+0000 7f21b25c7640 1 --2- 172.21.15.105:0/524713973 >> v2:172.21.15.154:3300/0 conn(0x7f21ac162b70 0x7f21ac0d3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:26.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.458+0000 7f21b15c5640 1 -- 172.21.15.105:0/524713973 >> v2:172.21.15.154:3300/0 conn(0x7f21ac162b70 msgr2=0x7f21ac0d3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:26.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.458+0000 7f21b15c5640 1 --2- 172.21.15.105:0/524713973 >> v2:172.21.15.154:3300/0 conn(0x7f21ac162b70 0x7f21ac0d3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:26.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.458+0000 7f21b15c5640 1 -- 172.21.15.105:0/524713973 >> v1:172.21.15.105:6789/0 conn(0x7f21ac15eb60 legacy=0x7f21ac0d11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:26.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.458+0000 7f21b0dc4640 1 -- 172.21.15.105:0/524713973 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4210595130 0 0) 0x7f21ac15ef40 con 0x7f21ac15eb60 2024-04-15T15:21:26.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.458+0000 7f21b15c5640 1 -- 172.21.15.105:0/524713973 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21ac163260 con 0x7f21ac0d18e0 2024-04-15T15:21:26.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.458+0000 7f21b15c5640 1 --2- 172.21.15.105:0/524713973 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21ac0d18e0 0x7f21ac162630 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f21ac0d1eb0 tx=0x7f21a002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fac332ae5f78639 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:26.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.459+0000 7f21b0dc4640 1 -- 172.21.15.105:0/524713973 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a0035020 con 0x7f21ac0d18e0 2024-04-15T15:21:26.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.459+0000 7f21b0dc4640 1 -- 172.21.15.105:0/524713973 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21a0031830 con 0x7f21ac0d18e0 2024-04-15T15:21:26.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.459+0000 7f21b0dc4640 1 -- 172.21.15.105:0/524713973 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a0031c00 con 0x7f21ac0d18e0 2024-04-15T15:21:26.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.459+0000 7f21b402a640 1 -- 172.21.15.105:0/524713973 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21ac0d18e0 msgr2=0x7f21ac162630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:26.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.459+0000 7f21b402a640 1 --2- 172.21.15.105:0/524713973 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21ac0d18e0 0x7f21ac162630 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f21ac0d1eb0 tx=0x7f21a002b8d0 comp rx=0 tx=0).stop 2024-04-15T15:21:26.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.459+0000 7f21b402a640 1 -- 172.21.15.105:0/524713973 shutdown_connections 2024-04-15T15:21:26.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.459+0000 7f21b402a640 1 --2- 172.21.15.105:0/524713973 >> v2:172.21.15.154:3300/0 conn(0x7f21ac162b70 0x7f21ac0d3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:26.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.459+0000 7f21b402a640 1 --2- 172.21.15.105:0/524713973 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21ac0d18e0 0x7f21ac162630 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:26.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.459+0000 7f21b402a640 1 -- 172.21.15.105:0/524713973 >> 172.21.15.105:0/524713973 conn(0x7f21ac067720 msgr2=0x7f21ac067b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:26.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.459+0000 7f21b402a640 1 -- 172.21.15.105:0/524713973 shutdown_connections 2024-04-15T15:21:26.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.459+0000 7f21b402a640 1 -- 172.21.15.105:0/524713973 wait complete. 2024-04-15T15:21:26.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.460+0000 7f21b402a640 1 Processor -- start 2024-04-15T15:21:26.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.460+0000 7f21b402a640 1 -- start start 2024-04-15T15:21:26.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.461+0000 7f21b402a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f21ac0d18e0 0x7f21ac15ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:26.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.461+0000 7f21b402a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21ac15eb60 0x7f21ac0b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:26.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.461+0000 7f21b402a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f21ac162b70 0x7f21ac1f4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:26.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.461+0000 7f21b402a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f21ac162f80 con 0x7f21ac15eb60 2024-04-15T15:21:26.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.461+0000 7f21b402a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f21ac0d2c50 con 0x7f21ac0d18e0 2024-04-15T15:21:26.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.461+0000 7f21b402a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f21ac0b78f0 con 0x7f21ac162b70 2024-04-15T15:21:26.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.461+0000 7f21b15c5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21ac15eb60 0x7f21ac0b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:26.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.461+0000 7f21b15c5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21ac15eb60 0x7f21ac0b2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47136/0 (socket says 172.21.15.105:47136) 2024-04-15T15:21:26.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.461+0000 7f21b15c5640 1 -- 172.21.15.105:0/1730940063 learned_addr learned my addr 172.21.15.105:0/1730940063 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:26.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.461+0000 7f21b25c7640 1 --2- 172.21.15.105:0/1730940063 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f21ac162b70 0x7f21ac1f4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:26.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.461+0000 7f21b1dc6640 1 --2- 172.21.15.105:0/1730940063 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f21ac0d18e0 0x7f21ac15ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:26.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.462+0000 7f21b15c5640 1 -- 172.21.15.105:0/1730940063 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f21ac162b70 msgr2=0x7f21ac1f4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:26.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.462+0000 7f21b15c5640 1 --2- 172.21.15.105:0/1730940063 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f21ac162b70 0x7f21ac1f4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:26.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.462+0000 7f21b15c5640 1 -- 172.21.15.105:0/1730940063 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f21ac0d18e0 msgr2=0x7f21ac15ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:26.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.462+0000 7f21b15c5640 1 --2- 172.21.15.105:0/1730940063 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f21ac0d18e0 0x7f21ac15ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:26.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.462+0000 7f21b15c5640 1 -- 172.21.15.105:0/1730940063 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21ac1f52e0 con 0x7f21ac15eb60 2024-04-15T15:21:26.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.462+0000 7f21b1dc6640 1 --2- 172.21.15.105:0/1730940063 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f21ac0d18e0 0x7f21ac15ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:21:26.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.462+0000 7f21b15c5640 1 --2- 172.21.15.105:0/1730940063 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21ac15eb60 0x7f21ac0b2b90 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f21a002bee0 tx=0x7f21a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:26.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.462+0000 7f21aa7fc640 1 -- 172.21.15.105:0/1730940063 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a003ba80 con 0x7f21ac15eb60 2024-04-15T15:21:26.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.462+0000 7f21aa7fc640 1 -- 172.21.15.105:0/1730940063 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21a003bc40 con 0x7f21ac15eb60 2024-04-15T15:21:26.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.462+0000 7f21b402a640 1 -- 172.21.15.105:0/1730940063 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21ac15f520 con 0x7f21ac15eb60 2024-04-15T15:21:26.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.462+0000 7f21aa7fc640 1 -- 172.21.15.105:0/1730940063 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a00453f0 con 0x7f21ac15eb60 2024-04-15T15:21:26.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.462+0000 7f21b402a640 1 -- 172.21.15.105:0/1730940063 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21ac15fae0 con 0x7f21ac15eb60 2024-04-15T15:21:26.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.463+0000 7f21b402a640 1 -- 172.21.15.105:0/1730940063 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21ac1604c0 con 0x7f21ac15eb60 2024-04-15T15:21:26.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.463+0000 7f21aa7fc640 1 -- 172.21.15.105:0/1730940063 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f21a0038020 con 0x7f21ac15eb60 2024-04-15T15:21:26.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.464+0000 7f21aa7fc640 1 --2- 172.21.15.105:0/1730940063 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f219806c980 0x7f219806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:26.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.464+0000 7f21aa7fc640 1 -- 172.21.15.105:0/1730940063 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5029+0+0 (secure 0 0 0) 0x7f21a00ad780 con 0x7f21ac15eb60 2024-04-15T15:21:26.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.464+0000 7f21b1dc6640 1 --2- 172.21.15.105:0/1730940063 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f219806c980 0x7f219806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:26.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.464+0000 7f21b1dc6640 1 --2- 172.21.15.105:0/1730940063 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f219806c980 0x7f219806ee40 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f219c007fd0 tx=0x7f219c005b70 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:26.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.468+0000 7f21aa7fc640 1 -- 172.21.15.105:0/1730940063 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f21ac1f52e0 con 0x7f21ac15eb60 2024-04-15T15:21:26.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.641+0000 7f21b402a640 1 -- 172.21.15.105:0/1730940063 --> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f21ac068a40 con 0x7f219806c980 2024-04-15T15:21:26.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.642+0000 7f21aa7fc640 1 -- 172.21.15.105:0/1730940063 <== mgr.4107 v2:172.21.15.105:6832/3506549322 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13482 (secure 0 0 0) 0x7f21ac068a40 con 0x7f219806c980 2024-04-15T15:21:26.644 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:21:26.644 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-04-15T15:21:26.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.644+0000 7f21b402a640 1 -- 172.21.15.105:0/1730940063 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f219806c980 msgr2=0x7f219806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:26.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.644+0000 7f21b402a640 1 --2- 172.21.15.105:0/1730940063 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f219806c980 0x7f219806ee40 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f219c007fd0 tx=0x7f219c005b70 comp rx=0 tx=0).stop 2024-04-15T15:21:26.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.644+0000 7f21b402a640 1 -- 172.21.15.105:0/1730940063 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21ac15eb60 msgr2=0x7f21ac0b2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:26.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.644+0000 7f21b402a640 1 --2- 172.21.15.105:0/1730940063 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21ac15eb60 0x7f21ac0b2b90 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f21a002bee0 tx=0x7f21a00046c0 comp rx=0 tx=0).stop 2024-04-15T15:21:26.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.644+0000 7f21b402a640 1 -- 172.21.15.105:0/1730940063 shutdown_connections 2024-04-15T15:21:26.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.644+0000 7f21b402a640 1 --2- 172.21.15.105:0/1730940063 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f21ac162b70 0x7f21ac1f4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:26.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.644+0000 7f21b402a640 1 --2- 172.21.15.105:0/1730940063 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f219806c980 0x7f219806ee40 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:26.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.644+0000 7f21b402a640 1 --2- 172.21.15.105:0/1730940063 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21ac15eb60 0x7f21ac0b2b90 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:26.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.644+0000 7f21b402a640 1 --2- 172.21.15.105:0/1730940063 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f21ac0d18e0 0x7f21ac15ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:26.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.644+0000 7f21b402a640 1 -- 172.21.15.105:0/1730940063 >> 172.21.15.105:0/1730940063 conn(0x7f21ac067720 msgr2=0x7f21ac0c82d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:26.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.644+0000 7f21b402a640 1 -- 172.21.15.105:0/1730940063 shutdown_connections 2024-04-15T15:21:26.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.644+0000 7f21b402a640 1 -- 172.21.15.105:0/1730940063 wait complete. 2024-04-15T15:21:26.657 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":97,"stamp":"2024-04-15T15:21:25.757016+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":162,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":5,"kb":754974720,"kb_used":220716,"kb_used_data":4540,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754004,"statfs":{"total":773094113280,"available":772868100096,"internally_reserved":0,"allocated":4648960,"data_stored":2559902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-62722,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-79,"num_read_kb":-130,"num_write":-93,"num_write_kb":-97,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001120"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":204,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-15T15:21:22.138790+0000","last_change":"2024-04-15T15:19:14.878269+0000","last_active":"2024-04-15T15:21:22.138790+0000","last_peered":"2024-04-15T15:21:22.138790+0000","last_clean":"2024-04-15T15:21:22.138790+0000","last_became_active":"2024-04-15T15:19:14.877367+0000","last_became_peered":"2024-04-15T15:19:14.877367+0000","last_unstale":"2024-04-15T15:21:22.138790+0000","last_undegraded":"2024-04-15T15:21:22.138790+0000","last_fullsized":"2024-04-15T15:21:22.138790+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:13.827511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:13.827511+0000","last_clean_scrub_stamp":"2024-04-15T15:19:13.827511+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:32:55.902561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574877,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1019904,"data_stored":760628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574879,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":170260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574878,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":170260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574877,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":170260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574878,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27456,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":442368,"data_stored":175911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574879,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27456,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":442368,"data_stored":175911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574878,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":430080,"data_stored":170393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574878,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343804,"statfs":{"total":96636764160,"available":96608055296,"internally_reserved":0,"allocated":1036288,"data_stored":766279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-15T15:21:26.658 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-15T15:21:26.658 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-15T15:21:26.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.759+0000 7fd4f3d82640 1 Processor -- start 2024-04-15T15:21:26.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.760+0000 7fd4f3d82640 1 -- start start 2024-04-15T15:21:26.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.760+0000 7fd4f3d82640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd4ec15eb60 0x7fd4ec0d11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:26.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.760+0000 7fd4f3d82640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd4ec162d40 0x7fd4ec0d3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:26.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.760+0000 7fd4f3d82640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4ec15ef40 con 0x7fd4ec0d1710 2024-04-15T15:21:26.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.760+0000 7fd4f3d82640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fd4ec0d1b60 con 0x7fd4ec162d40 2024-04-15T15:21:26.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.760+0000 7fd4f3d82640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fd4ec163540 con 0x7fd4ec15eb60 2024-04-15T15:21:26.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.761+0000 7fd4f231f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd4ec162d40 0x7fd4ec0d3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:26.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.761+0000 7fd4f231f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd4ec162d40 0x7fd4ec0d3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47152/0 (socket says 172.21.15.105:47152) 2024-04-15T15:21:26.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.761+0000 7fd4f231f640 1 -- 172.21.15.105:0/4145383882 learned_addr learned my addr 172.21.15.105:0/4145383882 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:26.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.761+0000 7fd4f1b1e640 1 --2- 172.21.15.105:0/4145383882 >> v2:172.21.15.154:3300/0 conn(0x7fd4ec15eb60 0x7fd4ec0d11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:26.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.761+0000 7fd4f231f640 1 -- 172.21.15.105:0/4145383882 >> v2:172.21.15.154:3300/0 conn(0x7fd4ec15eb60 msgr2=0x7fd4ec0d11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:26.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.761+0000 7fd4f231f640 1 --2- 172.21.15.105:0/4145383882 >> v2:172.21.15.154:3300/0 conn(0x7fd4ec15eb60 0x7fd4ec0d11d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:26.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.761+0000 7fd4f231f640 1 -- 172.21.15.105:0/4145383882 >> v1:172.21.15.154:6789/0 conn(0x7fd4ec0d1710 legacy=0x7fd4ec162630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:26.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.761+0000 7fd4f231f640 1 -- 172.21.15.105:0/4145383882 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4ec163260 con 0x7fd4ec162d40 2024-04-15T15:21:26.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.762+0000 7fd4f231f640 1 --2- 172.21.15.105:0/4145383882 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4ec162d40 0x7fd4ec0d3450 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec0d5880 tx=0x7fd4e402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a1247b563faf75c server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:26.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.762+0000 7fd4f0b1c640 1 -- 172.21.15.105:0/4145383882 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e402ca80 con 0x7fd4ec162d40 2024-04-15T15:21:26.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.762+0000 7fd4f0b1c640 1 -- 172.21.15.105:0/4145383882 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4e402cc20 con 0x7fd4ec162d40 2024-04-15T15:21:26.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.762+0000 7fd4f0b1c640 1 -- 172.21.15.105:0/4145383882 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e40343f0 con 0x7fd4ec162d40 2024-04-15T15:21:26.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.762+0000 7fd4f3d82640 1 -- 172.21.15.105:0/4145383882 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4ec162d40 msgr2=0x7fd4ec0d3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:26.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.762+0000 7fd4f3d82640 1 --2- 172.21.15.105:0/4145383882 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4ec162d40 0x7fd4ec0d3450 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec0d5880 tx=0x7fd4e402bee0 comp rx=0 tx=0).stop 2024-04-15T15:21:26.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.762+0000 7fd4f3d82640 1 -- 172.21.15.105:0/4145383882 shutdown_connections 2024-04-15T15:21:26.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.762+0000 7fd4f3d82640 1 --2- 172.21.15.105:0/4145383882 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4ec162d40 0x7fd4ec0d3450 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:26.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.762+0000 7fd4f3d82640 1 --2- 172.21.15.105:0/4145383882 >> v2:172.21.15.154:3300/0 conn(0x7fd4ec15eb60 0x7fd4ec0d11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:26.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.762+0000 7fd4f3d82640 1 -- 172.21.15.105:0/4145383882 >> 172.21.15.105:0/4145383882 conn(0x7fd4ec067720 msgr2=0x7fd4ec067b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:26.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.762+0000 7fd4f3d82640 1 -- 172.21.15.105:0/4145383882 shutdown_connections 2024-04-15T15:21:26.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.762+0000 7fd4f3d82640 1 -- 172.21.15.105:0/4145383882 wait complete. 2024-04-15T15:21:26.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.763+0000 7fd4f3d82640 1 Processor -- start 2024-04-15T15:21:26.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.764+0000 7fd4f3d82640 1 -- start start 2024-04-15T15:21:26.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.764+0000 7fd4f3d82640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4ec0d1710 0x7fd4ec159f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:26.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.764+0000 7fd4f3d82640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd4ec15eb60 0x7fd4ec158e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:26.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.764+0000 7fd4f3d82640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd4ec162d40 0x7fd4ec1e9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:26.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.764+0000 7fd4f3d82640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ec0d2c50 con 0x7fd4ec0d1710 2024-04-15T15:21:26.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.764+0000 7fd4f3d82640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ec0d2fc0 con 0x7fd4ec162d40 2024-04-15T15:21:26.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.764+0000 7fd4f3d82640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fd4ec0b78f0 con 0x7fd4ec15eb60 2024-04-15T15:21:26.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.764+0000 7fd4f1b1e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4ec0d1710 0x7fd4ec159f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:26.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.764+0000 7fd4f1b1e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4ec0d1710 0x7fd4ec159f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47162/0 (socket says 172.21.15.105:47162) 2024-04-15T15:21:26.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.764+0000 7fd4f1b1e640 1 -- 172.21.15.105:0/1103573894 learned_addr learned my addr 172.21.15.105:0/1103573894 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:26.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.765+0000 7fd4f231f640 1 --2- 172.21.15.105:0/1103573894 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd4ec162d40 0x7fd4ec1e9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:26.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.765+0000 7fd4f131d640 1 --2- 172.21.15.105:0/1103573894 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd4ec15eb60 0x7fd4ec158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:26.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.765+0000 7fd4f1b1e640 1 -- 172.21.15.105:0/1103573894 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd4ec15eb60 msgr2=0x7fd4ec158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:26.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.765+0000 7fd4f1b1e640 1 --2- 172.21.15.105:0/1103573894 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd4ec15eb60 0x7fd4ec158e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:26.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.765+0000 7fd4f1b1e640 1 -- 172.21.15.105:0/1103573894 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd4ec162d40 msgr2=0x7fd4ec1e9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:26.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.765+0000 7fd4f1b1e640 1 --2- 172.21.15.105:0/1103573894 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd4ec162d40 0x7fd4ec1e9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:26.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.765+0000 7fd4f1b1e640 1 -- 172.21.15.105:0/1103573894 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4ec159380 con 0x7fd4ec0d1710 2024-04-15T15:21:26.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.765+0000 7fd4f231f640 1 --2- 172.21.15.105:0/1103573894 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd4ec162d40 0x7fd4ec1e9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:21:26.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.765+0000 7fd4f1b1e640 1 --2- 172.21.15.105:0/1103573894 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4ec0d1710 0x7fd4ec159f90 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fd4e800c7f0 tx=0x7fd4e800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:26.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.765+0000 7fd4e27fc640 1 -- 172.21.15.105:0/1103573894 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e8015070 con 0x7fd4ec0d1710 2024-04-15T15:21:26.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.765+0000 7fd4e27fc640 1 -- 172.21.15.105:0/1103573894 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4e8004220 con 0x7fd4ec0d1710 2024-04-15T15:21:26.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.765+0000 7fd4f3d82640 1 -- 172.21.15.105:0/1103573894 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4ec15a610 con 0x7fd4ec0d1710 2024-04-15T15:21:26.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.765+0000 7fd4e27fc640 1 -- 172.21.15.105:0/1103573894 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e8007620 con 0x7fd4ec0d1710 2024-04-15T15:21:26.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.765+0000 7fd4f3d82640 1 -- 172.21.15.105:0/1103573894 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4ec15abd0 con 0x7fd4ec0d1710 2024-04-15T15:21:26.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.766+0000 7fd4f3d82640 1 -- 172.21.15.105:0/1103573894 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4ec068a40 con 0x7fd4ec0d1710 2024-04-15T15:21:26.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.767+0000 7fd4e27fc640 1 -- 172.21.15.105:0/1103573894 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fd4e80079d0 con 0x7fd4ec0d1710 2024-04-15T15:21:26.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.767+0000 7fd4e27fc640 1 --2- 172.21.15.105:0/1103573894 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fd4d006c980 0x7fd4d006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:26.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.767+0000 7fd4e27fc640 1 -- 172.21.15.105:0/1103573894 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5029+0+0 (secure 0 0 0) 0x7fd4e808bde0 con 0x7fd4ec0d1710 2024-04-15T15:21:26.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.767+0000 7fd4f131d640 1 --2- 172.21.15.105:0/1103573894 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fd4d006c980 0x7fd4d006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:26.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.767+0000 7fd4f131d640 1 --2- 172.21.15.105:0/1103573894 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fd4d006c980 0x7fd4d006ee40 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec0b7300 tx=0x7fd4dc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:26.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.771+0000 7fd4e27fc640 1 -- 172.21.15.105:0/1103573894 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd4e805a3b0 con 0x7fd4ec0d1710 2024-04-15T15:21:26.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.941+0000 7fd4f3d82640 1 -- 172.21.15.105:0/1103573894 --> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd4ec0d4cb0 con 0x7fd4d006c980 2024-04-15T15:21:26.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.942+0000 7fd4e27fc640 1 -- 172.21.15.105:0/1103573894 <== mgr.4107 v2:172.21.15.105:6832/3506549322 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13482 (secure 0 0 0) 0x7fd4ec0d4cb0 con 0x7fd4d006c980 2024-04-15T15:21:26.944 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:21:26.944 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-04-15T15:21:26.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.944+0000 7fd4f3d82640 1 -- 172.21.15.105:0/1103573894 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fd4d006c980 msgr2=0x7fd4d006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:26.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.944+0000 7fd4f3d82640 1 --2- 172.21.15.105:0/1103573894 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fd4d006c980 0x7fd4d006ee40 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec0b7300 tx=0x7fd4dc008040 comp rx=0 tx=0).stop 2024-04-15T15:21:26.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.944+0000 7fd4f3d82640 1 -- 172.21.15.105:0/1103573894 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4ec0d1710 msgr2=0x7fd4ec159f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:26.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.944+0000 7fd4f3d82640 1 --2- 172.21.15.105:0/1103573894 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4ec0d1710 0x7fd4ec159f90 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fd4e800c7f0 tx=0x7fd4e800ccb0 comp rx=0 tx=0).stop 2024-04-15T15:21:26.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.944+0000 7fd4f3d82640 1 -- 172.21.15.105:0/1103573894 shutdown_connections 2024-04-15T15:21:26.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.944+0000 7fd4f3d82640 1 --2- 172.21.15.105:0/1103573894 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd4ec15eb60 0x7fd4ec158e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:26.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.944+0000 7fd4f3d82640 1 --2- 172.21.15.105:0/1103573894 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fd4d006c980 0x7fd4d006ee40 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:26.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.944+0000 7fd4f3d82640 1 --2- 172.21.15.105:0/1103573894 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd4ec162d40 0x7fd4ec1e9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:26.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.944+0000 7fd4f3d82640 1 --2- 172.21.15.105:0/1103573894 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4ec0d1710 0x7fd4ec159f90 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:26.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.944+0000 7fd4f3d82640 1 -- 172.21.15.105:0/1103573894 >> 172.21.15.105:0/1103573894 conn(0x7fd4ec067720 msgr2=0x7fd4ec15c820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:26.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.945+0000 7fd4f3d82640 1 -- 172.21.15.105:0/1103573894 shutdown_connections 2024-04-15T15:21:26.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:26.945+0000 7fd4f3d82640 1 -- 172.21.15.105:0/1103573894 wait complete. 2024-04-15T15:21:26.957 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":97,"stamp":"2024-04-15T15:21:25.757016+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":162,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":5,"kb":754974720,"kb_used":220716,"kb_used_data":4540,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754004,"statfs":{"total":773094113280,"available":772868100096,"internally_reserved":0,"allocated":4648960,"data_stored":2559902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-62722,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-79,"num_read_kb":-130,"num_write":-93,"num_write_kb":-97,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001120"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":204,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-15T15:21:22.138790+0000","last_change":"2024-04-15T15:19:14.878269+0000","last_active":"2024-04-15T15:21:22.138790+0000","last_peered":"2024-04-15T15:21:22.138790+0000","last_clean":"2024-04-15T15:21:22.138790+0000","last_became_active":"2024-04-15T15:19:14.877367+0000","last_became_peered":"2024-04-15T15:19:14.877367+0000","last_unstale":"2024-04-15T15:21:22.138790+0000","last_undegraded":"2024-04-15T15:21:22.138790+0000","last_fullsized":"2024-04-15T15:21:22.138790+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:13.827511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:13.827511+0000","last_clean_scrub_stamp":"2024-04-15T15:19:13.827511+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:32:55.902561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574877,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1019904,"data_stored":760628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574879,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":170260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574878,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":170260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574877,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":170260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574878,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27456,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":442368,"data_stored":175911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574879,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27456,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":442368,"data_stored":175911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574878,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":430080,"data_stored":170393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574878,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343804,"statfs":{"total":96636764160,"available":96608055296,"internally_reserved":0,"allocated":1036288,"data_stored":766279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-15T15:21:26.958 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:21:27.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.060+0000 7f6aa6ed3640 1 Processor -- start 2024-04-15T15:21:27.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.060+0000 7f6aa6ed3640 1 -- start start 2024-04-15T15:21:27.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.061+0000 7f6aa6ed3640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6aa00b55b0 0x7f6aa00aeb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:27.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.061+0000 7f6aa6ed3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6aa006fa00 0x7f6aa0071e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:27.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.061+0000 7f6aa6ed3640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6aa006fe60 0x7f6aa00723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:27.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.061+0000 7f6aa6ed3640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f6aa00af070 con 0x7f6aa00b55b0 2024-04-15T15:21:27.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.061+0000 7f6aa6ed3640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f6aa0162820 con 0x7f6aa006fe60 2024-04-15T15:21:27.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.061+0000 7f6aa6ed3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f6aa0162b00 con 0x7f6aa006fa00 2024-04-15T15:21:27.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.061+0000 7f6aa4c6f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6aa00b55b0 0x7f6aa00aeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:27.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.061+0000 7f6aa4c6f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6aa00b55b0 0x7f6aa00aeb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47178/0 (socket says 172.21.15.105:47178) 2024-04-15T15:21:27.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.061+0000 7f6aa4c6f640 1 -- 172.21.15.105:0/725748 learned_addr learned my addr 172.21.15.105:0/725748 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:27.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.061+0000 7f6aa5470640 1 --2- 172.21.15.105:0/725748 >> v2:172.21.15.154:3300/0 conn(0x7f6aa006fe60 0x7f6aa00723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:27.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.061+0000 7f6a97fff640 1 --2- 172.21.15.105:0/725748 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6aa006fa00 0x7f6aa0071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:27.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.061+0000 7f6aa4c6f640 1 -- 172.21.15.105:0/725748 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6aa006fa00 msgr2=0x7f6aa0071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:27.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.061+0000 7f6aa4c6f640 1 --2- 172.21.15.105:0/725748 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6aa006fa00 0x7f6aa0071e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:27.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.061+0000 7f6aa4c6f640 1 -- 172.21.15.105:0/725748 >> v2:172.21.15.154:3300/0 conn(0x7f6aa006fe60 msgr2=0x7f6aa00723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:27.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.061+0000 7f6aa4c6f640 1 --2- 172.21.15.105:0/725748 >> v2:172.21.15.154:3300/0 conn(0x7f6aa006fe60 0x7f6aa00723a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:27.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.061+0000 7f6aa4c6f640 1 -- 172.21.15.105:0/725748 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aa0162de0 con 0x7f6aa00b55b0 2024-04-15T15:21:27.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.061+0000 7f6aa5470640 1 --2- 172.21.15.105:0/725748 >> v2:172.21.15.154:3300/0 conn(0x7f6aa006fe60 0x7f6aa00723a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:21:27.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.062+0000 7f6aa4c6f640 1 --2- 172.21.15.105:0/725748 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6aa00b55b0 0x7f6aa00aeb30 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0162450 tx=0x7f6a9c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=727a2b5f9b8425c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:27.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.062+0000 7f6a977fe640 1 -- 172.21.15.105:0/725748 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c02e070 con 0x7f6aa00b55b0 2024-04-15T15:21:27.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.062+0000 7f6a977fe640 1 -- 172.21.15.105:0/725748 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a9c037070 con 0x7f6aa00b55b0 2024-04-15T15:21:27.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.062+0000 7f6a977fe640 1 -- 172.21.15.105:0/725748 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c031a00 con 0x7f6aa00b55b0 2024-04-15T15:21:27.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.063+0000 7f6aa6ed3640 1 -- 172.21.15.105:0/725748 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6aa00b55b0 msgr2=0x7f6aa00aeb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:27.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.063+0000 7f6aa6ed3640 1 --2- 172.21.15.105:0/725748 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6aa00b55b0 0x7f6aa00aeb30 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0162450 tx=0x7f6a9c02b880 comp rx=0 tx=0).stop 2024-04-15T15:21:27.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.063+0000 7f6aa6ed3640 1 -- 172.21.15.105:0/725748 shutdown_connections 2024-04-15T15:21:27.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.063+0000 7f6aa6ed3640 1 --2- 172.21.15.105:0/725748 >> v2:172.21.15.154:3300/0 conn(0x7f6aa006fe60 0x7f6aa00723a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:27.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.063+0000 7f6aa6ed3640 1 --2- 172.21.15.105:0/725748 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6aa006fa00 0x7f6aa0071e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:27.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.063+0000 7f6aa6ed3640 1 --2- 172.21.15.105:0/725748 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6aa00b55b0 0x7f6aa00aeb30 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:27.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.063+0000 7f6aa6ed3640 1 -- 172.21.15.105:0/725748 >> 172.21.15.105:0/725748 conn(0x7f6aa0067d50 msgr2=0x7f6aa006b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:27.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.063+0000 7f6aa6ed3640 1 -- 172.21.15.105:0/725748 shutdown_connections 2024-04-15T15:21:27.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.063+0000 7f6aa6ed3640 1 -- 172.21.15.105:0/725748 wait complete. 2024-04-15T15:21:27.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.064+0000 7f6aa6ed3640 1 Processor -- start 2024-04-15T15:21:27.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.064+0000 7f6aa6ed3640 1 -- start start 2024-04-15T15:21:27.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.065+0000 7f6aa6ed3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6aa006fa00 0x7f6aa01433b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:27.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.065+0000 7f6aa6ed3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6aa006fe60 0x7f6aa013c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:27.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.065+0000 7f6aa6ed3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6aa00b55b0 0x7f6aa0145170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:27.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.065+0000 7f6aa6ed3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa01633a0 con 0x7f6aa006fe60 2024-04-15T15:21:27.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.065+0000 7f6aa6ed3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa00b5990 con 0x7f6aa00b55b0 2024-04-15T15:21:27.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.065+0000 7f6aa6ed3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f6aa01624f0 con 0x7f6aa006fa00 2024-04-15T15:21:27.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.065+0000 7f6a97fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6aa006fe60 0x7f6aa013c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:27.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.065+0000 7f6a97fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6aa006fe60 0x7f6aa013c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47194/0 (socket says 172.21.15.105:47194) 2024-04-15T15:21:27.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.065+0000 7f6a97fff640 1 -- 172.21.15.105:0/1063689843 learned_addr learned my addr 172.21.15.105:0/1063689843 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:27.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.065+0000 7f6aa4c6f640 1 --2- 172.21.15.105:0/1063689843 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6aa006fa00 0x7f6aa01433b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:27.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.065+0000 7f6aa5470640 1 --2- 172.21.15.105:0/1063689843 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6aa00b55b0 0x7f6aa0145170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:27.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.065+0000 7f6a97fff640 1 -- 172.21.15.105:0/1063689843 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6aa006fa00 msgr2=0x7f6aa01433b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:27.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.065+0000 7f6a97fff640 1 --2- 172.21.15.105:0/1063689843 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6aa006fa00 0x7f6aa01433b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:27.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.065+0000 7f6a97fff640 1 -- 172.21.15.105:0/1063689843 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6aa00b55b0 msgr2=0x7f6aa0145170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:27.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.065+0000 7f6a97fff640 1 --2- 172.21.15.105:0/1063689843 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6aa00b55b0 0x7f6aa0145170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:27.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.065+0000 7f6a97fff640 1 -- 172.21.15.105:0/1063689843 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aa013e660 con 0x7f6aa006fe60 2024-04-15T15:21:27.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.065+0000 7f6aa4c6f640 1 --2- 172.21.15.105:0/1063689843 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6aa006fa00 0x7f6aa01433b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:21:27.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.065+0000 7f6a97fff640 1 --2- 172.21.15.105:0/1063689843 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6aa006fe60 0x7f6aa013c3f0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f6a9000d800 tx=0x7f6a9000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:27.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.065+0000 7f6a957fa640 1 -- 172.21.15.105:0/1063689843 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9000b6c0 con 0x7f6aa006fe60 2024-04-15T15:21:27.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.065+0000 7f6a957fa640 1 -- 172.21.15.105:0/1063689843 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a90019070 con 0x7f6aa006fe60 2024-04-15T15:21:27.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.065+0000 7f6a957fa640 1 -- 172.21.15.105:0/1063689843 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9000ba80 con 0x7f6aa006fe60 2024-04-15T15:21:27.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.065+0000 7f6aa6ed3640 1 -- 172.21.15.105:0/1063689843 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6aa01428d0 con 0x7f6aa006fe60 2024-04-15T15:21:27.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.065+0000 7f6aa6ed3640 1 -- 172.21.15.105:0/1063689843 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6aa0142e90 con 0x7f6aa006fe60 2024-04-15T15:21:27.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.066+0000 7f6aa6ed3640 1 -- 172.21.15.105:0/1063689843 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6aa0068a40 con 0x7f6aa006fe60 2024-04-15T15:21:27.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.067+0000 7f6a957fa640 1 -- 172.21.15.105:0/1063689843 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f6a9000bc20 con 0x7f6aa006fe60 2024-04-15T15:21:27.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.067+0000 7f6a957fa640 1 --2- 172.21.15.105:0/1063689843 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f6a8c06c980 0x7f6a8c06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:27.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.068+0000 7f6aa4c6f640 1 --2- 172.21.15.105:0/1063689843 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f6a8c06c980 0x7f6a8c06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:27.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.068+0000 7f6a957fa640 1 -- 172.21.15.105:0/1063689843 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5029+0+0 (secure 0 0 0) 0x7f6a9008bd50 con 0x7f6aa006fe60 2024-04-15T15:21:27.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.068+0000 7f6aa4c6f640 1 --2- 172.21.15.105:0/1063689843 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f6a8c06c980 0x7f6a8c06ee40 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c036000 tx=0x7f6a9c03d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:27.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.072+0000 7f6a957fa640 1 -- 172.21.15.105:0/1063689843 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6a9005a4c0 con 0x7f6aa006fe60 2024-04-15T15:21:27.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.246+0000 7f6aa6ed3640 1 -- 172.21.15.105:0/1063689843 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6aa006f400 con 0x7f6aa006fe60 2024-04-15T15:21:27.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.247+0000 7f6a957fa640 1 -- 172.21.15.105:0/1063689843 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+13200 (secure 0 0 0) 0x7f6a90059dc0 con 0x7f6aa006fe60 2024-04-15T15:21:27.249 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:21:27.249 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":38,"fsid":"09e4d2a4-a324-4793-af77-644b25e520e1","created":"2024-04-15T15:19:01.524543+0000","modified":"2024-04-15T15:21:22.971632+0000","last_up_change":"2024-04-15T15:19:12.825815+0000","last_in_change":"2024-04-15T15:19:03.788126+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:19:13.748585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6f6292b-2d7e-4aa0-9d1c-6ddd0f97287c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6801","nonce":2873040054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6803","nonce":2873040054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6807","nonce":2873040054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2873040054},{"type":"v1","addr":"172.21.15.105:6805","nonce":2873040054}]},"public_addr":"172.21.15.105:6801/2873040054","cluster_addr":"172.21.15.105:6803/2873040054","heartbeat_back_addr":"172.21.15.105:6807/2873040054","heartbeat_front_addr":"172.21.15.105:6805/2873040054","state":["exists","up"]},{"osd":1,"uuid":"8900966a-b393-41e3-8f93-e5a43bd23622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6818","nonce":2494994479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6821","nonce":2494994479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6828","nonce":2494994479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6823","nonce":2494994479},{"type":"v1","addr":"172.21.15.105:6825","nonce":2494994479}]},"public_addr":"172.21.15.105:6818/2494994479","cluster_addr":"172.21.15.105:6821/2494994479","heartbeat_back_addr":"172.21.15.105:6828/2494994479","heartbeat_front_addr":"172.21.15.105:6825/2494994479","state":["exists","up"]},{"osd":2,"uuid":"81a5dc7a-9325-4ff5-941d-a2d5fba21f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6809","nonce":3798408699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6811","nonce":3798408699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6815","nonce":3798408699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3798408699},{"type":"v1","addr":"172.21.15.105:6813","nonce":3798408699}]},"public_addr":"172.21.15.105:6809/3798408699","cluster_addr":"172.21.15.105:6811/3798408699","heartbeat_back_addr":"172.21.15.105:6815/3798408699","heartbeat_front_addr":"172.21.15.105:6813/3798408699","state":["exists","up"]},{"osd":3,"uuid":"df766ed2-5c71-4805-96f4-3d4970edc960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6819","nonce":1920720186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6824","nonce":1920720186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6831","nonce":1920720186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6827","nonce":1920720186},{"type":"v1","addr":"172.21.15.105:6829","nonce":1920720186}]},"public_addr":"172.21.15.105:6819/1920720186","cluster_addr":"172.21.15.105:6824/1920720186","heartbeat_back_addr":"172.21.15.105:6831/1920720186","heartbeat_front_addr":"172.21.15.105:6829/1920720186","state":["exists","up"]},{"osd":4,"uuid":"e1039e03-28b8-49df-b951-c91930b93c9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6810","nonce":3516494456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6813","nonce":3516494456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6820","nonce":3516494456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":3516494456},{"type":"v1","addr":"172.21.15.154:6817","nonce":3516494456}]},"public_addr":"172.21.15.154:6810/3516494456","cluster_addr":"172.21.15.154:6813/3516494456","heartbeat_back_addr":"172.21.15.154:6820/3516494456","heartbeat_front_addr":"172.21.15.154:6817/3516494456","state":["exists","up"]},{"osd":5,"uuid":"a9f5eaf1-66ff-405a-a9e4-ec3340812a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6801","nonce":540367751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6803","nonce":540367751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6807","nonce":540367751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":540367751},{"type":"v1","addr":"172.21.15.154:6805","nonce":540367751}]},"public_addr":"172.21.15.154:6801/540367751","cluster_addr":"172.21.15.154:6803/540367751","heartbeat_back_addr":"172.21.15.154:6807/540367751","heartbeat_front_addr":"172.21.15.154:6805/540367751","state":["exists","up"]},{"osd":6,"uuid":"267d02ea-675f-428f-aae8-e0375e4d4feb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6825","nonce":1550829005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6827","nonce":1550829005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6831","nonce":1550829005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1550829005},{"type":"v1","addr":"172.21.15.154:6829","nonce":1550829005}]},"public_addr":"172.21.15.154:6825/1550829005","cluster_addr":"172.21.15.154:6827/1550829005","heartbeat_back_addr":"172.21.15.154:6831/1550829005","heartbeat_front_addr":"172.21.15.154:6829/1550829005","state":["exists","up"]},{"osd":7,"uuid":"759ac6cc-896a-423c-885f-9fe90d8dcb14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6809","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6811","nonce":3920089903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6816","nonce":3920089903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6823","nonce":3920089903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6819","nonce":3920089903},{"type":"v1","addr":"172.21.15.154:6821","nonce":3920089903}]},"public_addr":"172.21.15.154:6811/3920089903","cluster_addr":"172.21.15.154:6816/3920089903","heartbeat_back_addr":"172.21.15.154:6823/3920089903","heartbeat_front_addr":"172.21.15.154:6821/3920089903","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.949509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.984072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.907278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.947391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.975447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.998896+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:09.028739+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:19:08.894119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.105:6837/4212477415":"2024-04-16T15:21:19.966952+0000","172.21.15.105:6837/958066041":"2024-04-16T15:21:03.660841+0000","172.21.15.105:6835/4212477415":"2024-04-16T15:21:19.966952+0000","172.21.15.105:6835/958066041":"2024-04-16T15:21:03.660841+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:21:27.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.249+0000 7f6aa6ed3640 1 -- 172.21.15.105:0/1063689843 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f6a8c06c980 msgr2=0x7f6a8c06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:27.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.249+0000 7f6aa6ed3640 1 --2- 172.21.15.105:0/1063689843 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f6a8c06c980 0x7f6a8c06ee40 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c036000 tx=0x7f6a9c03d040 comp rx=0 tx=0).stop 2024-04-15T15:21:27.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.249+0000 7f6aa6ed3640 1 -- 172.21.15.105:0/1063689843 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6aa006fe60 msgr2=0x7f6aa013c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:27.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.249+0000 7f6aa6ed3640 1 --2- 172.21.15.105:0/1063689843 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6aa006fe60 0x7f6aa013c3f0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f6a9000d800 tx=0x7f6a9000dcc0 comp rx=0 tx=0).stop 2024-04-15T15:21:27.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.249+0000 7f6aa6ed3640 1 -- 172.21.15.105:0/1063689843 shutdown_connections 2024-04-15T15:21:27.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.249+0000 7f6aa6ed3640 1 --2- 172.21.15.105:0/1063689843 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f6a8c06c980 0x7f6a8c06ee40 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:27.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.249+0000 7f6aa6ed3640 1 --2- 172.21.15.105:0/1063689843 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6aa00b55b0 0x7f6aa0145170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:27.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.249+0000 7f6aa6ed3640 1 --2- 172.21.15.105:0/1063689843 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6aa006fe60 0x7f6aa013c3f0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:27.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.249+0000 7f6aa6ed3640 1 --2- 172.21.15.105:0/1063689843 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6aa006fa00 0x7f6aa01433b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:27.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.249+0000 7f6aa6ed3640 1 -- 172.21.15.105:0/1063689843 >> 172.21.15.105:0/1063689843 conn(0x7f6aa0067d50 msgr2=0x7f6aa006b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:27.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.249+0000 7f6aa6ed3640 1 -- 172.21.15.105:0/1063689843 shutdown_connections 2024-04-15T15:21:27.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:27.249+0000 7f6aa6ed3640 1 -- 172.21.15.105:0/1063689843 wait complete. 2024-04-15T15:21:28.262 INFO:tasks.ceph:Scrubbing osd.0 2024-04-15T15:21:28.263 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-15T15:21:28.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.370+0000 7f91ecab2640 1 Processor -- start 2024-04-15T15:21:28.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.370+0000 7f91ecab2640 1 -- start start 2024-04-15T15:21:28.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.371+0000 7f91ecab2640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f91e813ecd0 0x7f91e806efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:28.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.371+0000 7f91ecab2640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91e815f280 con 0x7f91e813e6f0 2024-04-15T15:21:28.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.371+0000 7f91ecab2640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91e815f560 con 0x7f91e806f500 2024-04-15T15:21:28.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.371+0000 7f91ecab2640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f91e815f840 con 0x7f91e813ecd0 2024-04-15T15:21:28.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.371+0000 7f91e659c640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f91e813e6f0 0x7f91e80cfb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:45226/0 (socket says 172.21.15.105:45226) 2024-04-15T15:21:28.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.371+0000 7f91e659c640 1 -- 172.21.15.105:0/2624697775 learned_addr learned my addr 172.21.15.105:0/2624697775 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:21:28.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.371+0000 7f91dffff640 1 --2- 172.21.15.105:0/2624697775 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f91e813ecd0 0x7f91e806efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:28.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.371+0000 7f91e5d9b640 1 -- 172.21.15.105:0/2624697775 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2626529305 0 0) 0x7f91e815f280 con 0x7f91e813e6f0 2024-04-15T15:21:28.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.371+0000 7f91e5d9b640 1 -- 172.21.15.105:0/2624697775 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f91d4003520 con 0x7f91e813e6f0 2024-04-15T15:21:28.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.371+0000 7f91e5d9b640 1 -- 172.21.15.105:0/2624697775 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1830457393 0 0) 0x7f91d4003520 con 0x7f91e813e6f0 2024-04-15T15:21:28.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.371+0000 7f91e5d9b640 1 -- 172.21.15.105:0/2624697775 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f91e813ecd0 msgr2=0x7f91e806efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:28.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.371+0000 7f91e5d9b640 1 --2- 172.21.15.105:0/2624697775 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f91e813ecd0 0x7f91e806efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:28.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.371+0000 7f91e5d9b640 1 -- 172.21.15.105:0/2624697775 >> v1:172.21.15.154:6789/0 conn(0x7f91e806f500 legacy=0x7f91e815bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:28.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.371+0000 7f91e5d9b640 1 -- 172.21.15.105:0/2624697775 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91e815fb20 con 0x7f91e813e6f0 2024-04-15T15:21:28.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.371+0000 7f91e5d9b640 1 -- 172.21.15.105:0/2624697775 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7f91d8001670 con 0x7f91e813e6f0 2024-04-15T15:21:28.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.371+0000 7f91e5d9b640 1 -- 172.21.15.105:0/2624697775 >> v1:172.21.15.105:6789/0 conn(0x7f91e813e6f0 legacy=0x7f91e80cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:28.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.371+0000 7f91e5d9b640 1 --2- 172.21.15.105:0/2624697775 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f91e813ecd0 0x7f91d40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:28.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.372+0000 7f91e5d9b640 1 --2- 172.21.15.105:0/2624697775 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91d4003f60 0x7f91d4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:28.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.372+0000 7f91e5d9b640 1 --2- 172.21.15.105:0/2624697775 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91d4004d40 0x7f91d40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:28.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.372+0000 7f91e5d9b640 1 -- 172.21.15.105:0/2624697775 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f91e815f840 con 0x7f91d4003f60 2024-04-15T15:21:28.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.372+0000 7f91e5d9b640 1 -- 172.21.15.105:0/2624697775 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f91d400bb50 con 0x7f91d4004d40 2024-04-15T15:21:28.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.372+0000 7f91e5d9b640 1 -- 172.21.15.105:0/2624697775 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f91d400bd50 con 0x7f91e813ecd0 2024-04-15T15:21:28.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.372+0000 7f91dffff640 1 --2- 172.21.15.105:0/2624697775 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91d4003f60 0x7f91d4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:28.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.372+0000 7f91e6d9d640 1 --2- 172.21.15.105:0/2624697775 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91d4004d40 0x7f91d40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:28.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.372+0000 7f91e659c640 1 --2- 172.21.15.105:0/2624697775 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f91e813ecd0 0x7f91d40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:28.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.372+0000 7f91dffff640 1 -- 172.21.15.105:0/2624697775 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f91e813ecd0 msgr2=0x7f91d40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:28.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.372+0000 7f91dffff640 1 --2- 172.21.15.105:0/2624697775 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f91e813ecd0 0x7f91d40039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:28.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.372+0000 7f91dffff640 1 -- 172.21.15.105:0/2624697775 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91d4004d40 msgr2=0x7f91d40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:28.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.372+0000 7f91dffff640 1 --2- 172.21.15.105:0/2624697775 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91d4004d40 0x7f91d40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:28.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.372+0000 7f91dffff640 1 -- 172.21.15.105:0/2624697775 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91d4003520 con 0x7f91d4003f60 2024-04-15T15:21:28.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.373+0000 7f91dffff640 1 --2- 172.21.15.105:0/2624697775 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91d4003f60 0x7f91d4004800 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f91d0009ac0 tx=0x7f91d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f564e1610e8196e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:28.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.373+0000 7f91e5d9b640 1 -- 172.21.15.105:0/2624697775 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d0039070 con 0x7f91d4003f60 2024-04-15T15:21:28.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.373+0000 7f91e5d9b640 1 -- 172.21.15.105:0/2624697775 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91d002c6d0 con 0x7f91d4003f60 2024-04-15T15:21:28.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.373+0000 7f91e5d9b640 1 -- 172.21.15.105:0/2624697775 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d0009690 con 0x7f91d4003f60 2024-04-15T15:21:28.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.373+0000 7f91ecab2640 1 -- 172.21.15.105:0/2624697775 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91d4003f60 msgr2=0x7f91d4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:28.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.373+0000 7f91ecab2640 1 --2- 172.21.15.105:0/2624697775 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91d4003f60 0x7f91d4004800 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f91d0009ac0 tx=0x7f91d4005830 comp rx=0 tx=0).stop 2024-04-15T15:21:28.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.373+0000 7f91e6d9d640 1 -- 172.21.15.105:0/2624697775 reap_dead start 2024-04-15T15:21:28.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.373+0000 7f91ecab2640 1 -- 172.21.15.105:0/2624697775 shutdown_connections 2024-04-15T15:21:28.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.373+0000 7f91ecab2640 1 -- 172.21.15.105:0/2624697775 >> 172.21.15.105:0/2624697775 conn(0x7f91e8067e10 msgr2=0x7f91e80b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:28.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.374+0000 7f91ecab2640 1 -- 172.21.15.105:0/2624697775 shutdown_connections 2024-04-15T15:21:28.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.374+0000 7f91ecab2640 1 -- 172.21.15.105:0/2624697775 wait complete. 2024-04-15T15:21:28.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.375+0000 7f91ecab2640 1 Processor -- start 2024-04-15T15:21:28.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.375+0000 7f91ecab2640 1 -- start start 2024-04-15T15:21:28.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.376+0000 7f91ecab2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91d4003f60 0x7f91e813d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:28.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.376+0000 7f91ecab2640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f91e813d950 0x7f91e8143520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:28.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.376+0000 7f91ecab2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91e81e7850 0x7f91e81e7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:28.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.376+0000 7f91ecab2640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f91e80cf670 con 0x7f91e81e7850 2024-04-15T15:21:28.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.376+0000 7f91ecab2640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f91e80d0230 con 0x7f91d4003f60 2024-04-15T15:21:28.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.376+0000 7f91ecab2640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f91e80cf7f0 con 0x7f91e813d950 2024-04-15T15:21:28.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.376+0000 7f91e6d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91e81e7850 0x7f91e81e7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:28.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.376+0000 7f91e6d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91e81e7850 0x7f91e81e7d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47212/0 (socket says 172.21.15.105:47212) 2024-04-15T15:21:28.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.376+0000 7f91e6d9d640 1 -- 172.21.15.105:0/3600569584 learned_addr learned my addr 172.21.15.105:0/3600569584 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:28.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.376+0000 7f91e659c640 1 --2- 172.21.15.105:0/3600569584 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91d4003f60 0x7f91e813d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:28.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.376+0000 7f91dffff640 1 --2- 172.21.15.105:0/3600569584 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f91e813d950 0x7f91e8143520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:28.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.376+0000 7f91e6d9d640 1 -- 172.21.15.105:0/3600569584 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f91e813d950 msgr2=0x7f91e8143520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:28.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.376+0000 7f91e6d9d640 1 --2- 172.21.15.105:0/3600569584 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f91e813d950 0x7f91e8143520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:28.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.376+0000 7f91e6d9d640 1 -- 172.21.15.105:0/3600569584 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91d4003f60 msgr2=0x7f91e813d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:28.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.376+0000 7f91e6d9d640 1 --2- 172.21.15.105:0/3600569584 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91d4003f60 0x7f91e813d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:28.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.376+0000 7f91e6d9d640 1 -- 172.21.15.105:0/3600569584 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91e8143e90 con 0x7f91e81e7850 2024-04-15T15:21:28.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.376+0000 7f91e659c640 1 --2- 172.21.15.105:0/3600569584 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91d4003f60 0x7f91e813d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:21:28.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.376+0000 7f91e6d9d640 1 --2- 172.21.15.105:0/3600569584 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91e81e7850 0x7f91e81e7d00 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f91e80b32e0 tx=0x7f91e000fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:28.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.377+0000 7f91deffd640 1 -- 172.21.15.105:0/3600569584 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e001a070 con 0x7f91e81e7850 2024-04-15T15:21:28.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.377+0000 7f91deffd640 1 -- 172.21.15.105:0/3600569584 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91e0006d60 con 0x7f91e81e7850 2024-04-15T15:21:28.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.377+0000 7f91ecab2640 1 -- 172.21.15.105:0/3600569584 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91e8144170 con 0x7f91e81e7850 2024-04-15T15:21:28.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.377+0000 7f91deffd640 1 -- 172.21.15.105:0/3600569584 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e0003b90 con 0x7f91e81e7850 2024-04-15T15:21:28.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.377+0000 7f91ecab2640 1 -- 172.21.15.105:0/3600569584 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91e81e4f50 con 0x7f91e81e7850 2024-04-15T15:21:28.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.378+0000 7f91ecab2640 1 -- 172.21.15.105:0/3600569584 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f91e8068a40 con 0x7f91e81e7850 2024-04-15T15:21:28.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.378+0000 7f91deffd640 1 -- 172.21.15.105:0/3600569584 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f91e001d070 con 0x7f91e81e7850 2024-04-15T15:21:28.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.378+0000 7f91deffd640 1 --2- 172.21.15.105:0/3600569584 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f91c806c980 0x7f91c806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:28.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.379+0000 7f91deffd640 1 -- 172.21.15.105:0/3600569584 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5029+0+0 (secure 0 0 0) 0x7f91e008af20 con 0x7f91e81e7850 2024-04-15T15:21:28.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.379+0000 7f91e659c640 1 --2- 172.21.15.105:0/3600569584 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f91c806c980 0x7f91c806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:28.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.379+0000 7f91deffd640 1 --2- 172.21.15.105:0/3600569584 >> [v2:172.21.15.105:6800/2873040054,v1:172.21.15.105:6801/2873040054] conn(0x7f91c80728e0 0x7f91c8074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:28.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.379+0000 7f91deffd640 1 -- 172.21.15.105:0/3600569584 --> [v2:172.21.15.105:6800/2873040054,v1:172.21.15.105:6801/2873040054] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f91e0011e50 con 0x7f91c80728e0 2024-04-15T15:21:28.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.379+0000 7f91deffd640 1 -- 172.21.15.105:0/3600569584 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f91e008d610 con 0x7f91e81e7850 2024-04-15T15:21:28.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.379+0000 7f91dffff640 1 --2- 172.21.15.105:0/3600569584 >> [v2:172.21.15.105:6800/2873040054,v1:172.21.15.105:6801/2873040054] conn(0x7f91c80728e0 0x7f91c8074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:28.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.379+0000 7f91e659c640 1 --2- 172.21.15.105:0/3600569584 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f91c806c980 0x7f91c806ee40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f91d8001eb0 tx=0x7f91d8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:28.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.379+0000 7f91dffff640 1 --2- 172.21.15.105:0/3600569584 >> [v2:172.21.15.105:6800/2873040054,v1:172.21.15.105:6801/2873040054] conn(0x7f91c80728e0 0x7f91c8074d00 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:28.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.380+0000 7f91deffd640 1 -- 172.21.15.105:0/3600569584 <== osd.0 v2:172.21.15.105:6800/2873040054 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f91e0011e50 con 0x7f91c80728e0 2024-04-15T15:21:28.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.405+0000 7f91ecab2640 1 -- 172.21.15.105:0/3600569584 --> [v2:172.21.15.105:6800/2873040054,v1:172.21.15.105:6801/2873040054] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f91e80b3800 con 0x7f91c80728e0 2024-04-15T15:21:28.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.405+0000 7f91deffd640 1 -- 172.21.15.105:0/3600569584 <== osd.0 v2:172.21.15.105:6800/2873040054 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f91e80b3800 con 0x7f91c80728e0 2024-04-15T15:21:28.407 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-04-15T15:21:28.407 INFO:teuthology.orchestra.run.smithi105.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-15T15:21:28.407 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-04-15T15:21:28.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.405+0000 7f91ecab2640 1 -- 172.21.15.105:0/3600569584 >> [v2:172.21.15.105:6800/2873040054,v1:172.21.15.105:6801/2873040054] conn(0x7f91c80728e0 msgr2=0x7f91c8074d00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:28.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.405+0000 7f91ecab2640 1 --2- 172.21.15.105:0/3600569584 >> [v2:172.21.15.105:6800/2873040054,v1:172.21.15.105:6801/2873040054] conn(0x7f91c80728e0 0x7f91c8074d00 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:28.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.405+0000 7f91ecab2640 1 -- 172.21.15.105:0/3600569584 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f91c806c980 msgr2=0x7f91c806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:28.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.405+0000 7f91ecab2640 1 --2- 172.21.15.105:0/3600569584 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f91c806c980 0x7f91c806ee40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f91d8001eb0 tx=0x7f91d8009040 comp rx=0 tx=0).stop 2024-04-15T15:21:28.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.405+0000 7f91ecab2640 1 -- 172.21.15.105:0/3600569584 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91e81e7850 msgr2=0x7f91e81e7d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:28.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.405+0000 7f91ecab2640 1 --2- 172.21.15.105:0/3600569584 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91e81e7850 0x7f91e81e7d00 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f91e80b32e0 tx=0x7f91e000fd40 comp rx=0 tx=0).stop 2024-04-15T15:21:28.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.405+0000 7f91e6d9d640 1 -- 172.21.15.105:0/3600569584 reap_dead start 2024-04-15T15:21:28.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.406+0000 7f91ecab2640 1 -- 172.21.15.105:0/3600569584 shutdown_connections 2024-04-15T15:21:28.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.406+0000 7f91ecab2640 1 -- 172.21.15.105:0/3600569584 >> 172.21.15.105:0/3600569584 conn(0x7f91e8067e10 msgr2=0x7f91e80b3a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:28.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.406+0000 7f91ecab2640 1 -- 172.21.15.105:0/3600569584 shutdown_connections 2024-04-15T15:21:28.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.406+0000 7f91ecab2640 1 -- 172.21.15.105:0/3600569584 wait complete. 2024-04-15T15:21:28.418 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-15T15:21:28.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.522+0000 7fde77635640 1 Processor -- start 2024-04-15T15:21:28.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.522+0000 7fde77635640 1 -- start start 2024-04-15T15:21:28.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.523+0000 7fde77635640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fde700d18e0 0x7fde70162630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:28.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.523+0000 7fde77635640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde7015ef40 con 0x7fde70162b70 2024-04-15T15:21:28.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.523+0000 7fde77635640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde70163260 con 0x7fde7015eb60 2024-04-15T15:21:28.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.523+0000 7fde77635640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fde70163540 con 0x7fde700d18e0 2024-04-15T15:21:28.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.523+0000 7fde74bd0640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fde700d18e0 0x7fde70162630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:28.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.523+0000 7fde75bd2640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fde70162b70 0x7fde700d3450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:45228/0 (socket says 172.21.15.105:45228) 2024-04-15T15:21:28.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.524+0000 7fde75bd2640 1 -- 172.21.15.105:0/1520252768 learned_addr learned my addr 172.21.15.105:0/1520252768 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:21:28.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.524+0000 7fde74bd0640 1 -- 172.21.15.105:0/1520252768 >> v1:172.21.15.154:6789/0 conn(0x7fde7015eb60 legacy=0x7fde700d11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:28.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.524+0000 7fde74bd0640 1 -- 172.21.15.105:0/1520252768 >> v1:172.21.15.105:6789/0 conn(0x7fde70162b70 legacy=0x7fde700d3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:28.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.524+0000 7fde74bd0640 1 -- 172.21.15.105:0/1520252768 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde700d2970 con 0x7fde700d18e0 2024-04-15T15:21:28.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.524+0000 7fde6ffff640 1 -- 172.21.15.105:0/1520252768 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3108666862 0 0) 0x7fde7015ef40 con 0x7fde70162b70 2024-04-15T15:21:28.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.525+0000 7fde74bd0640 1 --2- 172.21.15.105:0/1520252768 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde700d18e0 0x7fde70162630 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fde640097a0 tx=0x7fde6402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5eaa3faa7bf5db3 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:28.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.525+0000 7fde6ffff640 1 -- 172.21.15.105:0/1520252768 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde64034070 con 0x7fde700d18e0 2024-04-15T15:21:28.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.525+0000 7fde6ffff640 1 -- 172.21.15.105:0/1520252768 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde64030610 con 0x7fde700d18e0 2024-04-15T15:21:28.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.525+0000 7fde6ffff640 1 -- 172.21.15.105:0/1520252768 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde640309e0 con 0x7fde700d18e0 2024-04-15T15:21:28.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.525+0000 7fde77635640 1 -- 172.21.15.105:0/1520252768 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde700d18e0 msgr2=0x7fde70162630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:28.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.525+0000 7fde77635640 1 --2- 172.21.15.105:0/1520252768 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde700d18e0 0x7fde70162630 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fde640097a0 tx=0x7fde6402b960 comp rx=0 tx=0).stop 2024-04-15T15:21:28.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.525+0000 7fde77635640 1 -- 172.21.15.105:0/1520252768 shutdown_connections 2024-04-15T15:21:28.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.525+0000 7fde77635640 1 --2- 172.21.15.105:0/1520252768 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde700d18e0 0x7fde70162630 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:28.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.525+0000 7fde77635640 1 -- 172.21.15.105:0/1520252768 >> 172.21.15.105:0/1520252768 conn(0x7fde70067720 msgr2=0x7fde70067b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:28.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.525+0000 7fde77635640 1 -- 172.21.15.105:0/1520252768 shutdown_connections 2024-04-15T15:21:28.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.525+0000 7fde77635640 1 -- 172.21.15.105:0/1520252768 wait complete. 2024-04-15T15:21:28.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.526+0000 7fde77635640 1 Processor -- start 2024-04-15T15:21:28.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.527+0000 7fde77635640 1 -- start start 2024-04-15T15:21:28.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.527+0000 7fde77635640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde700d18e0 0x7fde700cdf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:28.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.527+0000 7fde77635640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde7015eb60 0x7fde7015f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:28.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.527+0000 7fde77635640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fde70162b70 0x7fde700cc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:28.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.527+0000 7fde77635640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fde70162f80 con 0x7fde7015eb60 2024-04-15T15:21:28.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.527+0000 7fde77635640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fde700d2fc0 con 0x7fde700d18e0 2024-04-15T15:21:28.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.527+0000 7fde77635640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fde700b78f0 con 0x7fde70162b70 2024-04-15T15:21:28.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.527+0000 7fde74bd0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde7015eb60 0x7fde7015f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:28.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.527+0000 7fde74bd0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde7015eb60 0x7fde7015f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47228/0 (socket says 172.21.15.105:47228) 2024-04-15T15:21:28.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.527+0000 7fde74bd0640 1 -- 172.21.15.105:0/3875143884 learned_addr learned my addr 172.21.15.105:0/3875143884 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:28.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.528+0000 7fde75bd2640 1 --2- 172.21.15.105:0/3875143884 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fde70162b70 0x7fde700cc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:28.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.528+0000 7fde753d1640 1 --2- 172.21.15.105:0/3875143884 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde700d18e0 0x7fde700cdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:28.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.528+0000 7fde74bd0640 1 -- 172.21.15.105:0/3875143884 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fde70162b70 msgr2=0x7fde700cc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:28.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.528+0000 7fde74bd0640 1 --2- 172.21.15.105:0/3875143884 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fde70162b70 0x7fde700cc8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:28.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.528+0000 7fde74bd0640 1 -- 172.21.15.105:0/3875143884 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde700d18e0 msgr2=0x7fde700cdf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:28.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.528+0000 7fde74bd0640 1 --2- 172.21.15.105:0/3875143884 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde700d18e0 0x7fde700cdf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:28.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.528+0000 7fde74bd0640 1 -- 172.21.15.105:0/3875143884 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde700b4200 con 0x7fde7015eb60 2024-04-15T15:21:28.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.528+0000 7fde74bd0640 1 --2- 172.21.15.105:0/3875143884 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde7015eb60 0x7fde7015f570 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fde6402bf70 tx=0x7fde640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:28.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.528+0000 7fde6dffb640 1 -- 172.21.15.105:0/3875143884 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde64034040 con 0x7fde7015eb60 2024-04-15T15:21:28.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.528+0000 7fde6dffb640 1 -- 172.21.15.105:0/3875143884 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde64039a10 con 0x7fde7015eb60 2024-04-15T15:21:28.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.528+0000 7fde6dffb640 1 -- 172.21.15.105:0/3875143884 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde64039e20 con 0x7fde7015eb60 2024-04-15T15:21:28.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.528+0000 7fde77635640 1 -- 172.21.15.105:0/3875143884 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde700b4a60 con 0x7fde7015eb60 2024-04-15T15:21:28.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.528+0000 7fde77635640 1 -- 172.21.15.105:0/3875143884 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde700b4780 con 0x7fde7015eb60 2024-04-15T15:21:28.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.529+0000 7fde77635640 1 -- 172.21.15.105:0/3875143884 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde70068a40 con 0x7fde7015eb60 2024-04-15T15:21:28.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.530+0000 7fde6dffb640 1 -- 172.21.15.105:0/3875143884 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fde64037070 con 0x7fde7015eb60 2024-04-15T15:21:28.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.530+0000 7fde6dffb640 1 --2- 172.21.15.105:0/3875143884 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fde5c06c980 0x7fde5c06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:28.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.530+0000 7fde6dffb640 1 -- 172.21.15.105:0/3875143884 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5029+0+0 (secure 0 0 0) 0x7fde640adb40 con 0x7fde7015eb60 2024-04-15T15:21:28.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.530+0000 7fde753d1640 1 --2- 172.21.15.105:0/3875143884 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fde5c06c980 0x7fde5c06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:28.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.531+0000 7fde753d1640 1 --2- 172.21.15.105:0/3875143884 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fde5c06c980 0x7fde5c06ee40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fde60007fd0 tx=0x7fde60009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:28.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.534+0000 7fde6dffb640 1 -- 172.21.15.105:0/3875143884 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fde6407c110 con 0x7fde7015eb60 2024-04-15T15:21:28.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.711+0000 7fde77635640 1 -- 172.21.15.105:0/3875143884 --> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7fde700d23f0 con 0x7fde5c06c980 2024-04-15T15:21:28.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.711+0000 7fde6dffb640 1 -- 172.21.15.105:0/3875143884 <== mgr.4107 v2:172.21.15.105:6832/3506549322 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fde700d23f0 con 0x7fde5c06c980 2024-04-15T15:21:28.713 INFO:teuthology.orchestra.run.smithi105.stderr:instructed osd(s) 0 to deep-scrub 2024-04-15T15:21:28.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.713+0000 7fde77635640 1 -- 172.21.15.105:0/3875143884 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fde5c06c980 msgr2=0x7fde5c06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:28.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.713+0000 7fde77635640 1 --2- 172.21.15.105:0/3875143884 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fde5c06c980 0x7fde5c06ee40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fde60007fd0 tx=0x7fde60009040 comp rx=0 tx=0).stop 2024-04-15T15:21:28.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.713+0000 7fde77635640 1 -- 172.21.15.105:0/3875143884 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde7015eb60 msgr2=0x7fde7015f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:28.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.713+0000 7fde77635640 1 --2- 172.21.15.105:0/3875143884 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde7015eb60 0x7fde7015f570 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fde6402bf70 tx=0x7fde640046c0 comp rx=0 tx=0).stop 2024-04-15T15:21:28.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.713+0000 7fde77635640 1 -- 172.21.15.105:0/3875143884 shutdown_connections 2024-04-15T15:21:28.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.713+0000 7fde77635640 1 --2- 172.21.15.105:0/3875143884 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fde70162b70 0x7fde700cc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:28.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.713+0000 7fde77635640 1 --2- 172.21.15.105:0/3875143884 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fde5c06c980 0x7fde5c06ee40 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:28.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.713+0000 7fde77635640 1 --2- 172.21.15.105:0/3875143884 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde7015eb60 0x7fde7015f570 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:28.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.713+0000 7fde77635640 1 --2- 172.21.15.105:0/3875143884 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde700d18e0 0x7fde700cdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:28.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.713+0000 7fde77635640 1 -- 172.21.15.105:0/3875143884 >> 172.21.15.105:0/3875143884 conn(0x7fde70067720 msgr2=0x7fde70162380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:28.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.713+0000 7fde77635640 1 -- 172.21.15.105:0/3875143884 shutdown_connections 2024-04-15T15:21:28.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.713+0000 7fde77635640 1 -- 172.21.15.105:0/3875143884 wait complete. 2024-04-15T15:21:28.725 INFO:tasks.ceph:Scrubbing osd.1 2024-04-15T15:21:28.726 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-15T15:21:28.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.832+0000 7f09fc729640 1 Processor -- start 2024-04-15T15:21:28.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.832+0000 7f09fc729640 1 -- start start 2024-04-15T15:21:28.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.833+0000 7f09fc729640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f09f415e830 0x7f09f40d0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:28.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.833+0000 7f09fc729640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09f40ba0f0 0x7f09f40cf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:28.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.833+0000 7f09fc729640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09f40d17d0 con 0x7f09f40cf970 2024-04-15T15:21:28.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.833+0000 7f09fc729640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f09f40d2070 con 0x7f09f415e830 2024-04-15T15:21:28.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.833+0000 7f09fc729640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f09f40cfde0 con 0x7f09f40ba0f0 2024-04-15T15:21:28.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.834+0000 7f09facc6640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f09f40cf970 0x7f09f41624d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:45240/0 (socket says 172.21.15.105:45240) 2024-04-15T15:21:28.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.834+0000 7f09facc6640 1 -- 172.21.15.105:0/2237054674 learned_addr learned my addr 172.21.15.105:0/2237054674 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:21:28.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.834+0000 7f09f9cc4640 1 --2- 172.21.15.105:0/2237054674 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09f40ba0f0 0x7f09f40cf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:28.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.834+0000 7f09fa4c5640 1 --2- 172.21.15.105:0/2237054674 >> v2:172.21.15.154:3300/0 conn(0x7f09f415e830 0x7f09f40d0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:28.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.834+0000 7f09f94c3640 1 -- 172.21.15.105:0/2237054674 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1809677517 0 0) 0x7f09f40d17d0 con 0x7f09f40cf970 2024-04-15T15:21:28.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.834+0000 7f09f94c3640 1 -- 172.21.15.105:0/2237054674 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f09e0003520 con 0x7f09f40cf970 2024-04-15T15:21:28.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.834+0000 7f09fa4c5640 1 -- 172.21.15.105:0/2237054674 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09f40ba0f0 msgr2=0x7f09f40cf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:28.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.834+0000 7f09f94c3640 1 -- 172.21.15.105:0/2237054674 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1140852784 0 0) 0x7f09e0003520 con 0x7f09f40cf970 2024-04-15T15:21:28.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.834+0000 7f09fa4c5640 1 --2- 172.21.15.105:0/2237054674 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09f40ba0f0 0x7f09f40cf430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:28.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.834+0000 7f09fa4c5640 1 -- 172.21.15.105:0/2237054674 >> v1:172.21.15.105:6789/0 conn(0x7f09f40cf970 legacy=0x7f09f41624d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:28.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.834+0000 7f09fa4c5640 1 -- 172.21.15.105:0/2237054674 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09f40d1ab0 con 0x7f09f415e830 2024-04-15T15:21:28.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.834+0000 7f09f9cc4640 1 --2- 172.21.15.105:0/2237054674 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09f40ba0f0 0x7f09f40cf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:21:28.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.835+0000 7f09fa4c5640 1 --2- 172.21.15.105:0/2237054674 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09f415e830 0x7f09f40d0f00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f09f00097a0 tx=0x7f09f002b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a42687632a84959 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:28.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.835+0000 7f09f94c3640 1 -- 172.21.15.105:0/2237054674 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f0037070 con 0x7f09f415e830 2024-04-15T15:21:28.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.835+0000 7f09f94c3640 1 -- 172.21.15.105:0/2237054674 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09f0030510 con 0x7f09f415e830 2024-04-15T15:21:28.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.835+0000 7f09f94c3640 1 -- 172.21.15.105:0/2237054674 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09ec0019f0 con 0x7f09f415e830 2024-04-15T15:21:28.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.836+0000 7f09fc729640 1 -- 172.21.15.105:0/2237054674 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09f415e830 msgr2=0x7f09f40d0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:28.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.836+0000 7f09fc729640 1 --2- 172.21.15.105:0/2237054674 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09f415e830 0x7f09f40d0f00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f09f00097a0 tx=0x7f09f002b790 comp rx=0 tx=0).stop 2024-04-15T15:21:28.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.836+0000 7f09fc729640 1 -- 172.21.15.105:0/2237054674 shutdown_connections 2024-04-15T15:21:28.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.836+0000 7f09fc729640 1 --2- 172.21.15.105:0/2237054674 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09f40ba0f0 0x7f09f40cf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:28.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.836+0000 7f09fc729640 1 --2- 172.21.15.105:0/2237054674 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09f415e830 0x7f09f40d0f00 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:28.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.836+0000 7f09fc729640 1 -- 172.21.15.105:0/2237054674 >> 172.21.15.105:0/2237054674 conn(0x7f09f406ccb0 msgr2=0x7f09f406d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:28.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.836+0000 7f09fc729640 1 -- 172.21.15.105:0/2237054674 shutdown_connections 2024-04-15T15:21:28.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.836+0000 7f09fc729640 1 -- 172.21.15.105:0/2237054674 wait complete. 2024-04-15T15:21:28.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.837+0000 7f09fc729640 1 Processor -- start 2024-04-15T15:21:28.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.837+0000 7f09fc729640 1 -- start start 2024-04-15T15:21:28.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.838+0000 7f09fc729640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f09f40ba0f0 0x7f09f4155f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:28.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.838+0000 7f09fc729640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09f40cf970 0x7f09f4147a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:28.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.838+0000 7f09fc729640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09f415e830 0x7f09f4149b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:28.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.838+0000 7f09fc729640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f09f415ec10 con 0x7f09f40ba0f0 2024-04-15T15:21:28.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.838+0000 7f09fc729640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f09f40d1440 con 0x7f09f40cf970 2024-04-15T15:21:28.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.838+0000 7f09fc729640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f09f415ed90 con 0x7f09f415e830 2024-04-15T15:21:28.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.838+0000 7f09fa4c5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f09f40ba0f0 0x7f09f4155f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:28.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.838+0000 7f09fa4c5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f09f40ba0f0 0x7f09f4155f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47236/0 (socket says 172.21.15.105:47236) 2024-04-15T15:21:28.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.838+0000 7f09fa4c5640 1 -- 172.21.15.105:0/915768133 learned_addr learned my addr 172.21.15.105:0/915768133 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:28.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.838+0000 7f09fa4c5640 1 -- 172.21.15.105:0/915768133 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09f415e830 msgr2=0x7f09f4149b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:28.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.838+0000 7f09fa4c5640 1 --2- 172.21.15.105:0/915768133 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09f415e830 0x7f09f4149b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:28.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.838+0000 7f09fa4c5640 1 -- 172.21.15.105:0/915768133 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09f40cf970 msgr2=0x7f09f4147a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:28.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.838+0000 7f09f9cc4640 1 --2- 172.21.15.105:0/915768133 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09f40cf970 0x7f09f4147a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:28.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.838+0000 7f09fa4c5640 1 --2- 172.21.15.105:0/915768133 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09f40cf970 0x7f09f4147a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:28.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.838+0000 7f09fa4c5640 1 -- 172.21.15.105:0/915768133 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09f4156450 con 0x7f09f40ba0f0 2024-04-15T15:21:28.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.838+0000 7f09fa4c5640 1 --2- 172.21.15.105:0/915768133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f09f40ba0f0 0x7f09f4155f10 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f09f002b7d0 tx=0x7f09f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:28.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.838+0000 7f09eaffd640 1 -- 172.21.15.105:0/915768133 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f0041020 con 0x7f09f40ba0f0 2024-04-15T15:21:28.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.838+0000 7f09eaffd640 1 -- 172.21.15.105:0/915768133 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09f003c870 con 0x7f09f40ba0f0 2024-04-15T15:21:28.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.838+0000 7f09fc729640 1 -- 172.21.15.105:0/915768133 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09f414a5c0 con 0x7f09f40ba0f0 2024-04-15T15:21:28.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.838+0000 7f09eaffd640 1 -- 172.21.15.105:0/915768133 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f0037040 con 0x7f09f40ba0f0 2024-04-15T15:21:28.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.838+0000 7f09fc729640 1 -- 172.21.15.105:0/915768133 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09f415ce30 con 0x7f09f40ba0f0 2024-04-15T15:21:28.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.839+0000 7f09fc729640 1 -- 172.21.15.105:0/915768133 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f09f414a9c0 con 0x7f09f40ba0f0 2024-04-15T15:21:28.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.840+0000 7f09eaffd640 1 -- 172.21.15.105:0/915768133 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f09f0044020 con 0x7f09f40ba0f0 2024-04-15T15:21:28.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.840+0000 7f09eaffd640 1 --2- 172.21.15.105:0/915768133 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f09d806c980 0x7f09d806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:28.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.840+0000 7f09eaffd640 1 -- 172.21.15.105:0/915768133 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5029+0+0 (secure 0 0 0) 0x7f09f00ad0d0 con 0x7f09f40ba0f0 2024-04-15T15:21:28.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.840+0000 7f09f9cc4640 1 --2- 172.21.15.105:0/915768133 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f09d806c980 0x7f09d806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:28.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.841+0000 7f09eaffd640 1 --2- 172.21.15.105:0/915768133 >> [v2:172.21.15.105:6816/2494994479,v1:172.21.15.105:6818/2494994479] conn(0x7f09d80728e0 0x7f09d8074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:28.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.841+0000 7f09eaffd640 1 -- 172.21.15.105:0/915768133 --> [v2:172.21.15.105:6816/2494994479,v1:172.21.15.105:6818/2494994479] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f09f0034ec0 con 0x7f09d80728e0 2024-04-15T15:21:28.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.841+0000 7f09eaffd640 1 -- 172.21.15.105:0/915768133 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f09f00b4050 con 0x7f09f40ba0f0 2024-04-15T15:21:28.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.841+0000 7f09facc6640 1 --2- 172.21.15.105:0/915768133 >> [v2:172.21.15.105:6816/2494994479,v1:172.21.15.105:6818/2494994479] conn(0x7f09d80728e0 0x7f09d8074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:28.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.841+0000 7f09f9cc4640 1 --2- 172.21.15.105:0/915768133 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f09d806c980 0x7f09d806ee40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f09e4009560 tx=0x7f09e4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:28.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.841+0000 7f09facc6640 1 --2- 172.21.15.105:0/915768133 >> [v2:172.21.15.105:6816/2494994479,v1:172.21.15.105:6818/2494994479] conn(0x7f09d80728e0 0x7f09d8074d00 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:28.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.842+0000 7f09eaffd640 1 -- 172.21.15.105:0/915768133 <== osd.1 v2:172.21.15.105:6816/2494994479 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f09f0034ec0 con 0x7f09d80728e0 2024-04-15T15:21:28.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.855+0000 7f09fc729640 1 -- 172.21.15.105:0/915768133 --> [v2:172.21.15.105:6816/2494994479,v1:172.21.15.105:6818/2494994479] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f09f4068a40 con 0x7f09d80728e0 2024-04-15T15:21:28.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.855+0000 7f09eaffd640 1 -- 172.21.15.105:0/915768133 <== osd.1 v2:172.21.15.105:6816/2494994479 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f09f4068a40 con 0x7f09d80728e0 2024-04-15T15:21:28.857 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-04-15T15:21:28.857 INFO:teuthology.orchestra.run.smithi105.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-15T15:21:28.857 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-04-15T15:21:28.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.855+0000 7f09fc729640 1 -- 172.21.15.105:0/915768133 >> [v2:172.21.15.105:6816/2494994479,v1:172.21.15.105:6818/2494994479] conn(0x7f09d80728e0 msgr2=0x7f09d8074d00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:28.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.855+0000 7f09fc729640 1 --2- 172.21.15.105:0/915768133 >> [v2:172.21.15.105:6816/2494994479,v1:172.21.15.105:6818/2494994479] conn(0x7f09d80728e0 0x7f09d8074d00 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:28.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.855+0000 7f09fc729640 1 -- 172.21.15.105:0/915768133 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f09d806c980 msgr2=0x7f09d806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:28.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.855+0000 7f09fc729640 1 --2- 172.21.15.105:0/915768133 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f09d806c980 0x7f09d806ee40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f09e4009560 tx=0x7f09e4008040 comp rx=0 tx=0).stop 2024-04-15T15:21:28.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.855+0000 7f09fc729640 1 -- 172.21.15.105:0/915768133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f09f40ba0f0 msgr2=0x7f09f4155f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:28.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.855+0000 7f09fc729640 1 --2- 172.21.15.105:0/915768133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f09f40ba0f0 0x7f09f4155f10 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f09f002b7d0 tx=0x7f09f00046c0 comp rx=0 tx=0).stop 2024-04-15T15:21:28.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.855+0000 7f09facc6640 1 -- 172.21.15.105:0/915768133 reap_dead start 2024-04-15T15:21:28.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.855+0000 7f09fc729640 1 -- 172.21.15.105:0/915768133 shutdown_connections 2024-04-15T15:21:28.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.855+0000 7f09fc729640 1 -- 172.21.15.105:0/915768133 >> 172.21.15.105:0/915768133 conn(0x7f09f406ccb0 msgr2=0x7f09f40c8340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:28.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.855+0000 7f09fc729640 1 -- 172.21.15.105:0/915768133 shutdown_connections 2024-04-15T15:21:28.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.855+0000 7f09fc729640 1 -- 172.21.15.105:0/915768133 wait complete. 2024-04-15T15:21:28.867 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-15T15:21:28.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.969+0000 7f7bf7493640 1 Processor -- start 2024-04-15T15:21:28.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.969+0000 7f7bf7493640 1 -- start start 2024-04-15T15:21:28.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.970+0000 7f7bf7493640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7bf0145e00 0x7f7bf00ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:28.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.970+0000 7f7bf7493640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7bf0072570 0x7f7bf006df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:28.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.970+0000 7f7bf7493640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bf00ccfb0 con 0x7f7bf00ced30 2024-04-15T15:21:28.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.970+0000 7f7bf7493640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f7bf00cdbc0 con 0x7f7bf0145e00 2024-04-15T15:21:28.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.970+0000 7f7bf7493640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f7bf00cd290 con 0x7f7bf0072570 2024-04-15T15:21:28.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.970+0000 7f7bf522f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7bf0145e00 0x7f7bf00ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:28.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.970+0000 7f7bf522f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7bf0145e00 0x7f7bf00ce7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47242/0 (socket says 172.21.15.105:47242) 2024-04-15T15:21:28.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.970+0000 7f7bf522f640 1 -- 172.21.15.105:0/2695740779 learned_addr learned my addr 172.21.15.105:0/2695740779 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:28.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.971+0000 7f7bf5a30640 1 --2- 172.21.15.105:0/2695740779 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7bf0072570 0x7f7bf006df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:28.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.971+0000 7f7bf522f640 1 -- 172.21.15.105:0/2695740779 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7bf0072570 msgr2=0x7f7bf006df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:28.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.971+0000 7f7bf522f640 1 --2- 172.21.15.105:0/2695740779 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7bf0072570 0x7f7bf006df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:28.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.971+0000 7f7bf522f640 1 -- 172.21.15.105:0/2695740779 >> v1:172.21.15.154:6789/0 conn(0x7f7bf00ced30 legacy=0x7f7bf0071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:28.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.971+0000 7f7bf522f640 1 -- 172.21.15.105:0/2695740779 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bf00cd570 con 0x7f7bf0145e00 2024-04-15T15:21:28.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.971+0000 7f7bf522f640 1 --2- 172.21.15.105:0/2695740779 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bf0145e00 0x7f7bf00ce7f0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f7bec009870 tx=0x7f7bec02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d90889cf9b007397 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:28.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.972+0000 7f7be7fff640 1 -- 172.21.15.105:0/2695740779 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bec035020 con 0x7f7bf0145e00 2024-04-15T15:21:28.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.972+0000 7f7be7fff640 1 -- 172.21.15.105:0/2695740779 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bec031640 con 0x7f7bf0145e00 2024-04-15T15:21:28.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.972+0000 7f7be7fff640 1 -- 172.21.15.105:0/2695740779 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bec02d040 con 0x7f7bf0145e00 2024-04-15T15:21:28.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.972+0000 7f7bf7493640 1 -- 172.21.15.105:0/2695740779 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bf0145e00 msgr2=0x7f7bf00ce7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:28.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.972+0000 7f7bf7493640 1 --2- 172.21.15.105:0/2695740779 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bf0145e00 0x7f7bf00ce7f0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f7bec009870 tx=0x7f7bec02b8f0 comp rx=0 tx=0).stop 2024-04-15T15:21:28.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.972+0000 7f7bf7493640 1 -- 172.21.15.105:0/2695740779 shutdown_connections 2024-04-15T15:21:28.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.972+0000 7f7bf7493640 1 --2- 172.21.15.105:0/2695740779 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7bf0072570 0x7f7bf006df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:28.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.972+0000 7f7bf7493640 1 --2- 172.21.15.105:0/2695740779 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bf0145e00 0x7f7bf00ce7f0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:28.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.972+0000 7f7bf7493640 1 -- 172.21.15.105:0/2695740779 >> 172.21.15.105:0/2695740779 conn(0x7f7bf0067d50 msgr2=0x7f7bf006b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:28.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.972+0000 7f7bf7493640 1 -- 172.21.15.105:0/2695740779 shutdown_connections 2024-04-15T15:21:28.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.972+0000 7f7bf7493640 1 -- 172.21.15.105:0/2695740779 wait complete. 2024-04-15T15:21:28.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.973+0000 7f7bf7493640 1 Processor -- start 2024-04-15T15:21:28.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.973+0000 7f7bf7493640 1 -- start start 2024-04-15T15:21:28.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.974+0000 7f7bf7493640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7bf0072570 0x7f7bf00aeee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:28.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.974+0000 7f7bf7493640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bf00ced30 0x7f7bf0144d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:28.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.974+0000 7f7bf7493640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7bf0145e00 0x7f7bf015e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:28.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.974+0000 7f7bf7493640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf01461e0 con 0x7f7bf00ced30 2024-04-15T15:21:28.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.974+0000 7f7bf7493640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf00ce370 con 0x7f7bf0072570 2024-04-15T15:21:28.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.974+0000 7f7bf7493640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f7bf0146360 con 0x7f7bf0145e00 2024-04-15T15:21:28.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.974+0000 7f7bf4a2e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bf00ced30 0x7f7bf0144d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:28.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.974+0000 7f7bf4a2e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bf00ced30 0x7f7bf0144d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47248/0 (socket says 172.21.15.105:47248) 2024-04-15T15:21:28.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.974+0000 7f7bf4a2e640 1 -- 172.21.15.105:0/771750264 learned_addr learned my addr 172.21.15.105:0/771750264 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:28.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.974+0000 7f7bf5a30640 1 --2- 172.21.15.105:0/771750264 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7bf0145e00 0x7f7bf015e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:28.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.974+0000 7f7bf522f640 1 --2- 172.21.15.105:0/771750264 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7bf0072570 0x7f7bf00aeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:28.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.975+0000 7f7bf4a2e640 1 -- 172.21.15.105:0/771750264 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7bf0145e00 msgr2=0x7f7bf015e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:28.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.975+0000 7f7bf4a2e640 1 --2- 172.21.15.105:0/771750264 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7bf0145e00 0x7f7bf015e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:28.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.975+0000 7f7bf4a2e640 1 -- 172.21.15.105:0/771750264 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7bf0072570 msgr2=0x7f7bf00aeee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:28.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.975+0000 7f7bf4a2e640 1 --2- 172.21.15.105:0/771750264 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7bf0072570 0x7f7bf00aeee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:28.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.975+0000 7f7bf4a2e640 1 -- 172.21.15.105:0/771750264 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bf00ae120 con 0x7f7bf00ced30 2024-04-15T15:21:28.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.975+0000 7f7bf5a30640 1 --2- 172.21.15.105:0/771750264 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7bf0145e00 0x7f7bf015e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:21:28.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.975+0000 7f7bf522f640 1 --2- 172.21.15.105:0/771750264 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7bf0072570 0x7f7bf00aeee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:21:28.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.975+0000 7f7bf4a2e640 1 --2- 172.21.15.105:0/771750264 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bf00ced30 0x7f7bf0144d40 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f7be000c460 tx=0x7f7be000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:28.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.975+0000 7f7be5ffb640 1 -- 172.21.15.105:0/771750264 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be0016020 con 0x7f7bf00ced30 2024-04-15T15:21:28.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.975+0000 7f7be5ffb640 1 -- 172.21.15.105:0/771750264 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7be0012870 con 0x7f7bf00ced30 2024-04-15T15:21:28.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.975+0000 7f7bf7493640 1 -- 172.21.15.105:0/771750264 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bf00ae400 con 0x7f7bf00ced30 2024-04-15T15:21:28.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.975+0000 7f7be5ffb640 1 -- 172.21.15.105:0/771750264 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be0012c60 con 0x7f7bf00ced30 2024-04-15T15:21:28.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.975+0000 7f7bf7493640 1 -- 172.21.15.105:0/771750264 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bf00ae9c0 con 0x7f7bf00ced30 2024-04-15T15:21:28.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.976+0000 7f7bf7493640 1 -- 172.21.15.105:0/771750264 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bf00af480 con 0x7f7bf00ced30 2024-04-15T15:21:28.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.977+0000 7f7be5ffb640 1 -- 172.21.15.105:0/771750264 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f7be0019020 con 0x7f7bf00ced30 2024-04-15T15:21:28.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.977+0000 7f7be5ffb640 1 --2- 172.21.15.105:0/771750264 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f7bd406c980 0x7f7bd406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:28.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.977+0000 7f7be5ffb640 1 -- 172.21.15.105:0/771750264 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5029+0+0 (secure 0 0 0) 0x7f7be008b560 con 0x7f7bf00ced30 2024-04-15T15:21:28.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.977+0000 7f7bf522f640 1 --2- 172.21.15.105:0/771750264 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f7bd406c980 0x7f7bd406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:28.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.978+0000 7f7bf522f640 1 --2- 172.21.15.105:0/771750264 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f7bd406c980 0x7f7bd406ee40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7bec036000 tx=0x7f7bec03c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:28.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:28.982+0000 7f7be5ffb640 1 -- 172.21.15.105:0/771750264 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7be0020030 con 0x7f7bf00ced30 2024-04-15T15:21:29.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.159+0000 7f7bf7493640 1 -- 172.21.15.105:0/771750264 --> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f7bf0068a40 con 0x7f7bd406c980 2024-04-15T15:21:29.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.159+0000 7f7be5ffb640 1 -- 172.21.15.105:0/771750264 <== mgr.4107 v2:172.21.15.105:6832/3506549322 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f7bf0068a40 con 0x7f7bd406c980 2024-04-15T15:21:29.161 INFO:teuthology.orchestra.run.smithi105.stderr:instructed osd(s) 1 to deep-scrub 2024-04-15T15:21:29.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.161+0000 7f7bf7493640 1 -- 172.21.15.105:0/771750264 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f7bd406c980 msgr2=0x7f7bd406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:29.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.161+0000 7f7bf7493640 1 --2- 172.21.15.105:0/771750264 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f7bd406c980 0x7f7bd406ee40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7bec036000 tx=0x7f7bec03c040 comp rx=0 tx=0).stop 2024-04-15T15:21:29.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.161+0000 7f7bf7493640 1 -- 172.21.15.105:0/771750264 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bf00ced30 msgr2=0x7f7bf0144d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:29.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.161+0000 7f7bf7493640 1 --2- 172.21.15.105:0/771750264 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bf00ced30 0x7f7bf0144d40 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f7be000c460 tx=0x7f7be000c920 comp rx=0 tx=0).stop 2024-04-15T15:21:29.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.161+0000 7f7bf7493640 1 -- 172.21.15.105:0/771750264 shutdown_connections 2024-04-15T15:21:29.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.161+0000 7f7bf7493640 1 --2- 172.21.15.105:0/771750264 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7bf0145e00 0x7f7bf015e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:29.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.161+0000 7f7bf7493640 1 --2- 172.21.15.105:0/771750264 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f7bd406c980 0x7f7bd406ee40 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:29.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.161+0000 7f7bf7493640 1 --2- 172.21.15.105:0/771750264 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bf00ced30 0x7f7bf0144d40 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:29.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.161+0000 7f7bf7493640 1 --2- 172.21.15.105:0/771750264 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7bf0072570 0x7f7bf00aeee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:29.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.161+0000 7f7bf7493640 1 -- 172.21.15.105:0/771750264 >> 172.21.15.105:0/771750264 conn(0x7f7bf0067d50 msgr2=0x7f7bf01567f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:29.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.161+0000 7f7bf7493640 1 -- 172.21.15.105:0/771750264 shutdown_connections 2024-04-15T15:21:29.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.161+0000 7f7bf7493640 1 -- 172.21.15.105:0/771750264 wait complete. 2024-04-15T15:21:29.174 INFO:tasks.ceph:Scrubbing osd.2 2024-04-15T15:21:29.174 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-15T15:21:29.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.279+0000 7f5013b24640 1 Processor -- start 2024-04-15T15:21:29.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.279+0000 7f5013b24640 1 -- start start 2024-04-15T15:21:29.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.280+0000 7f5013b24640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f500c15e9c0 0x7f500c162ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:29.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.280+0000 7f5013b24640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f500c0b19d0 0x7f500c0cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:29.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.280+0000 7f5013b24640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f500c163530 con 0x7f500c0b3850 2024-04-15T15:21:29.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.280+0000 7f5013b24640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f500c0ca8d0 con 0x7f500c15e9c0 2024-04-15T15:21:29.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.280+0000 7f5013b24640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f500c0cabb0 con 0x7f500c0b19d0 2024-04-15T15:21:29.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.280+0000 7f50118c0640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f500c15e9c0 0x7f500c162ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:29.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.280+0000 7f50110bf640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f500c0b3850 0x7f500c0b12c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:45254/0 (socket says 172.21.15.105:45254) 2024-04-15T15:21:29.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.280+0000 7f50110bf640 1 -- 172.21.15.105:0/724384947 learned_addr learned my addr 172.21.15.105:0/724384947 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:21:29.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.281+0000 7f50120c1640 1 --2- 172.21.15.105:0/724384947 >> v2:172.21.15.154:3300/0 conn(0x7f500c0b19d0 0x7f500c0cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:29.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.281+0000 7f50108be640 1 -- 172.21.15.105:0/724384947 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3548525570 0 0) 0x7f500c163530 con 0x7f500c0b3850 2024-04-15T15:21:29.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.281+0000 7f50108be640 1 -- 172.21.15.105:0/724384947 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4ff8003520 con 0x7f500c0b3850 2024-04-15T15:21:29.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.281+0000 7f50118c0640 1 -- 172.21.15.105:0/724384947 >> v2:172.21.15.154:3300/0 conn(0x7f500c0b19d0 msgr2=0x7f500c0cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:29.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.281+0000 7f50118c0640 1 --2- 172.21.15.105:0/724384947 >> v2:172.21.15.154:3300/0 conn(0x7f500c0b19d0 0x7f500c0cf3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:29.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.281+0000 7f50118c0640 1 -- 172.21.15.105:0/724384947 >> v1:172.21.15.105:6789/0 conn(0x7f500c0b3850 legacy=0x7f500c0b12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:29.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.281+0000 7f50118c0640 1 -- 172.21.15.105:0/724384947 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f500c0cae90 con 0x7f500c15e9c0 2024-04-15T15:21:29.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.281+0000 7f50120c1640 1 --2- 172.21.15.105:0/724384947 >> v2:172.21.15.154:3300/0 conn(0x7f500c0b19d0 0x7f500c0cf3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:21:29.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.282+0000 7f50118c0640 1 --2- 172.21.15.105:0/724384947 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f500c15e9c0 0x7f500c162ff0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f500c071960 tx=0x7f500802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81f9e1207073130 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:29.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.282+0000 7f50108be640 1 -- 172.21.15.105:0/724384947 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5008039070 con 0x7f500c15e9c0 2024-04-15T15:21:29.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.282+0000 7f50108be640 1 -- 172.21.15.105:0/724384947 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f500802cbf0 con 0x7f500c15e9c0 2024-04-15T15:21:29.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.282+0000 7f50108be640 1 -- 172.21.15.105:0/724384947 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5008034440 con 0x7f500c15e9c0 2024-04-15T15:21:29.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.282+0000 7f5013b24640 1 -- 172.21.15.105:0/724384947 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f500c15e9c0 msgr2=0x7f500c162ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:29.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.282+0000 7f5013b24640 1 --2- 172.21.15.105:0/724384947 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f500c15e9c0 0x7f500c162ff0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f500c071960 tx=0x7f500802be40 comp rx=0 tx=0).stop 2024-04-15T15:21:29.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.282+0000 7f5013b24640 1 -- 172.21.15.105:0/724384947 shutdown_connections 2024-04-15T15:21:29.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.282+0000 7f5013b24640 1 --2- 172.21.15.105:0/724384947 >> v2:172.21.15.154:3300/0 conn(0x7f500c0b19d0 0x7f500c0cf3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:29.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.282+0000 7f5013b24640 1 --2- 172.21.15.105:0/724384947 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f500c15e9c0 0x7f500c162ff0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:29.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.282+0000 7f5013b24640 1 -- 172.21.15.105:0/724384947 >> 172.21.15.105:0/724384947 conn(0x7f500c09aa40 msgr2=0x7f500c0b3140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:29.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.282+0000 7f5013b24640 1 -- 172.21.15.105:0/724384947 shutdown_connections 2024-04-15T15:21:29.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.282+0000 7f5013b24640 1 -- 172.21.15.105:0/724384947 wait complete. 2024-04-15T15:21:29.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.283+0000 7f5013b24640 1 Processor -- start 2024-04-15T15:21:29.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.284+0000 7f5013b24640 1 -- start start 2024-04-15T15:21:29.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.284+0000 7f5013b24640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f500c0b19d0 0x7f500c0cd280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:29.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.284+0000 7f5013b24640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f500c0b3850 0x7f500c149d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:29.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.284+0000 7f5013b24640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f500c15e9c0 0x7f500c0b4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:29.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.284+0000 7f5013b24640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f500c15eda0 con 0x7f500c0b19d0 2024-04-15T15:21:29.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.284+0000 7f5013b24640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f500c0cb450 con 0x7f500c15e9c0 2024-04-15T15:21:29.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.284+0000 7f5013b24640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f500c0cf910 con 0x7f500c0b3850 2024-04-15T15:21:29.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.284+0000 7f50118c0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f500c0b19d0 0x7f500c0cd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:29.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.284+0000 7f50118c0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f500c0b19d0 0x7f500c0cd280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47264/0 (socket says 172.21.15.105:47264) 2024-04-15T15:21:29.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.284+0000 7f50118c0640 1 -- 172.21.15.105:0/2384085953 learned_addr learned my addr 172.21.15.105:0/2384085953 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:29.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.285+0000 7f50120c1640 1 --2- 172.21.15.105:0/2384085953 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f500c15e9c0 0x7f500c0b4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:29.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.285+0000 7f50110bf640 1 --2- 172.21.15.105:0/2384085953 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f500c0b3850 0x7f500c149d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:29.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.285+0000 7f50118c0640 1 -- 172.21.15.105:0/2384085953 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f500c0b3850 msgr2=0x7f500c149d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:29.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.285+0000 7f50118c0640 1 --2- 172.21.15.105:0/2384085953 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f500c0b3850 0x7f500c149d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:29.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.285+0000 7f50118c0640 1 -- 172.21.15.105:0/2384085953 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f500c15e9c0 msgr2=0x7f500c0b4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:29.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.285+0000 7f50118c0640 1 --2- 172.21.15.105:0/2384085953 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f500c15e9c0 0x7f500c0b4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:29.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.285+0000 7f50118c0640 1 -- 172.21.15.105:0/2384085953 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f500c14a290 con 0x7f500c0b19d0 2024-04-15T15:21:29.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.285+0000 7f50120c1640 1 --2- 172.21.15.105:0/2384085953 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f500c15e9c0 0x7f500c0b4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:21:29.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.285+0000 7f50110bf640 1 --2- 172.21.15.105:0/2384085953 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f500c0b3850 0x7f500c149d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:21:29.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.285+0000 7f50118c0640 1 --2- 172.21.15.105:0/2384085953 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f500c0b19d0 0x7f500c0cd280 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f5008037040 tx=0x7f5008004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:29.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.285+0000 7f50027fc640 1 -- 172.21.15.105:0/2384085953 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5008039050 con 0x7f500c0b19d0 2024-04-15T15:21:29.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.285+0000 7f50027fc640 1 -- 172.21.15.105:0/2384085953 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5008045070 con 0x7f500c0b19d0 2024-04-15T15:21:29.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.285+0000 7f5013b24640 1 -- 172.21.15.105:0/2384085953 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f500c0cd9d0 con 0x7f500c0b19d0 2024-04-15T15:21:29.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.285+0000 7f50027fc640 1 -- 172.21.15.105:0/2384085953 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500803e850 con 0x7f500c0b19d0 2024-04-15T15:21:29.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.285+0000 7f5013b24640 1 -- 172.21.15.105:0/2384085953 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f500c0cdf90 con 0x7f500c0b19d0 2024-04-15T15:21:29.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.286+0000 7f5013b24640 1 -- 172.21.15.105:0/2384085953 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f500c0b2020 con 0x7f500c0b19d0 2024-04-15T15:21:29.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.286+0000 7f50027fc640 1 -- 172.21.15.105:0/2384085953 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f500803e9f0 con 0x7f500c0b19d0 2024-04-15T15:21:29.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.287+0000 7f50027fc640 1 --2- 172.21.15.105:0/2384085953 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f4ff006c9b0 0x7f4ff006ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:29.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.287+0000 7f50027fc640 1 -- 172.21.15.105:0/2384085953 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5029+0+0 (secure 0 0 0) 0x7f50080adfd0 con 0x7f500c0b19d0 2024-04-15T15:21:29.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.287+0000 7f50110bf640 1 --2- 172.21.15.105:0/2384085953 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f4ff006c9b0 0x7f4ff006ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:29.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.287+0000 7f50027fc640 1 --2- 172.21.15.105:0/2384085953 >> [v2:172.21.15.105:6808/3798408699,v1:172.21.15.105:6809/3798408699] conn(0x7f4ff0072910 0x7f4ff0074d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:29.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.287+0000 7f50027fc640 1 -- 172.21.15.105:0/2384085953 --> [v2:172.21.15.105:6808/3798408699,v1:172.21.15.105:6809/3798408699] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f50080091a0 con 0x7f4ff0072910 2024-04-15T15:21:29.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.287+0000 7f50027fc640 1 -- 172.21.15.105:0/2384085953 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f50080b5050 con 0x7f500c0b19d0 2024-04-15T15:21:29.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.287+0000 7f50120c1640 1 --2- 172.21.15.105:0/2384085953 >> [v2:172.21.15.105:6808/3798408699,v1:172.21.15.105:6809/3798408699] conn(0x7f4ff0072910 0x7f4ff0074d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:29.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.287+0000 7f50110bf640 1 --2- 172.21.15.105:0/2384085953 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f4ff006c9b0 0x7f4ff006ee70 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f500c1621b0 tx=0x7f4ffc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:29.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.287+0000 7f50120c1640 1 --2- 172.21.15.105:0/2384085953 >> [v2:172.21.15.105:6808/3798408699,v1:172.21.15.105:6809/3798408699] conn(0x7f4ff0072910 0x7f4ff0074d30 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:29.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.288+0000 7f50027fc640 1 -- 172.21.15.105:0/2384085953 <== osd.2 v2:172.21.15.105:6808/3798408699 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f50080091a0 con 0x7f4ff0072910 2024-04-15T15:21:29.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.301+0000 7f5013b24640 1 -- 172.21.15.105:0/2384085953 --> [v2:172.21.15.105:6808/3798408699,v1:172.21.15.105:6809/3798408699] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f500c068a40 con 0x7f4ff0072910 2024-04-15T15:21:29.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.301+0000 7f50027fc640 1 -- 172.21.15.105:0/2384085953 <== osd.2 v2:172.21.15.105:6808/3798408699 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f500c068a40 con 0x7f4ff0072910 2024-04-15T15:21:29.303 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-04-15T15:21:29.304 INFO:teuthology.orchestra.run.smithi105.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-15T15:21:29.304 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-04-15T15:21:29.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.302+0000 7f5013b24640 1 -- 172.21.15.105:0/2384085953 >> [v2:172.21.15.105:6808/3798408699,v1:172.21.15.105:6809/3798408699] conn(0x7f4ff0072910 msgr2=0x7f4ff0074d30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:29.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.302+0000 7f5013b24640 1 --2- 172.21.15.105:0/2384085953 >> [v2:172.21.15.105:6808/3798408699,v1:172.21.15.105:6809/3798408699] conn(0x7f4ff0072910 0x7f4ff0074d30 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:29.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.302+0000 7f5013b24640 1 -- 172.21.15.105:0/2384085953 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f4ff006c9b0 msgr2=0x7f4ff006ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:29.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.302+0000 7f5013b24640 1 --2- 172.21.15.105:0/2384085953 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f4ff006c9b0 0x7f4ff006ee70 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f500c1621b0 tx=0x7f4ffc009040 comp rx=0 tx=0).stop 2024-04-15T15:21:29.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.302+0000 7f5013b24640 1 -- 172.21.15.105:0/2384085953 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f500c0b19d0 msgr2=0x7f500c0cd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:29.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.302+0000 7f5013b24640 1 --2- 172.21.15.105:0/2384085953 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f500c0b19d0 0x7f500c0cd280 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f5008037040 tx=0x7f5008004a40 comp rx=0 tx=0).stop 2024-04-15T15:21:29.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.302+0000 7f50120c1640 1 -- 172.21.15.105:0/2384085953 reap_dead start 2024-04-15T15:21:29.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.302+0000 7f5013b24640 1 -- 172.21.15.105:0/2384085953 shutdown_connections 2024-04-15T15:21:29.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.302+0000 7f5013b24640 1 -- 172.21.15.105:0/2384085953 >> 172.21.15.105:0/2384085953 conn(0x7f500c09aa40 msgr2=0x7f500c0ae590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:29.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.302+0000 7f5013b24640 1 -- 172.21.15.105:0/2384085953 shutdown_connections 2024-04-15T15:21:29.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.302+0000 7f5013b24640 1 -- 172.21.15.105:0/2384085953 wait complete. 2024-04-15T15:21:29.314 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-15T15:21:29.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.416+0000 7f4506b22640 1 Processor -- start 2024-04-15T15:21:29.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.416+0000 7f4506b22640 1 -- start start 2024-04-15T15:21:29.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.417+0000 7f4506b22640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f45001554d0 0x7f45000afaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:29.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.417+0000 7f4506b22640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f45000b5580 0x7f45000cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:29.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.417+0000 7f4506b22640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45000b5a30 con 0x7f45000cfc50 2024-04-15T15:21:29.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.417+0000 7f4506b22640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f45000cf280 con 0x7f45000b5580 2024-04-15T15:21:29.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.417+0000 7f4506b22640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f450015bf70 con 0x7f45001554d0 2024-04-15T15:21:29.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.417+0000 7f44fffff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f45000b5580 0x7f45000cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:29.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.417+0000 7f45050bf640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f45000cfc50 0x7f45000b3ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:45264/0 (socket says 172.21.15.105:45264) 2024-04-15T15:21:29.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.417+0000 7f45050bf640 1 -- 172.21.15.105:0/1123585131 learned_addr learned my addr 172.21.15.105:0/1123585131 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:21:29.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.417+0000 7f45048be640 1 --2- 172.21.15.105:0/1123585131 >> v2:172.21.15.154:3300/0 conn(0x7f45001554d0 0x7f45000afaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:29.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.417+0000 7f44fffff640 1 -- 172.21.15.105:0/1123585131 >> v2:172.21.15.154:3300/0 conn(0x7f45001554d0 msgr2=0x7f45000afaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:29.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.417+0000 7f44fffff640 1 --2- 172.21.15.105:0/1123585131 >> v2:172.21.15.154:3300/0 conn(0x7f45001554d0 0x7f45000afaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:29.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.417+0000 7f44fffff640 1 -- 172.21.15.105:0/1123585131 >> v1:172.21.15.105:6789/0 conn(0x7f45000cfc50 legacy=0x7f45000b3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:29.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.417+0000 7f44fffff640 1 -- 172.21.15.105:0/1123585131 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45000cec30 con 0x7f45000b5580 2024-04-15T15:21:29.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.417+0000 7f44ff7fe640 1 -- 172.21.15.105:0/1123585131 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2152818094 0 0) 0x7f45000b5a30 con 0x7f45000cfc50 2024-04-15T15:21:29.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.417+0000 7f45048be640 1 --2- 172.21.15.105:0/1123585131 >> v2:172.21.15.154:3300/0 conn(0x7f45001554d0 0x7f45000afaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:21:29.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.418+0000 7f44fffff640 1 --2- 172.21.15.105:0/1123585131 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f45000b5580 0x7f45000cf710 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f4500073300 tx=0x7f44ec02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c79f1c741fa7009 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:29.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.418+0000 7f44ff7fe640 1 -- 172.21.15.105:0/1123585131 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44ec035020 con 0x7f45000b5580 2024-04-15T15:21:29.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.418+0000 7f44ff7fe640 1 -- 172.21.15.105:0/1123585131 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44ec031830 con 0x7f45000b5580 2024-04-15T15:21:29.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.418+0000 7f44ff7fe640 1 -- 172.21.15.105:0/1123585131 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44ec031c00 con 0x7f45000b5580 2024-04-15T15:21:29.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.419+0000 7f4506b22640 1 -- 172.21.15.105:0/1123585131 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f45000b5580 msgr2=0x7f45000cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:29.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.419+0000 7f4506b22640 1 --2- 172.21.15.105:0/1123585131 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f45000b5580 0x7f45000cf710 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f4500073300 tx=0x7f44ec02b8d0 comp rx=0 tx=0).stop 2024-04-15T15:21:29.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.419+0000 7f4506b22640 1 -- 172.21.15.105:0/1123585131 shutdown_connections 2024-04-15T15:21:29.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.419+0000 7f4506b22640 1 --2- 172.21.15.105:0/1123585131 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f45000b5580 0x7f45000cf710 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:29.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.419+0000 7f4506b22640 1 --2- 172.21.15.105:0/1123585131 >> v2:172.21.15.154:3300/0 conn(0x7f45001554d0 0x7f45000afaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:29.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.419+0000 7f4506b22640 1 -- 172.21.15.105:0/1123585131 >> 172.21.15.105:0/1123585131 conn(0x7f4500067d50 msgr2=0x7f450006b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:29.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.419+0000 7f4506b22640 1 -- 172.21.15.105:0/1123585131 shutdown_connections 2024-04-15T15:21:29.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.419+0000 7f4506b22640 1 -- 172.21.15.105:0/1123585131 wait complete. 2024-04-15T15:21:29.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.420+0000 7f4506b22640 1 Processor -- start 2024-04-15T15:21:29.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.420+0000 7f4506b22640 1 -- start start 2024-04-15T15:21:29.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.421+0000 7f4506b22640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f45000b5580 0x7f4500146490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:29.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.421+0000 7f4506b22640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f45000cfc50 0x7f45000af380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:29.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.421+0000 7f4506b22640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45001554d0 0x7f450014abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:29.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.421+0000 7f4506b22640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f45000b41c0 con 0x7f45000cfc50 2024-04-15T15:21:29.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.421+0000 7f4506b22640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f450015c250 con 0x7f45001554d0 2024-04-15T15:21:29.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.421+0000 7f4506b22640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f45000b4340 con 0x7f45000b5580 2024-04-15T15:21:29.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.421+0000 7f44fffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f45000cfc50 0x7f45000af380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:29.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.421+0000 7f44fffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f45000cfc50 0x7f45000af380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47280/0 (socket says 172.21.15.105:47280) 2024-04-15T15:21:29.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.421+0000 7f44fffff640 1 -- 172.21.15.105:0/2935074132 learned_addr learned my addr 172.21.15.105:0/2935074132 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:29.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.421+0000 7f45048be640 1 --2- 172.21.15.105:0/2935074132 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f45000b5580 0x7f4500146490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:29.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.421+0000 7f45050bf640 1 --2- 172.21.15.105:0/2935074132 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45001554d0 0x7f450014abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:29.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.421+0000 7f44fffff640 1 -- 172.21.15.105:0/2935074132 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f45000b5580 msgr2=0x7f4500146490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:29.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.421+0000 7f44fffff640 1 --2- 172.21.15.105:0/2935074132 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f45000b5580 0x7f4500146490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:29.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.421+0000 7f44fffff640 1 -- 172.21.15.105:0/2935074132 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45001554d0 msgr2=0x7f450014abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:29.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.421+0000 7f44fffff640 1 --2- 172.21.15.105:0/2935074132 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45001554d0 0x7f450014abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:29.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.421+0000 7f44fffff640 1 -- 172.21.15.105:0/2935074132 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f450014f510 con 0x7f45000cfc50 2024-04-15T15:21:29.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.421+0000 7f44fffff640 1 --2- 172.21.15.105:0/2935074132 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f45000cfc50 0x7f45000af380 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f44ec02bee0 tx=0x7f44ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:29.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.422+0000 7f44fd7fa640 1 -- 172.21.15.105:0/2935074132 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44ec041070 con 0x7f45000cfc50 2024-04-15T15:21:29.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.422+0000 7f44fd7fa640 1 -- 172.21.15.105:0/2935074132 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44ec03a950 con 0x7f45000cfc50 2024-04-15T15:21:29.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.422+0000 7f4506b22640 1 -- 172.21.15.105:0/2935074132 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f450014f780 con 0x7f45000cfc50 2024-04-15T15:21:29.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.422+0000 7f44fd7fa640 1 -- 172.21.15.105:0/2935074132 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44ec035040 con 0x7f45000cfc50 2024-04-15T15:21:29.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.422+0000 7f4506b22640 1 -- 172.21.15.105:0/2935074132 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f450014fd40 con 0x7f45000cfc50 2024-04-15T15:21:29.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.423+0000 7f4506b22640 1 -- 172.21.15.105:0/2935074132 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4500068a40 con 0x7f45000cfc50 2024-04-15T15:21:29.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.423+0000 7f44fd7fa640 1 -- 172.21.15.105:0/2935074132 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f44ec038020 con 0x7f45000cfc50 2024-04-15T15:21:29.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.424+0000 7f44fd7fa640 1 --2- 172.21.15.105:0/2935074132 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f44e406c980 0x7f44e406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:29.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.424+0000 7f44fd7fa640 1 -- 172.21.15.105:0/2935074132 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5029+0+0 (secure 0 0 0) 0x7f44ec0ad310 con 0x7f45000cfc50 2024-04-15T15:21:29.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.424+0000 7f45048be640 1 --2- 172.21.15.105:0/2935074132 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f44e406c980 0x7f44e406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:29.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.424+0000 7f45048be640 1 --2- 172.21.15.105:0/2935074132 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f44e406c980 0x7f44e406ee40 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f44f400b840 tx=0x7f44f4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:29.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.428+0000 7f44fd7fa640 1 -- 172.21.15.105:0/2935074132 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f44ec07ba80 con 0x7f45000cfc50 2024-04-15T15:21:29.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.604+0000 7f4506b22640 1 -- 172.21.15.105:0/2935074132 --> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f4500150620 con 0x7f44e406c980 2024-04-15T15:21:29.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.605+0000 7f44fd7fa640 1 -- 172.21.15.105:0/2935074132 <== mgr.4107 v2:172.21.15.105:6832/3506549322 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f4500150620 con 0x7f44e406c980 2024-04-15T15:21:29.607 INFO:teuthology.orchestra.run.smithi105.stderr:instructed osd(s) 2 to deep-scrub 2024-04-15T15:21:29.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.607+0000 7f4506b22640 1 -- 172.21.15.105:0/2935074132 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f44e406c980 msgr2=0x7f44e406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:29.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.607+0000 7f4506b22640 1 --2- 172.21.15.105:0/2935074132 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f44e406c980 0x7f44e406ee40 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f44f400b840 tx=0x7f44f4008040 comp rx=0 tx=0).stop 2024-04-15T15:21:29.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.607+0000 7f4506b22640 1 -- 172.21.15.105:0/2935074132 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f45000cfc50 msgr2=0x7f45000af380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:29.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.607+0000 7f4506b22640 1 --2- 172.21.15.105:0/2935074132 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f45000cfc50 0x7f45000af380 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f44ec02bee0 tx=0x7f44ec0046c0 comp rx=0 tx=0).stop 2024-04-15T15:21:29.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.607+0000 7f4506b22640 1 -- 172.21.15.105:0/2935074132 shutdown_connections 2024-04-15T15:21:29.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.607+0000 7f4506b22640 1 --2- 172.21.15.105:0/2935074132 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f44e406c980 0x7f44e406ee40 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:29.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.607+0000 7f4506b22640 1 --2- 172.21.15.105:0/2935074132 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45001554d0 0x7f450014abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:29.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.607+0000 7f4506b22640 1 --2- 172.21.15.105:0/2935074132 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f45000cfc50 0x7f45000af380 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:29.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.607+0000 7f4506b22640 1 --2- 172.21.15.105:0/2935074132 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f45000b5580 0x7f4500146490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:29.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.607+0000 7f4506b22640 1 -- 172.21.15.105:0/2935074132 >> 172.21.15.105:0/2935074132 conn(0x7f4500067d50 msgr2=0x7f450006b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:29.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.607+0000 7f4506b22640 1 -- 172.21.15.105:0/2935074132 shutdown_connections 2024-04-15T15:21:29.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.607+0000 7f4506b22640 1 -- 172.21.15.105:0/2935074132 wait complete. 2024-04-15T15:21:29.619 INFO:tasks.ceph:Scrubbing osd.3 2024-04-15T15:21:29.620 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-15T15:21:29.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.726+0000 7f69de0a5640 1 Processor -- start 2024-04-15T15:21:29.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.726+0000 7f69de0a5640 1 -- start start 2024-04-15T15:21:29.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.727+0000 7f69de0a5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f69d80d0820 0x7f69d809b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:29.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.727+0000 7f69de0a5640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f69d815e250 0x7f69d815e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:29.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.727+0000 7f69de0a5640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69d80d0f90 con 0x7f69d809aa40 2024-04-15T15:21:29.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.727+0000 7f69de0a5640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f69d809be40 con 0x7f69d815e250 2024-04-15T15:21:29.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.727+0000 7f69de0a5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f69d80d1270 con 0x7f69d80d0820 2024-04-15T15:21:29.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.727+0000 7f69d7fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f69d815e250 0x7f69d815e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:29.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.727+0000 7f69d7fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f69d815e250 0x7f69d815e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47282/0 (socket says 172.21.15.105:47282) 2024-04-15T15:21:29.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.727+0000 7f69d7fff640 1 -- 172.21.15.105:0/3318023923 learned_addr learned my addr 172.21.15.105:0/3318023923 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:29.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.728+0000 7f69d6ffd640 1 --2- 172.21.15.105:0/3318023923 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f69d80d0820 0x7f69d809b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:29.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.728+0000 7f69d7fff640 1 -- 172.21.15.105:0/3318023923 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f69d80d0820 msgr2=0x7f69d809b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:29.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.728+0000 7f69d7fff640 1 --2- 172.21.15.105:0/3318023923 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f69d80d0820 0x7f69d809b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:29.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.728+0000 7f69d7fff640 1 -- 172.21.15.105:0/3318023923 >> v1:172.21.15.154:6789/0 conn(0x7f69d809aa40 legacy=0x7f69d809ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:29.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.728+0000 7f69d7fff640 1 -- 172.21.15.105:0/3318023923 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69d80d1550 con 0x7f69d815e250 2024-04-15T15:21:29.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.728+0000 7f69d7fff640 1 --2- 172.21.15.105:0/3318023923 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69d815e250 0x7f69d815e6e0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f69d0009e50 tx=0x7f69d002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f288ab368b222c2b server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:29.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.729+0000 7f69d67fc640 1 -- 172.21.15.105:0/3318023923 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d002ca70 con 0x7f69d815e250 2024-04-15T15:21:29.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.729+0000 7f69d67fc640 1 -- 172.21.15.105:0/3318023923 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69d002cc10 con 0x7f69d815e250 2024-04-15T15:21:29.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.729+0000 7f69d67fc640 1 -- 172.21.15.105:0/3318023923 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d00343f0 con 0x7f69d815e250 2024-04-15T15:21:29.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.729+0000 7f69de0a5640 1 -- 172.21.15.105:0/3318023923 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69d815e250 msgr2=0x7f69d815e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:29.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.729+0000 7f69de0a5640 1 --2- 172.21.15.105:0/3318023923 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69d815e250 0x7f69d815e6e0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f69d0009e50 tx=0x7f69d002bed0 comp rx=0 tx=0).stop 2024-04-15T15:21:29.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.729+0000 7f69de0a5640 1 -- 172.21.15.105:0/3318023923 shutdown_connections 2024-04-15T15:21:29.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.729+0000 7f69de0a5640 1 --2- 172.21.15.105:0/3318023923 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69d815e250 0x7f69d815e6e0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:29.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.729+0000 7f69de0a5640 1 --2- 172.21.15.105:0/3318023923 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f69d80d0820 0x7f69d809b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:29.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.729+0000 7f69de0a5640 1 -- 172.21.15.105:0/3318023923 >> 172.21.15.105:0/3318023923 conn(0x7f69d80b40d0 msgr2=0x7f69d80b44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:29.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.729+0000 7f69de0a5640 1 -- 172.21.15.105:0/3318023923 shutdown_connections 2024-04-15T15:21:29.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.729+0000 7f69de0a5640 1 -- 172.21.15.105:0/3318023923 wait complete. 2024-04-15T15:21:29.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.730+0000 7f69de0a5640 1 Processor -- start 2024-04-15T15:21:29.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.731+0000 7f69de0a5640 1 -- start start 2024-04-15T15:21:29.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.731+0000 7f69de0a5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f69d809aa40 0x7f69d815d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:29.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.731+0000 7f69de0a5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69d80d0820 0x7f69d80c7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:29.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.731+0000 7f69de0a5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69d815e250 0x7f69d80cda40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:29.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.731+0000 7f69de0a5640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f69d80d0cb0 con 0x7f69d80d0820 2024-04-15T15:21:29.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.731+0000 7f69de0a5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f69d80d1b10 con 0x7f69d815e250 2024-04-15T15:21:29.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.731+0000 7f69de0a5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f69d80b3e10 con 0x7f69d809aa40 2024-04-15T15:21:29.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.731+0000 7f69d6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69d80d0820 0x7f69d80c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:29.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.731+0000 7f69d6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69d80d0820 0x7f69d80c7860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47290/0 (socket says 172.21.15.105:47290) 2024-04-15T15:21:29.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.731+0000 7f69d6ffd640 1 -- 172.21.15.105:0/118462266 learned_addr learned my addr 172.21.15.105:0/118462266 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:29.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.732+0000 7f69d77fe640 1 --2- 172.21.15.105:0/118462266 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f69d809aa40 0x7f69d815d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:29.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.732+0000 7f69d7fff640 1 --2- 172.21.15.105:0/118462266 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69d815e250 0x7f69d80cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:29.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.732+0000 7f69d6ffd640 1 -- 172.21.15.105:0/118462266 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f69d809aa40 msgr2=0x7f69d815d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:29.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.732+0000 7f69d6ffd640 1 --2- 172.21.15.105:0/118462266 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f69d809aa40 0x7f69d815d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:29.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.732+0000 7f69d6ffd640 1 -- 172.21.15.105:0/118462266 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69d815e250 msgr2=0x7f69d80cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:29.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.732+0000 7f69d6ffd640 1 --2- 172.21.15.105:0/118462266 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69d815e250 0x7f69d80cda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:29.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.732+0000 7f69d6ffd640 1 -- 172.21.15.105:0/118462266 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69d80c81c0 con 0x7f69d80d0820 2024-04-15T15:21:29.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.732+0000 7f69d6ffd640 1 --2- 172.21.15.105:0/118462266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69d80d0820 0x7f69d80c7860 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f69cc00c7f0 tx=0x7f69cc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:29.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.732+0000 7f69bffff640 1 -- 172.21.15.105:0/118462266 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69cc015070 con 0x7f69d80d0820 2024-04-15T15:21:29.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.732+0000 7f69bffff640 1 -- 172.21.15.105:0/118462266 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69cc004220 con 0x7f69d80d0820 2024-04-15T15:21:29.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.732+0000 7f69de0a5640 1 -- 172.21.15.105:0/118462266 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69d80c84a0 con 0x7f69d80d0820 2024-04-15T15:21:29.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.732+0000 7f69bffff640 1 -- 172.21.15.105:0/118462266 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69cc007620 con 0x7f69d80d0820 2024-04-15T15:21:29.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.732+0000 7f69de0a5640 1 -- 172.21.15.105:0/118462266 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69d80c8a60 con 0x7f69d80d0820 2024-04-15T15:21:29.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.733+0000 7f69de0a5640 1 -- 172.21.15.105:0/118462266 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f69d8069150 con 0x7f69d80d0820 2024-04-15T15:21:29.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.734+0000 7f69bffff640 1 -- 172.21.15.105:0/118462266 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f69cc0079d0 con 0x7f69d80d0820 2024-04-15T15:21:29.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.734+0000 7f69bffff640 1 --2- 172.21.15.105:0/118462266 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f69c406c980 0x7f69c406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:29.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.734+0000 7f69bffff640 1 -- 172.21.15.105:0/118462266 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5029+0+0 (secure 0 0 0) 0x7f69cc08bfa0 con 0x7f69d80d0820 2024-04-15T15:21:29.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.734+0000 7f69d77fe640 1 --2- 172.21.15.105:0/118462266 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f69c406c980 0x7f69c406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:29.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.734+0000 7f69bffff640 1 --2- 172.21.15.105:0/118462266 >> [v2:172.21.15.105:6817/1920720186,v1:172.21.15.105:6819/1920720186] conn(0x7f69c40728e0 0x7f69c4074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:29.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.734+0000 7f69bffff640 1 -- 172.21.15.105:0/118462266 --> [v2:172.21.15.105:6817/1920720186,v1:172.21.15.105:6819/1920720186] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f69cc010e50 con 0x7f69c40728e0 2024-04-15T15:21:29.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.734+0000 7f69bffff640 1 -- 172.21.15.105:0/118462266 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f69cc08c480 con 0x7f69d80d0820 2024-04-15T15:21:29.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.734+0000 7f69d7fff640 1 --2- 172.21.15.105:0/118462266 >> [v2:172.21.15.105:6817/1920720186,v1:172.21.15.105:6819/1920720186] conn(0x7f69c40728e0 0x7f69c4074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:29.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.735+0000 7f69d77fe640 1 --2- 172.21.15.105:0/118462266 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f69c406c980 0x7f69c406ee40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f69c8006fd0 tx=0x7f69c8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:29.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.735+0000 7f69d7fff640 1 --2- 172.21.15.105:0/118462266 >> [v2:172.21.15.105:6817/1920720186,v1:172.21.15.105:6819/1920720186] conn(0x7f69c40728e0 0x7f69c4074d00 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:29.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.735+0000 7f69bffff640 1 -- 172.21.15.105:0/118462266 <== osd.3 v2:172.21.15.105:6817/1920720186 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f69cc010e50 con 0x7f69c40728e0 2024-04-15T15:21:29.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.748+0000 7f69de0a5640 1 -- 172.21.15.105:0/118462266 --> [v2:172.21.15.105:6817/1920720186,v1:172.21.15.105:6819/1920720186] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f69d80b8240 con 0x7f69c40728e0 2024-04-15T15:21:29.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.749+0000 7f69bffff640 1 -- 172.21.15.105:0/118462266 <== osd.3 v2:172.21.15.105:6817/1920720186 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f69d80b8240 con 0x7f69c40728e0 2024-04-15T15:21:29.751 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-04-15T15:21:29.751 INFO:teuthology.orchestra.run.smithi105.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-15T15:21:29.751 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-04-15T15:21:29.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.749+0000 7f69de0a5640 1 -- 172.21.15.105:0/118462266 >> [v2:172.21.15.105:6817/1920720186,v1:172.21.15.105:6819/1920720186] conn(0x7f69c40728e0 msgr2=0x7f69c4074d00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:29.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.749+0000 7f69de0a5640 1 --2- 172.21.15.105:0/118462266 >> [v2:172.21.15.105:6817/1920720186,v1:172.21.15.105:6819/1920720186] conn(0x7f69c40728e0 0x7f69c4074d00 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:29.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.749+0000 7f69de0a5640 1 -- 172.21.15.105:0/118462266 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f69c406c980 msgr2=0x7f69c406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:29.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.749+0000 7f69de0a5640 1 --2- 172.21.15.105:0/118462266 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f69c406c980 0x7f69c406ee40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f69c8006fd0 tx=0x7f69c8008040 comp rx=0 tx=0).stop 2024-04-15T15:21:29.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.749+0000 7f69de0a5640 1 -- 172.21.15.105:0/118462266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69d80d0820 msgr2=0x7f69d80c7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:29.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.749+0000 7f69de0a5640 1 --2- 172.21.15.105:0/118462266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69d80d0820 0x7f69d80c7860 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f69cc00c7f0 tx=0x7f69cc00ccb0 comp rx=0 tx=0).stop 2024-04-15T15:21:29.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.749+0000 7f69d7fff640 1 -- 172.21.15.105:0/118462266 reap_dead start 2024-04-15T15:21:29.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.749+0000 7f69de0a5640 1 -- 172.21.15.105:0/118462266 shutdown_connections 2024-04-15T15:21:29.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.749+0000 7f69de0a5640 1 -- 172.21.15.105:0/118462266 >> 172.21.15.105:0/118462266 conn(0x7f69d80b40d0 msgr2=0x7f69d809b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:29.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.749+0000 7f69de0a5640 1 -- 172.21.15.105:0/118462266 shutdown_connections 2024-04-15T15:21:29.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.749+0000 7f69de0a5640 1 -- 172.21.15.105:0/118462266 wait complete. 2024-04-15T15:21:29.761 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-15T15:21:29.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.863+0000 7fefdfaa5640 1 Processor -- start 2024-04-15T15:21:29.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.863+0000 7fefdfaa5640 1 -- start start 2024-04-15T15:21:29.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.864+0000 7fefdfaa5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fefd80af6b0 0x7fefd80725f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:29.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.864+0000 7fefdfaa5640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefd80afcd0 con 0x7fefd8147730 2024-04-15T15:21:29.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.864+0000 7fefdfaa5640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefd806e130 con 0x7fefd8072160 2024-04-15T15:21:29.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.864+0000 7fefdfaa5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fefd80affb0 con 0x7fefd80af6b0 2024-04-15T15:21:29.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.864+0000 7fefdd841640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fefd8147730 0x7fefd80b3940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:45270/0 (socket says 172.21.15.105:45270) 2024-04-15T15:21:29.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.864+0000 7fefdd841640 1 -- 172.21.15.105:0/519375644 learned_addr learned my addr 172.21.15.105:0/519375644 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:21:29.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.864+0000 7fefde042640 1 --2- 172.21.15.105:0/519375644 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fefd80af6b0 0x7fefd80725f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:29.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.864+0000 7fefdc83f640 1 -- 172.21.15.105:0/519375644 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3402822456 0 0) 0x7fefd80afcd0 con 0x7fefd8147730 2024-04-15T15:21:29.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.864+0000 7fefdc83f640 1 -- 172.21.15.105:0/519375644 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fefc4003520 con 0x7fefd8147730 2024-04-15T15:21:29.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.864+0000 7fefdc83f640 1 -- 172.21.15.105:0/519375644 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2342142555 0 0) 0x7fefd806e130 con 0x7fefd8072160 2024-04-15T15:21:29.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.864+0000 7fefdc83f640 1 -- 172.21.15.105:0/519375644 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fefd80afcd0 con 0x7fefd8072160 2024-04-15T15:21:29.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.864+0000 7fefde042640 1 -- 172.21.15.105:0/519375644 >> v1:172.21.15.154:6789/0 conn(0x7fefd8072160 legacy=0x7fefd80aefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:29.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.864+0000 7fefde042640 1 -- 172.21.15.105:0/519375644 >> v1:172.21.15.105:6789/0 conn(0x7fefd8147730 legacy=0x7fefd80b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:29.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.864+0000 7fefdc83f640 1 -- 172.21.15.105:0/519375644 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1395646523 0 0) 0x7fefc4003520 con 0x7fefd8147730 2024-04-15T15:21:29.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.864+0000 7fefde042640 1 -- 172.21.15.105:0/519375644 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefd806e410 con 0x7fefd80af6b0 2024-04-15T15:21:29.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.865+0000 7fefde042640 1 --2- 172.21.15.105:0/519375644 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fefd80af6b0 0x7fefd80725f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fefd0009df0 tx=0x7fefd002bd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70e1a8e31aa8b6a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:29.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.865+0000 7fefdc83f640 1 -- 172.21.15.105:0/519375644 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd0039070 con 0x7fefd80af6b0 2024-04-15T15:21:29.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.865+0000 7fefdc83f640 1 -- 172.21.15.105:0/519375644 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefd002c880 con 0x7fefd80af6b0 2024-04-15T15:21:29.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.865+0000 7fefdc83f640 1 -- 172.21.15.105:0/519375644 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd4001670 con 0x7fefd80af6b0 2024-04-15T15:21:29.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.866+0000 7fefdfaa5640 1 -- 172.21.15.105:0/519375644 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fefd80af6b0 msgr2=0x7fefd80725f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:29.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.866+0000 7fefdfaa5640 1 --2- 172.21.15.105:0/519375644 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fefd80af6b0 0x7fefd80725f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fefd0009df0 tx=0x7fefd002bd50 comp rx=0 tx=0).stop 2024-04-15T15:21:29.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.866+0000 7fefdfaa5640 1 -- 172.21.15.105:0/519375644 shutdown_connections 2024-04-15T15:21:29.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.866+0000 7fefdfaa5640 1 --2- 172.21.15.105:0/519375644 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fefd80af6b0 0x7fefd80725f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:29.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.866+0000 7fefdfaa5640 1 -- 172.21.15.105:0/519375644 >> 172.21.15.105:0/519375644 conn(0x7fefd806b9c0 msgr2=0x7fefd806bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:29.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.866+0000 7fefdfaa5640 1 -- 172.21.15.105:0/519375644 shutdown_connections 2024-04-15T15:21:29.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.866+0000 7fefdfaa5640 1 -- 172.21.15.105:0/519375644 wait complete. 2024-04-15T15:21:29.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.867+0000 7fefdfaa5640 1 Processor -- start 2024-04-15T15:21:29.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.867+0000 7fefdfaa5640 1 -- start start 2024-04-15T15:21:29.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.868+0000 7fefdfaa5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fefd8072160 0x7fefd80cc320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:29.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.868+0000 7fefdfaa5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fefd80af6b0 0x7fefd80cfc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:29.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.868+0000 7fefdfaa5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefd8147730 0x7fefd80c58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:29.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.868+0000 7fefdfaa5640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fefd806ec10 con 0x7fefd8147730 2024-04-15T15:21:29.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.868+0000 7fefdfaa5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fefd8147b10 con 0x7fefd8072160 2024-04-15T15:21:29.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.868+0000 7fefdfaa5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fefd806ed90 con 0x7fefd80af6b0 2024-04-15T15:21:29.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.868+0000 7fefde042640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefd8147730 0x7fefd80c58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:29.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.868+0000 7fefde042640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefd8147730 0x7fefd80c58b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47292/0 (socket says 172.21.15.105:47292) 2024-04-15T15:21:29.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.868+0000 7fefde042640 1 -- 172.21.15.105:0/100601601 learned_addr learned my addr 172.21.15.105:0/100601601 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:29.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.868+0000 7fefdd841640 1 --2- 172.21.15.105:0/100601601 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fefd8072160 0x7fefd80cc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:29.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.868+0000 7fefdd040640 1 --2- 172.21.15.105:0/100601601 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fefd80af6b0 0x7fefd80cfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:29.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.868+0000 7fefde042640 1 -- 172.21.15.105:0/100601601 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fefd80af6b0 msgr2=0x7fefd80cfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:29.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.868+0000 7fefde042640 1 --2- 172.21.15.105:0/100601601 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fefd80af6b0 0x7fefd80cfc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:29.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.868+0000 7fefde042640 1 -- 172.21.15.105:0/100601601 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fefd8072160 msgr2=0x7fefd80cc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:29.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.868+0000 7fefde042640 1 --2- 172.21.15.105:0/100601601 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fefd8072160 0x7fefd80cc320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:29.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.868+0000 7fefde042640 1 -- 172.21.15.105:0/100601601 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefd8138fe0 con 0x7fefd8147730 2024-04-15T15:21:29.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.868+0000 7fefdd841640 1 --2- 172.21.15.105:0/100601601 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fefd8072160 0x7fefd80cc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:21:29.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.868+0000 7fefde042640 1 --2- 172.21.15.105:0/100601601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefd8147730 0x7fefd80c58b0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fefd0009f10 tx=0x7fefd0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:29.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.869+0000 7fefce7fc640 1 -- 172.21.15.105:0/100601601 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd0039050 con 0x7fefd8147730 2024-04-15T15:21:29.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.869+0000 7fefce7fc640 1 -- 172.21.15.105:0/100601601 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefd0045070 con 0x7fefd8147730 2024-04-15T15:21:29.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.869+0000 7fefce7fc640 1 -- 172.21.15.105:0/100601601 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd0009450 con 0x7fefd8147730 2024-04-15T15:21:29.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.869+0000 7fefdfaa5640 1 -- 172.21.15.105:0/100601601 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefd81392c0 con 0x7fefd8147730 2024-04-15T15:21:29.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.869+0000 7fefdfaa5640 1 -- 172.21.15.105:0/100601601 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefd8139880 con 0x7fefd8147730 2024-04-15T15:21:29.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.870+0000 7fefdfaa5640 1 -- 172.21.15.105:0/100601601 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefd8068a40 con 0x7fefd8147730 2024-04-15T15:21:29.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.870+0000 7fefce7fc640 1 -- 172.21.15.105:0/100601601 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fefd0004650 con 0x7fefd8147730 2024-04-15T15:21:29.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.871+0000 7fefce7fc640 1 --2- 172.21.15.105:0/100601601 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fefbc06c980 0x7fefbc06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:29.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.871+0000 7fefdd841640 1 --2- 172.21.15.105:0/100601601 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fefbc06c980 0x7fefbc06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:29.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.871+0000 7fefce7fc640 1 -- 172.21.15.105:0/100601601 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5029+0+0 (secure 0 0 0) 0x7fefd00ae1e0 con 0x7fefd8147730 2024-04-15T15:21:29.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.871+0000 7fefdd841640 1 --2- 172.21.15.105:0/100601601 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fefbc06c980 0x7fefbc06ee40 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fefd4001340 tx=0x7fefd4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:29.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:29.875+0000 7fefce7fc640 1 -- 172.21.15.105:0/100601601 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fefd007c830 con 0x7fefd8147730 2024-04-15T15:21:30.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.050+0000 7fefdfaa5640 1 -- 172.21.15.105:0/100601601 --> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7fefd806c080 con 0x7fefbc06c980 2024-04-15T15:21:30.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.051+0000 7fefce7fc640 1 -- 172.21.15.105:0/100601601 <== mgr.4107 v2:172.21.15.105:6832/3506549322 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fefd806c080 con 0x7fefbc06c980 2024-04-15T15:21:30.053 INFO:teuthology.orchestra.run.smithi105.stderr:instructed osd(s) 3 to deep-scrub 2024-04-15T15:21:30.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.053+0000 7fefdfaa5640 1 -- 172.21.15.105:0/100601601 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fefbc06c980 msgr2=0x7fefbc06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:30.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.053+0000 7fefdfaa5640 1 --2- 172.21.15.105:0/100601601 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fefbc06c980 0x7fefbc06ee40 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fefd4001340 tx=0x7fefd4009040 comp rx=0 tx=0).stop 2024-04-15T15:21:30.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.053+0000 7fefdfaa5640 1 -- 172.21.15.105:0/100601601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefd8147730 msgr2=0x7fefd80c58b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:30.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.053+0000 7fefdfaa5640 1 --2- 172.21.15.105:0/100601601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefd8147730 0x7fefd80c58b0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fefd0009f10 tx=0x7fefd0004a40 comp rx=0 tx=0).stop 2024-04-15T15:21:30.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.053+0000 7fefdfaa5640 1 -- 172.21.15.105:0/100601601 shutdown_connections 2024-04-15T15:21:30.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.053+0000 7fefdfaa5640 1 --2- 172.21.15.105:0/100601601 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fefd80af6b0 0x7fefd80cfc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:30.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.053+0000 7fefdfaa5640 1 --2- 172.21.15.105:0/100601601 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fefbc06c980 0x7fefbc06ee40 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:30.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.053+0000 7fefdfaa5640 1 --2- 172.21.15.105:0/100601601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefd8147730 0x7fefd80c58b0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:30.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.053+0000 7fefdfaa5640 1 --2- 172.21.15.105:0/100601601 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fefd8072160 0x7fefd80cc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:30.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.053+0000 7fefdfaa5640 1 -- 172.21.15.105:0/100601601 >> 172.21.15.105:0/100601601 conn(0x7fefd806b9c0 msgr2=0x7fefd80b3730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:30.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.053+0000 7fefdfaa5640 1 -- 172.21.15.105:0/100601601 shutdown_connections 2024-04-15T15:21:30.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.053+0000 7fefdfaa5640 1 -- 172.21.15.105:0/100601601 wait complete. 2024-04-15T15:21:30.065 INFO:tasks.ceph:Scrubbing osd.4 2024-04-15T15:21:30.065 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-15T15:21:30.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.171+0000 7f0952688640 1 Processor -- start 2024-04-15T15:21:30.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.171+0000 7f0952688640 1 -- start start 2024-04-15T15:21:30.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.172+0000 7f0952688640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f094c13e6f0 0x7f094c0cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:30.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.172+0000 7f0952688640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f094c06f6d0 0x7f094c15bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:30.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.172+0000 7f0952688640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f094c15f280 con 0x7f094c0d0060 2024-04-15T15:21:30.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.172+0000 7f0952688640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f094c15f560 con 0x7f094c06f6d0 2024-04-15T15:21:30.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.172+0000 7f0952688640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f094c15f840 con 0x7f094c13e6f0 2024-04-15T15:21:30.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.172+0000 7f0950c25640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f094c06f6d0 0x7f094c15bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:30.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.172+0000 7f0950c25640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f094c06f6d0 0x7f094c15bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47294/0 (socket says 172.21.15.105:47294) 2024-04-15T15:21:30.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.172+0000 7f0950c25640 1 -- 172.21.15.105:0/1689623148 learned_addr learned my addr 172.21.15.105:0/1689623148 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:30.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.172+0000 7f0950c25640 1 -- 172.21.15.105:0/1689623148 >> v2:172.21.15.154:3300/0 conn(0x7f094c13e6f0 msgr2=0x7f094c0cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:30.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.172+0000 7f094bfff640 1 --2- 172.21.15.105:0/1689623148 >> v2:172.21.15.154:3300/0 conn(0x7f094c13e6f0 0x7f094c0cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:30.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.172+0000 7f0950c25640 1 --2- 172.21.15.105:0/1689623148 >> v2:172.21.15.154:3300/0 conn(0x7f094c13e6f0 0x7f094c0cfb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:30.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.172+0000 7f0950c25640 1 -- 172.21.15.105:0/1689623148 >> v1:172.21.15.154:6789/0 conn(0x7f094c0d0060 legacy=0x7f094c06efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:30.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.172+0000 7f0950c25640 1 -- 172.21.15.105:0/1689623148 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f094c15fb20 con 0x7f094c06f6d0 2024-04-15T15:21:30.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.173+0000 7f0950c25640 1 --2- 172.21.15.105:0/1689623148 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f094c06f6d0 0x7f094c15bcd0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f094c0703c0 tx=0x7f094402e060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82a530bb4bb2189b server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:30.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.173+0000 7f094affd640 1 -- 172.21.15.105:0/1689623148 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094402ea00 con 0x7f094c06f6d0 2024-04-15T15:21:30.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.173+0000 7f094affd640 1 -- 172.21.15.105:0/1689623148 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f094402eba0 con 0x7f094c06f6d0 2024-04-15T15:21:30.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.173+0000 7f094affd640 1 -- 172.21.15.105:0/1689623148 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0944009600 con 0x7f094c06f6d0 2024-04-15T15:21:30.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.173+0000 7f0952688640 1 -- 172.21.15.105:0/1689623148 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f094c06f6d0 msgr2=0x7f094c15bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:30.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.173+0000 7f0952688640 1 --2- 172.21.15.105:0/1689623148 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f094c06f6d0 0x7f094c15bcd0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f094c0703c0 tx=0x7f094402e060 comp rx=0 tx=0).stop 2024-04-15T15:21:30.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.173+0000 7f0952688640 1 -- 172.21.15.105:0/1689623148 shutdown_connections 2024-04-15T15:21:30.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.173+0000 7f0952688640 1 --2- 172.21.15.105:0/1689623148 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f094c06f6d0 0x7f094c15bcd0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:30.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.173+0000 7f0952688640 1 --2- 172.21.15.105:0/1689623148 >> v2:172.21.15.154:3300/0 conn(0x7f094c13e6f0 0x7f094c0cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:30.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.173+0000 7f0952688640 1 -- 172.21.15.105:0/1689623148 >> 172.21.15.105:0/1689623148 conn(0x7f094c067e10 msgr2=0x7f094c0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:30.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.174+0000 7f0952688640 1 -- 172.21.15.105:0/1689623148 shutdown_connections 2024-04-15T15:21:30.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.174+0000 7f0952688640 1 -- 172.21.15.105:0/1689623148 wait complete. 2024-04-15T15:21:30.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.175+0000 7f0952688640 1 Processor -- start 2024-04-15T15:21:30.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.175+0000 7f0952688640 1 -- start start 2024-04-15T15:21:30.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.176+0000 7f0952688640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f094c06f6d0 0x7f094c1f0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:30.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.176+0000 7f0952688640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f094c0d0060 0x7f094c13b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:30.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.176+0000 7f0952688640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f094c13e6f0 0x7f094c15e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:30.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.176+0000 7f0952688640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f094c0cf7f0 con 0x7f094c06f6d0 2024-04-15T15:21:30.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.176+0000 7f0952688640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f094c0cf670 con 0x7f094c13e6f0 2024-04-15T15:21:30.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.176+0000 7f0952688640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f094c13ecd0 con 0x7f094c0d0060 2024-04-15T15:21:30.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.176+0000 7f094bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f094c06f6d0 0x7f094c1f0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:30.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.176+0000 7f094bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f094c06f6d0 0x7f094c1f0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47298/0 (socket says 172.21.15.105:47298) 2024-04-15T15:21:30.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.176+0000 7f094bfff640 1 -- 172.21.15.105:0/809861162 learned_addr learned my addr 172.21.15.105:0/809861162 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:30.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.176+0000 7f094bfff640 1 -- 172.21.15.105:0/809861162 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f094c0d0060 msgr2=0x7f094c13b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:30.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.176+0000 7f094bfff640 1 --2- 172.21.15.105:0/809861162 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f094c0d0060 0x7f094c13b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:30.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.176+0000 7f094bfff640 1 -- 172.21.15.105:0/809861162 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f094c13e6f0 msgr2=0x7f094c15e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:30.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.176+0000 7f094bfff640 1 --2- 172.21.15.105:0/809861162 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f094c13e6f0 0x7f094c15e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:30.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.176+0000 7f094bfff640 1 -- 172.21.15.105:0/809861162 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f094c15d270 con 0x7f094c06f6d0 2024-04-15T15:21:30.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.176+0000 7f094bfff640 1 --2- 172.21.15.105:0/809861162 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f094c06f6d0 0x7f094c1f0270 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f093c00c7f0 tx=0x7f093c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:30.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.176+0000 7f0948ff9640 1 -- 172.21.15.105:0/809861162 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f093c015070 con 0x7f094c06f6d0 2024-04-15T15:21:30.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.176+0000 7f0948ff9640 1 -- 172.21.15.105:0/809861162 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f093c004220 con 0x7f094c06f6d0 2024-04-15T15:21:30.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.176+0000 7f0948ff9640 1 -- 172.21.15.105:0/809861162 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f093c007620 con 0x7f094c06f6d0 2024-04-15T15:21:30.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.176+0000 7f0952688640 1 -- 172.21.15.105:0/809861162 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f094c15d550 con 0x7f094c06f6d0 2024-04-15T15:21:30.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.176+0000 7f0952688640 1 -- 172.21.15.105:0/809861162 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f094c15db10 con 0x7f094c06f6d0 2024-04-15T15:21:30.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.177+0000 7f0952688640 1 -- 172.21.15.105:0/809861162 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f094c13be30 con 0x7f094c06f6d0 2024-04-15T15:21:30.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.178+0000 7f0948ff9640 1 -- 172.21.15.105:0/809861162 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f093c0079d0 con 0x7f094c06f6d0 2024-04-15T15:21:30.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.178+0000 7f0948ff9640 1 --2- 172.21.15.105:0/809861162 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f093806c980 0x7f093806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:30.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.178+0000 7f0948ff9640 1 -- 172.21.15.105:0/809861162 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5029+0+0 (secure 0 0 0) 0x7f093c08bd50 con 0x7f094c06f6d0 2024-04-15T15:21:30.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.178+0000 7f094b7fe640 1 --2- 172.21.15.105:0/809861162 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f093806c980 0x7f093806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:30.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.178+0000 7f0948ff9640 1 --2- 172.21.15.105:0/809861162 >> [v2:172.21.15.154:6808/3516494456,v1:172.21.15.154:6810/3516494456] conn(0x7f09380728e0 0x7f0938074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:30.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.178+0000 7f0948ff9640 1 -- 172.21.15.105:0/809861162 --> [v2:172.21.15.154:6808/3516494456,v1:172.21.15.154:6810/3516494456] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f093c003e50 con 0x7f09380728e0 2024-04-15T15:21:30.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.178+0000 7f0948ff9640 1 -- 172.21.15.105:0/809861162 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f093c08e4a0 con 0x7f094c06f6d0 2024-04-15T15:21:30.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.179+0000 7f0950c25640 1 --2- 172.21.15.105:0/809861162 >> [v2:172.21.15.154:6808/3516494456,v1:172.21.15.154:6810/3516494456] conn(0x7f09380728e0 0x7f0938074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:30.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.179+0000 7f094b7fe640 1 --2- 172.21.15.105:0/809861162 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f093806c980 0x7f093806ee40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f094c15b460 tx=0x7f09400079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:30.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.179+0000 7f0950c25640 1 --2- 172.21.15.105:0/809861162 >> [v2:172.21.15.154:6808/3516494456,v1:172.21.15.154:6810/3516494456] conn(0x7f09380728e0 0x7f0938074d00 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:30.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.180+0000 7f0948ff9640 1 -- 172.21.15.105:0/809861162 <== osd.4 v2:172.21.15.154:6808/3516494456 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f093c003e50 con 0x7f09380728e0 2024-04-15T15:21:30.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.193+0000 7f0952688640 1 -- 172.21.15.105:0/809861162 --> [v2:172.21.15.154:6808/3516494456,v1:172.21.15.154:6810/3516494456] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f094c068a40 con 0x7f09380728e0 2024-04-15T15:21:30.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.193+0000 7f0948ff9640 1 -- 172.21.15.105:0/809861162 <== osd.4 v2:172.21.15.154:6808/3516494456 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f094c068a40 con 0x7f09380728e0 2024-04-15T15:21:30.195 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-04-15T15:21:30.195 INFO:teuthology.orchestra.run.smithi105.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-15T15:21:30.195 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-04-15T15:21:30.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.193+0000 7f0952688640 1 -- 172.21.15.105:0/809861162 >> [v2:172.21.15.154:6808/3516494456,v1:172.21.15.154:6810/3516494456] conn(0x7f09380728e0 msgr2=0x7f0938074d00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:30.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.193+0000 7f0952688640 1 --2- 172.21.15.105:0/809861162 >> [v2:172.21.15.154:6808/3516494456,v1:172.21.15.154:6810/3516494456] conn(0x7f09380728e0 0x7f0938074d00 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:30.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.193+0000 7f0952688640 1 -- 172.21.15.105:0/809861162 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f093806c980 msgr2=0x7f093806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:30.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.193+0000 7f0952688640 1 --2- 172.21.15.105:0/809861162 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f093806c980 0x7f093806ee40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f094c15b460 tx=0x7f09400079b0 comp rx=0 tx=0).stop 2024-04-15T15:21:30.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.193+0000 7f0952688640 1 -- 172.21.15.105:0/809861162 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f094c06f6d0 msgr2=0x7f094c1f0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:30.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.193+0000 7f0952688640 1 --2- 172.21.15.105:0/809861162 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f094c06f6d0 0x7f094c1f0270 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f093c00c7f0 tx=0x7f093c00ccb0 comp rx=0 tx=0).stop 2024-04-15T15:21:30.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.193+0000 7f0950c25640 1 -- 172.21.15.105:0/809861162 reap_dead start 2024-04-15T15:21:30.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.193+0000 7f0952688640 1 -- 172.21.15.105:0/809861162 shutdown_connections 2024-04-15T15:21:30.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.193+0000 7f0952688640 1 -- 172.21.15.105:0/809861162 >> 172.21.15.105:0/809861162 conn(0x7f094c067e10 msgr2=0x7f094c13ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:30.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.194+0000 7f0952688640 1 -- 172.21.15.105:0/809861162 shutdown_connections 2024-04-15T15:21:30.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.194+0000 7f0952688640 1 -- 172.21.15.105:0/809861162 wait complete. 2024-04-15T15:21:30.206 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-15T15:21:30.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.307+0000 7fc23031c640 1 Processor -- start 2024-04-15T15:21:30.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.307+0000 7fc23031c640 1 -- start start 2024-04-15T15:21:30.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.308+0000 7fc23031c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc22813e680 0x7fc2280cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:30.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.308+0000 7fc23031c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc22813ec60 0x7fc22806ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:30.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.308+0000 7fc23031c640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc22815bc90 con 0x7fc22806f4d0 2024-04-15T15:21:30.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.308+0000 7fc23031c640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fc22815bf70 con 0x7fc22813e680 2024-04-15T15:21:30.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.308+0000 7fc23031c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fc22815c250 con 0x7fc22813ec60 2024-04-15T15:21:30.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.308+0000 7fc22e8b9640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fc22806f4d0 0x7fc228071e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:45278/0 (socket says 172.21.15.105:45278) 2024-04-15T15:21:30.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.308+0000 7fc22e8b9640 1 -- 172.21.15.105:0/1513601806 learned_addr learned my addr 172.21.15.105:0/1513601806 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:21:30.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.308+0000 7fc22e0b8640 1 --2- 172.21.15.105:0/1513601806 >> v2:172.21.15.154:3300/0 conn(0x7fc22813e680 0x7fc2280cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:30.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.308+0000 7fc22d8b7640 1 --2- 172.21.15.105:0/1513601806 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc22813ec60 0x7fc22806ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:30.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.308+0000 7fc22d0b6640 1 -- 172.21.15.105:0/1513601806 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1467227369 0 0) 0x7fc22815bc90 con 0x7fc22806f4d0 2024-04-15T15:21:30.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.309+0000 7fc22d0b6640 1 -- 172.21.15.105:0/1513601806 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc214003520 con 0x7fc22806f4d0 2024-04-15T15:21:30.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.309+0000 7fc22d0b6640 1 -- 172.21.15.105:0/1513601806 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3077345991 0 0) 0x7fc214003520 con 0x7fc22806f4d0 2024-04-15T15:21:30.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.309+0000 7fc22d0b6640 1 -- 172.21.15.105:0/1513601806 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc22813ec60 msgr2=0x7fc22806ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:30.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.309+0000 7fc22d0b6640 1 --2- 172.21.15.105:0/1513601806 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc22813ec60 0x7fc22806ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:30.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.309+0000 7fc22d0b6640 1 -- 172.21.15.105:0/1513601806 >> v2:172.21.15.154:3300/0 conn(0x7fc22813e680 msgr2=0x7fc2280cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:30.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.309+0000 7fc22d0b6640 1 --2- 172.21.15.105:0/1513601806 >> v2:172.21.15.154:3300/0 conn(0x7fc22813e680 0x7fc2280cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:30.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.309+0000 7fc22d0b6640 1 -- 172.21.15.105:0/1513601806 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc22815c530 con 0x7fc22806f4d0 2024-04-15T15:21:30.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.309+0000 7fc22d0b6640 1 -- 172.21.15.105:0/1513601806 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7fc2200019f0 con 0x7fc22806f4d0 2024-04-15T15:21:30.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.309+0000 7fc22d0b6640 1 -- 172.21.15.105:0/1513601806 >> v1:172.21.15.105:6789/0 conn(0x7fc22806f4d0 legacy=0x7fc228071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:30.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.309+0000 7fc22d0b6640 1 --2- 172.21.15.105:0/1513601806 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2140039b0 0x7fc214003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:30.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.309+0000 7fc22d0b6640 1 --2- 172.21.15.105:0/1513601806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc214004800 0x7fc214004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:30.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.309+0000 7fc22d0b6640 1 --2- 172.21.15.105:0/1513601806 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc22813ec60 0x7fc2140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:30.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.309+0000 7fc22d0b6640 1 -- 172.21.15.105:0/1513601806 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc22815bf70 con 0x7fc214004800 2024-04-15T15:21:30.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.309+0000 7fc22d0b6640 1 -- 172.21.15.105:0/1513601806 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc22815c250 con 0x7fc2140039b0 2024-04-15T15:21:30.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.309+0000 7fc22d0b6640 1 -- 172.21.15.105:0/1513601806 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fc21400b9c0 con 0x7fc22813ec60 2024-04-15T15:21:30.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.309+0000 7fc22d8b7640 1 --2- 172.21.15.105:0/1513601806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc214004800 0x7fc214004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:30.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.309+0000 7fc22e0b8640 1 --2- 172.21.15.105:0/1513601806 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2140039b0 0x7fc214003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:30.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.309+0000 7fc22e8b9640 1 --2- 172.21.15.105:0/1513601806 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc22813ec60 0x7fc2140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:30.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.309+0000 7fc22d8b7640 1 -- 172.21.15.105:0/1513601806 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc22813ec60 msgr2=0x7fc2140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:30.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.309+0000 7fc22d8b7640 1 --2- 172.21.15.105:0/1513601806 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc22813ec60 0x7fc2140051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:30.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.309+0000 7fc22d8b7640 1 -- 172.21.15.105:0/1513601806 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2140039b0 msgr2=0x7fc214003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:30.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.309+0000 7fc22d8b7640 1 --2- 172.21.15.105:0/1513601806 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2140039b0 0x7fc214003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:30.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.309+0000 7fc22d8b7640 1 -- 172.21.15.105:0/1513601806 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc214003520 con 0x7fc214004800 2024-04-15T15:21:30.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.309+0000 7fc22e8b9640 1 --2- 172.21.15.105:0/1513601806 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc22813ec60 0x7fc2140051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:21:30.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.310+0000 7fc22d8b7640 1 --2- 172.21.15.105:0/1513601806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc214004800 0x7fc214004c80 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fc218009ac0 tx=0x7fc214005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdd3344ff2d4df07 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:30.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.310+0000 7fc22d0b6640 1 -- 172.21.15.105:0/1513601806 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc218039070 con 0x7fc214004800 2024-04-15T15:21:30.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.310+0000 7fc22d0b6640 1 -- 172.21.15.105:0/1513601806 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc21802c6b0 con 0x7fc214004800 2024-04-15T15:21:30.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.310+0000 7fc22d0b6640 1 -- 172.21.15.105:0/1513601806 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc218009690 con 0x7fc214004800 2024-04-15T15:21:30.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.311+0000 7fc23031c640 1 -- 172.21.15.105:0/1513601806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc214004800 msgr2=0x7fc214004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:30.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.311+0000 7fc23031c640 1 --2- 172.21.15.105:0/1513601806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc214004800 0x7fc214004c80 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fc218009ac0 tx=0x7fc214005830 comp rx=0 tx=0).stop 2024-04-15T15:21:30.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.311+0000 7fc22e8b9640 1 -- 172.21.15.105:0/1513601806 reap_dead start 2024-04-15T15:21:30.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.311+0000 7fc23031c640 1 -- 172.21.15.105:0/1513601806 shutdown_connections 2024-04-15T15:21:30.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.311+0000 7fc23031c640 1 -- 172.21.15.105:0/1513601806 >> 172.21.15.105:0/1513601806 conn(0x7fc228067d50 msgr2=0x7fc22806b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:30.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.311+0000 7fc23031c640 1 -- 172.21.15.105:0/1513601806 shutdown_connections 2024-04-15T15:21:30.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.311+0000 7fc23031c640 1 -- 172.21.15.105:0/1513601806 wait complete. 2024-04-15T15:21:30.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.312+0000 7fc23031c640 1 Processor -- start 2024-04-15T15:21:30.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.312+0000 7fc23031c640 1 -- start start 2024-04-15T15:21:30.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.313+0000 7fc23031c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc214004800 0x7fc2281595f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:30.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.313+0000 7fc23031c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc2281f6380 0x7fc2280ab090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:30.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.313+0000 7fc23031c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2280ab5d0 0x7fc2281f2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:30.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.313+0000 7fc23031c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc2280d0230 con 0x7fc214004800 2024-04-15T15:21:30.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.313+0000 7fc23031c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc2280d00b0 con 0x7fc2280ab5d0 2024-04-15T15:21:30.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.313+0000 7fc23031c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fc2280cf590 con 0x7fc2281f6380 2024-04-15T15:21:30.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.313+0000 7fc22e0b8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc214004800 0x7fc2281595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:30.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.313+0000 7fc22e0b8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc214004800 0x7fc2281595f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47314/0 (socket says 172.21.15.105:47314) 2024-04-15T15:21:30.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.313+0000 7fc22e0b8640 1 -- 172.21.15.105:0/3836272389 learned_addr learned my addr 172.21.15.105:0/3836272389 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:30.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.313+0000 7fc22d8b7640 1 --2- 172.21.15.105:0/3836272389 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc2281f6380 0x7fc2280ab090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:30.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.313+0000 7fc22e8b9640 1 --2- 172.21.15.105:0/3836272389 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2280ab5d0 0x7fc2281f2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:30.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.313+0000 7fc22e0b8640 1 -- 172.21.15.105:0/3836272389 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc2281f6380 msgr2=0x7fc2280ab090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:30.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.313+0000 7fc22e0b8640 1 --2- 172.21.15.105:0/3836272389 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc2281f6380 0x7fc2280ab090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:30.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.313+0000 7fc22e0b8640 1 -- 172.21.15.105:0/3836272389 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2280ab5d0 msgr2=0x7fc2281f2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:30.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.313+0000 7fc22e0b8640 1 --2- 172.21.15.105:0/3836272389 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2280ab5d0 0x7fc2281f2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:30.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.313+0000 7fc22e0b8640 1 -- 172.21.15.105:0/3836272389 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2280ae4e0 con 0x7fc214004800 2024-04-15T15:21:30.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.313+0000 7fc22e0b8640 1 --2- 172.21.15.105:0/3836272389 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc214004800 0x7fc2281595f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fc224011a70 tx=0x7fc224011f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:30.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.313+0000 7fc21effd640 1 -- 172.21.15.105:0/3836272389 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22400fb50 con 0x7fc214004800 2024-04-15T15:21:30.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.313+0000 7fc21effd640 1 -- 172.21.15.105:0/3836272389 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc22400fcf0 con 0x7fc214004800 2024-04-15T15:21:30.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.313+0000 7fc23031c640 1 -- 172.21.15.105:0/3836272389 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2280ad440 con 0x7fc214004800 2024-04-15T15:21:30.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.313+0000 7fc21effd640 1 -- 172.21.15.105:0/3836272389 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22400fea0 con 0x7fc214004800 2024-04-15T15:21:30.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.313+0000 7fc23031c640 1 -- 172.21.15.105:0/3836272389 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2280ada00 con 0x7fc214004800 2024-04-15T15:21:30.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.314+0000 7fc23031c640 1 -- 172.21.15.105:0/3836272389 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc228068a40 con 0x7fc214004800 2024-04-15T15:21:30.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.315+0000 7fc21effd640 1 -- 172.21.15.105:0/3836272389 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fc224012040 con 0x7fc214004800 2024-04-15T15:21:30.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.315+0000 7fc21effd640 1 --2- 172.21.15.105:0/3836272389 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fc20c06c980 0x7fc20c06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:30.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.315+0000 7fc22d8b7640 1 --2- 172.21.15.105:0/3836272389 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fc20c06c980 0x7fc20c06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:30.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.315+0000 7fc21effd640 1 -- 172.21.15.105:0/3836272389 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5029+0+0 (secure 0 0 0) 0x7fc22408bf90 con 0x7fc214004800 2024-04-15T15:21:30.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.316+0000 7fc22d8b7640 1 --2- 172.21.15.105:0/3836272389 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fc20c06c980 0x7fc20c06ee40 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc218036000 tx=0x7fc218030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:30.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.320+0000 7fc21effd640 1 -- 172.21.15.105:0/3836272389 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc224019030 con 0x7fc214004800 2024-04-15T15:21:30.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.496+0000 7fc23031c640 1 -- 172.21.15.105:0/3836272389 --> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7fc2280b5300 con 0x7fc20c06c980 2024-04-15T15:21:30.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.496+0000 7fc21effd640 1 -- 172.21.15.105:0/3836272389 <== mgr.4107 v2:172.21.15.105:6832/3506549322 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fc2280b5300 con 0x7fc20c06c980 2024-04-15T15:21:30.498 INFO:teuthology.orchestra.run.smithi105.stderr:instructed osd(s) 4 to deep-scrub 2024-04-15T15:21:30.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.498+0000 7fc23031c640 1 -- 172.21.15.105:0/3836272389 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fc20c06c980 msgr2=0x7fc20c06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:30.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.498+0000 7fc23031c640 1 --2- 172.21.15.105:0/3836272389 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fc20c06c980 0x7fc20c06ee40 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc218036000 tx=0x7fc218030000 comp rx=0 tx=0).stop 2024-04-15T15:21:30.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.498+0000 7fc23031c640 1 -- 172.21.15.105:0/3836272389 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc214004800 msgr2=0x7fc2281595f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:30.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.498+0000 7fc23031c640 1 --2- 172.21.15.105:0/3836272389 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc214004800 0x7fc2281595f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fc224011a70 tx=0x7fc224011f30 comp rx=0 tx=0).stop 2024-04-15T15:21:30.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.498+0000 7fc23031c640 1 -- 172.21.15.105:0/3836272389 shutdown_connections 2024-04-15T15:21:30.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.498+0000 7fc23031c640 1 --2- 172.21.15.105:0/3836272389 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc2281f6380 0x7fc2280ab090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:30.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.498+0000 7fc23031c640 1 --2- 172.21.15.105:0/3836272389 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fc20c06c980 0x7fc20c06ee40 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:30.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.498+0000 7fc23031c640 1 --2- 172.21.15.105:0/3836272389 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2280ab5d0 0x7fc2281f2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:30.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.498+0000 7fc23031c640 1 --2- 172.21.15.105:0/3836272389 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc214004800 0x7fc2281595f0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:30.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.498+0000 7fc23031c640 1 -- 172.21.15.105:0/3836272389 >> 172.21.15.105:0/3836272389 conn(0x7fc228067d50 msgr2=0x7fc2280b3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:30.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.498+0000 7fc23031c640 1 -- 172.21.15.105:0/3836272389 shutdown_connections 2024-04-15T15:21:30.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.498+0000 7fc23031c640 1 -- 172.21.15.105:0/3836272389 wait complete. 2024-04-15T15:21:30.511 INFO:tasks.ceph:Scrubbing osd.5 2024-04-15T15:21:30.512 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-15T15:21:30.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.617+0000 7f1a28900640 1 Processor -- start 2024-04-15T15:21:30.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.617+0000 7f1a28900640 1 -- start start 2024-04-15T15:21:30.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.618+0000 7f1a28900640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1a2413e6f0 0x7f1a240cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:30.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.618+0000 7f1a28900640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1a2406f6d0 0x7f1a2415bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:30.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.618+0000 7f1a28900640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a2415f280 con 0x7f1a240d0060 2024-04-15T15:21:30.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.618+0000 7f1a28900640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f1a2415f560 con 0x7f1a2413e6f0 2024-04-15T15:21:30.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.618+0000 7f1a28900640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f1a2415f840 con 0x7f1a2406f6d0 2024-04-15T15:21:30.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.618+0000 7f1a2259c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1a2413e6f0 0x7f1a240cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:30.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.618+0000 7f1a2259c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1a2413e6f0 0x7f1a240cfb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51014/0 (socket says 172.21.15.105:51014) 2024-04-15T15:21:30.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.618+0000 7f1a2259c640 1 -- 172.21.15.105:0/3352758864 learned_addr learned my addr 172.21.15.105:0/3352758864 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:30.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.618+0000 7f1a22d9d640 1 --2- 172.21.15.105:0/3352758864 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1a2406f6d0 0x7f1a2415bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:30.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.618+0000 7f1a2259c640 1 -- 172.21.15.105:0/3352758864 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1a2406f6d0 msgr2=0x7f1a2415bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:30.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.618+0000 7f1a2259c640 1 --2- 172.21.15.105:0/3352758864 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1a2406f6d0 0x7f1a2415bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:30.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.618+0000 7f1a2259c640 1 -- 172.21.15.105:0/3352758864 >> v1:172.21.15.154:6789/0 conn(0x7f1a240d0060 legacy=0x7f1a2406efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:30.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.618+0000 7f1a2259c640 1 -- 172.21.15.105:0/3352758864 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a2415fb20 con 0x7f1a2413e6f0 2024-04-15T15:21:30.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.619+0000 7f1a2259c640 1 --2- 172.21.15.105:0/3352758864 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a2413e6f0 0x7f1a240cfb20 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f1a1400bb10 tx=0x7f1a1402e030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83cead025c485b52 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:30.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.619+0000 7f1a21d9b640 1 -- 172.21.15.105:0/3352758864 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a1403b070 con 0x7f1a2413e6f0 2024-04-15T15:21:30.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.619+0000 7f1a21d9b640 1 -- 172.21.15.105:0/3352758864 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a1402ede0 con 0x7f1a2413e6f0 2024-04-15T15:21:30.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.619+0000 7f1a21d9b640 1 -- 172.21.15.105:0/3352758864 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a14036660 con 0x7f1a2413e6f0 2024-04-15T15:21:30.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.620+0000 7f1a28900640 1 -- 172.21.15.105:0/3352758864 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a2413e6f0 msgr2=0x7f1a240cfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:30.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.620+0000 7f1a28900640 1 --2- 172.21.15.105:0/3352758864 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a2413e6f0 0x7f1a240cfb20 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f1a1400bb10 tx=0x7f1a1402e030 comp rx=0 tx=0).stop 2024-04-15T15:21:30.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.620+0000 7f1a28900640 1 -- 172.21.15.105:0/3352758864 shutdown_connections 2024-04-15T15:21:30.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.620+0000 7f1a28900640 1 --2- 172.21.15.105:0/3352758864 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1a2406f6d0 0x7f1a2415bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:30.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.620+0000 7f1a28900640 1 --2- 172.21.15.105:0/3352758864 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a2413e6f0 0x7f1a240cfb20 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:30.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.620+0000 7f1a28900640 1 -- 172.21.15.105:0/3352758864 >> 172.21.15.105:0/3352758864 conn(0x7f1a24067e10 msgr2=0x7f1a240b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:30.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.620+0000 7f1a28900640 1 -- 172.21.15.105:0/3352758864 shutdown_connections 2024-04-15T15:21:30.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.620+0000 7f1a28900640 1 -- 172.21.15.105:0/3352758864 wait complete. 2024-04-15T15:21:30.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.621+0000 7f1a28900640 1 Processor -- start 2024-04-15T15:21:30.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.621+0000 7f1a28900640 1 -- start start 2024-04-15T15:21:30.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.622+0000 7f1a28900640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a2406f6d0 0x7f1a241424d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:30.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.622+0000 7f1a28900640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1a240d0060 0x7f1a241f7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:30.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.622+0000 7f1a28900640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1a2413e6f0 0x7f1a2415a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:30.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.622+0000 7f1a28900640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1a240cf7f0 con 0x7f1a2406f6d0 2024-04-15T15:21:30.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.622+0000 7f1a28900640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1a240cf670 con 0x7f1a2413e6f0 2024-04-15T15:21:30.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.622+0000 7f1a28900640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f1a2413ecd0 con 0x7f1a240d0060 2024-04-15T15:21:30.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.622+0000 7f1a2259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a2406f6d0 0x7f1a241424d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:30.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.622+0000 7f1a2259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a2406f6d0 0x7f1a241424d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51016/0 (socket says 172.21.15.105:51016) 2024-04-15T15:21:30.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.622+0000 7f1a2259c640 1 -- 172.21.15.105:0/303528897 learned_addr learned my addr 172.21.15.105:0/303528897 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:30.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.622+0000 7f1a2259c640 1 -- 172.21.15.105:0/303528897 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1a240d0060 msgr2=0x7f1a241f7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:30.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.622+0000 7f1a22d9d640 1 --2- 172.21.15.105:0/303528897 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1a2413e6f0 0x7f1a2415a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:30.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.622+0000 7f1a2259c640 1 --2- 172.21.15.105:0/303528897 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1a240d0060 0x7f1a241f7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:30.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.622+0000 7f1a2259c640 1 -- 172.21.15.105:0/303528897 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1a2413e6f0 msgr2=0x7f1a2415a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:30.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.622+0000 7f1a2259c640 1 --2- 172.21.15.105:0/303528897 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1a2413e6f0 0x7f1a2415a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:30.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.622+0000 7f1a2259c640 1 -- 172.21.15.105:0/303528897 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a24142a10 con 0x7f1a2406f6d0 2024-04-15T15:21:30.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.622+0000 7f1a2259c640 1 --2- 172.21.15.105:0/303528897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a2406f6d0 0x7f1a241424d0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f1a1400b2c0 tx=0x7f1a14030070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:30.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.623+0000 7f1a1affd640 1 -- 172.21.15.105:0/303528897 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a14046070 con 0x7f1a2406f6d0 2024-04-15T15:21:30.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.623+0000 7f1a1affd640 1 -- 172.21.15.105:0/303528897 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a1403f550 con 0x7f1a2406f6d0 2024-04-15T15:21:30.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.623+0000 7f1a28900640 1 -- 172.21.15.105:0/303528897 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a241f4920 con 0x7f1a2406f6d0 2024-04-15T15:21:30.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.623+0000 7f1a1affd640 1 -- 172.21.15.105:0/303528897 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a1403b070 con 0x7f1a2406f6d0 2024-04-15T15:21:30.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.623+0000 7f1a28900640 1 -- 172.21.15.105:0/303528897 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a24143590 con 0x7f1a2406f6d0 2024-04-15T15:21:30.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.623+0000 7f1a28900640 1 -- 172.21.15.105:0/303528897 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1a24143060 con 0x7f1a2406f6d0 2024-04-15T15:21:30.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.624+0000 7f1a1affd640 1 -- 172.21.15.105:0/303528897 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f1a14049070 con 0x7f1a2406f6d0 2024-04-15T15:21:30.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.624+0000 7f1a1affd640 1 --2- 172.21.15.105:0/303528897 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f1a0406c980 0x7f1a0406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:30.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.625+0000 7f1a1affd640 1 -- 172.21.15.105:0/303528897 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5029+0+0 (secure 0 0 0) 0x7f1a140ad530 con 0x7f1a2406f6d0 2024-04-15T15:21:30.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.625+0000 7f1a1bfff640 1 --2- 172.21.15.105:0/303528897 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f1a0406c980 0x7f1a0406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:30.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.625+0000 7f1a1affd640 1 --2- 172.21.15.105:0/303528897 >> [v2:172.21.15.154:6800/540367751,v1:172.21.15.154:6801/540367751] conn(0x7f1a040728e0 0x7f1a04074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:30.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.625+0000 7f1a1affd640 1 -- 172.21.15.105:0/303528897 --> [v2:172.21.15.154:6800/540367751,v1:172.21.15.154:6801/540367751] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1a1400b110 con 0x7f1a040728e0 2024-04-15T15:21:30.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.625+0000 7f1a1affd640 1 -- 172.21.15.105:0/303528897 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f1a140b4050 con 0x7f1a2406f6d0 2024-04-15T15:21:30.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.625+0000 7f1a22d9d640 1 --2- 172.21.15.105:0/303528897 >> [v2:172.21.15.154:6800/540367751,v1:172.21.15.154:6801/540367751] conn(0x7f1a040728e0 0x7f1a04074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:30.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.625+0000 7f1a1bfff640 1 --2- 172.21.15.105:0/303528897 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f1a0406c980 0x7f1a0406ee40 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1a240af5d0 tx=0x7f1a0c0079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:30.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.625+0000 7f1a22d9d640 1 --2- 172.21.15.105:0/303528897 >> [v2:172.21.15.154:6800/540367751,v1:172.21.15.154:6801/540367751] conn(0x7f1a040728e0 0x7f1a04074d00 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:30.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.626+0000 7f1a1affd640 1 -- 172.21.15.105:0/303528897 <== osd.5 v2:172.21.15.154:6800/540367751 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1a1400b110 con 0x7f1a040728e0 2024-04-15T15:21:30.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.639+0000 7f1a28900640 1 -- 172.21.15.105:0/303528897 --> [v2:172.21.15.154:6800/540367751,v1:172.21.15.154:6801/540367751] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f1a24143f80 con 0x7f1a040728e0 2024-04-15T15:21:30.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.640+0000 7f1a1affd640 1 -- 172.21.15.105:0/303528897 <== osd.5 v2:172.21.15.154:6800/540367751 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f1a24143f80 con 0x7f1a040728e0 2024-04-15T15:21:30.641 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-04-15T15:21:30.642 INFO:teuthology.orchestra.run.smithi105.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-15T15:21:30.642 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-04-15T15:21:30.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.640+0000 7f1a28900640 1 -- 172.21.15.105:0/303528897 >> [v2:172.21.15.154:6800/540367751,v1:172.21.15.154:6801/540367751] conn(0x7f1a040728e0 msgr2=0x7f1a04074d00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:30.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.640+0000 7f1a28900640 1 --2- 172.21.15.105:0/303528897 >> [v2:172.21.15.154:6800/540367751,v1:172.21.15.154:6801/540367751] conn(0x7f1a040728e0 0x7f1a04074d00 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:30.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.640+0000 7f1a28900640 1 -- 172.21.15.105:0/303528897 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f1a0406c980 msgr2=0x7f1a0406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:30.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.640+0000 7f1a28900640 1 --2- 172.21.15.105:0/303528897 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f1a0406c980 0x7f1a0406ee40 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1a240af5d0 tx=0x7f1a0c0079b0 comp rx=0 tx=0).stop 2024-04-15T15:21:30.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.640+0000 7f1a28900640 1 -- 172.21.15.105:0/303528897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a2406f6d0 msgr2=0x7f1a241424d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:30.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.640+0000 7f1a28900640 1 --2- 172.21.15.105:0/303528897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a2406f6d0 0x7f1a241424d0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f1a1400b2c0 tx=0x7f1a14030070 comp rx=0 tx=0).stop 2024-04-15T15:21:30.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.640+0000 7f1a22d9d640 1 -- 172.21.15.105:0/303528897 reap_dead start 2024-04-15T15:21:30.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.640+0000 7f1a28900640 1 -- 172.21.15.105:0/303528897 shutdown_connections 2024-04-15T15:21:30.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.640+0000 7f1a28900640 1 -- 172.21.15.105:0/303528897 >> 172.21.15.105:0/303528897 conn(0x7f1a24067e10 msgr2=0x7f1a240b1840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:30.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.640+0000 7f1a28900640 1 -- 172.21.15.105:0/303528897 shutdown_connections 2024-04-15T15:21:30.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.640+0000 7f1a28900640 1 -- 172.21.15.105:0/303528897 wait complete. 2024-04-15T15:21:30.652 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-15T15:21:30.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.755+0000 7fa22f700640 1 Processor -- start 2024-04-15T15:21:30.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.755+0000 7fa22f700640 1 -- start start 2024-04-15T15:21:30.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.756+0000 7fa22f700640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa22815ec60 0x7fa2280d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:30.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.756+0000 7fa22f700640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2280714f0 con 0x7fa22809b150 2024-04-15T15:21:30.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.756+0000 7fa22f700640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2280d1940 con 0x7fa22809b780 2024-04-15T15:21:30.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.756+0000 7fa22f700640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fa22806cad0 con 0x7fa22815ec60 2024-04-15T15:21:30.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.756+0000 7fa22d49c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa22815ec60 0x7fa2280d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:30.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.756+0000 7fa22dc9d640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fa22809b150 0x7fa2280cd840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:43968/0 (socket says 172.21.15.105:43968) 2024-04-15T15:21:30.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.756+0000 7fa22dc9d640 1 -- 172.21.15.105:0/1274246684 learned_addr learned my addr 172.21.15.105:0/1274246684 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:21:30.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.756+0000 7fa22d49c640 1 -- 172.21.15.105:0/1274246684 >> v1:172.21.15.154:6789/0 conn(0x7fa22809b780 legacy=0x7fa22809aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:30.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.756+0000 7fa22d49c640 1 -- 172.21.15.105:0/1274246684 >> v1:172.21.15.105:6789/0 conn(0x7fa22809b150 legacy=0x7fa2280cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:30.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.756+0000 7fa21ffff640 1 -- 172.21.15.105:0/1274246684 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1209854187 0 0) 0x7fa2280714f0 con 0x7fa22809b150 2024-04-15T15:21:30.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.756+0000 7fa22d49c640 1 -- 172.21.15.105:0/1274246684 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2280ccd60 con 0x7fa22815ec60 2024-04-15T15:21:30.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.757+0000 7fa22d49c640 1 --2- 172.21.15.105:0/1274246684 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa22815ec60 0x7fa2280d1210 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fa2240097a0 tx=0x7fa22402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a5f9304a7dc7b90 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:30.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.757+0000 7fa21ffff640 1 -- 172.21.15.105:0/1274246684 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa224034070 con 0x7fa22815ec60 2024-04-15T15:21:30.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.757+0000 7fa21ffff640 1 -- 172.21.15.105:0/1274246684 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa224030610 con 0x7fa22815ec60 2024-04-15T15:21:30.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.757+0000 7fa21ffff640 1 -- 172.21.15.105:0/1274246684 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2240309e0 con 0x7fa22815ec60 2024-04-15T15:21:30.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.758+0000 7fa22f700640 1 -- 172.21.15.105:0/1274246684 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa22815ec60 msgr2=0x7fa2280d1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:30.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.758+0000 7fa22f700640 1 --2- 172.21.15.105:0/1274246684 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa22815ec60 0x7fa2280d1210 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fa2240097a0 tx=0x7fa22402b960 comp rx=0 tx=0).stop 2024-04-15T15:21:30.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.758+0000 7fa22f700640 1 -- 172.21.15.105:0/1274246684 shutdown_connections 2024-04-15T15:21:30.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.758+0000 7fa22f700640 1 --2- 172.21.15.105:0/1274246684 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa22815ec60 0x7fa2280d1210 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:30.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.758+0000 7fa22f700640 1 -- 172.21.15.105:0/1274246684 >> 172.21.15.105:0/1274246684 conn(0x7fa2280d2570 msgr2=0x7fa2280d2980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:30.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.758+0000 7fa22f700640 1 -- 172.21.15.105:0/1274246684 shutdown_connections 2024-04-15T15:21:30.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.758+0000 7fa22f700640 1 -- 172.21.15.105:0/1274246684 wait complete. 2024-04-15T15:21:30.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.759+0000 7fa22f700640 1 Processor -- start 2024-04-15T15:21:30.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.759+0000 7fa22f700640 1 -- start start 2024-04-15T15:21:30.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.760+0000 7fa22f700640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa22809b150 0x7fa228150a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:30.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.760+0000 7fa22f700640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa22809b780 0x7fa2280b6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:30.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.760+0000 7fa22f700640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa2280b6dd0 0x7fa2280ca770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:30.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.760+0000 7fa22f700640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa2280cdf50 con 0x7fa22809b150 2024-04-15T15:21:30.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.760+0000 7fa22f700640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa2280cd3b0 con 0x7fa22809b780 2024-04-15T15:21:30.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.760+0000 7fa22f700640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fa2280ce0d0 con 0x7fa2280b6dd0 2024-04-15T15:21:30.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.760+0000 7fa22d49c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa22809b150 0x7fa228150a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:30.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.760+0000 7fa22d49c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa22809b150 0x7fa228150a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51022/0 (socket says 172.21.15.105:51022) 2024-04-15T15:21:30.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.760+0000 7fa22d49c640 1 -- 172.21.15.105:0/3669109070 learned_addr learned my addr 172.21.15.105:0/3669109070 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:30.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.760+0000 7fa22cc9b640 1 --2- 172.21.15.105:0/3669109070 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa22809b780 0x7fa2280b6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:30.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.760+0000 7fa22d49c640 1 -- 172.21.15.105:0/3669109070 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa2280b6dd0 msgr2=0x7fa2280ca770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:30.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.760+0000 7fa22d49c640 1 --2- 172.21.15.105:0/3669109070 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa2280b6dd0 0x7fa2280ca770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:30.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.760+0000 7fa22d49c640 1 -- 172.21.15.105:0/3669109070 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa22809b780 msgr2=0x7fa2280b6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:30.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.760+0000 7fa22d49c640 1 --2- 172.21.15.105:0/3669109070 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa22809b780 0x7fa2280b6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:30.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.760+0000 7fa22d49c640 1 -- 172.21.15.105:0/3669109070 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2280cb430 con 0x7fa22809b150 2024-04-15T15:21:30.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.760+0000 7fa22d49c640 1 --2- 172.21.15.105:0/3669109070 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa22809b150 0x7fa228150a80 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fa2240097a0 tx=0x7fa2240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:30.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.761+0000 7fa21dffb640 1 -- 172.21.15.105:0/3669109070 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa224047070 con 0x7fa22809b150 2024-04-15T15:21:30.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.761+0000 7fa21dffb640 1 -- 172.21.15.105:0/3669109070 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa224004030 con 0x7fa22809b150 2024-04-15T15:21:30.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.761+0000 7fa22f700640 1 -- 172.21.15.105:0/3669109070 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2280cb710 con 0x7fa22809b150 2024-04-15T15:21:30.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.761+0000 7fa21dffb640 1 -- 172.21.15.105:0/3669109070 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa224034040 con 0x7fa22809b150 2024-04-15T15:21:30.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.761+0000 7fa22f700640 1 -- 172.21.15.105:0/3669109070 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2280cbcd0 con 0x7fa22809b150 2024-04-15T15:21:30.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.762+0000 7fa22f700640 1 -- 172.21.15.105:0/3669109070 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa228068a40 con 0x7fa22809b150 2024-04-15T15:21:30.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.762+0000 7fa21dffb640 1 -- 172.21.15.105:0/3669109070 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fa22404a070 con 0x7fa22809b150 2024-04-15T15:21:30.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.763+0000 7fa21dffb640 1 --2- 172.21.15.105:0/3669109070 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fa20c06c980 0x7fa20c06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:30.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.763+0000 7fa21dffb640 1 -- 172.21.15.105:0/3669109070 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5029+0+0 (secure 0 0 0) 0x7fa2240ad490 con 0x7fa22809b150 2024-04-15T15:21:30.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.763+0000 7fa22cc9b640 1 --2- 172.21.15.105:0/3669109070 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fa20c06c980 0x7fa20c06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:30.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.763+0000 7fa22cc9b640 1 --2- 172.21.15.105:0/3669109070 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fa20c06c980 0x7fa20c06ee40 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa2280b7cd0 tx=0x7fa218009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:30.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.767+0000 7fa21dffb640 1 -- 172.21.15.105:0/3669109070 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa22407baf0 con 0x7fa22809b150 2024-04-15T15:21:30.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.941+0000 7fa22f700640 1 -- 172.21.15.105:0/3669109070 --> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7fa2280b83f0 con 0x7fa20c06c980 2024-04-15T15:21:30.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.942+0000 7fa21dffb640 1 -- 172.21.15.105:0/3669109070 <== mgr.4107 v2:172.21.15.105:6832/3506549322 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fa2280b83f0 con 0x7fa20c06c980 2024-04-15T15:21:30.944 INFO:teuthology.orchestra.run.smithi105.stderr:instructed osd(s) 5 to deep-scrub 2024-04-15T15:21:30.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.943+0000 7fa22f700640 1 -- 172.21.15.105:0/3669109070 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fa20c06c980 msgr2=0x7fa20c06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:30.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.943+0000 7fa22f700640 1 --2- 172.21.15.105:0/3669109070 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fa20c06c980 0x7fa20c06ee40 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa2280b7cd0 tx=0x7fa218009040 comp rx=0 tx=0).stop 2024-04-15T15:21:30.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.943+0000 7fa22f700640 1 -- 172.21.15.105:0/3669109070 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa22809b150 msgr2=0x7fa228150a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:30.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.943+0000 7fa22f700640 1 --2- 172.21.15.105:0/3669109070 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa22809b150 0x7fa228150a80 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fa2240097a0 tx=0x7fa2240046c0 comp rx=0 tx=0).stop 2024-04-15T15:21:30.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.943+0000 7fa22f700640 1 -- 172.21.15.105:0/3669109070 shutdown_connections 2024-04-15T15:21:30.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.943+0000 7fa22f700640 1 --2- 172.21.15.105:0/3669109070 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa2280b6dd0 0x7fa2280ca770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:30.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.943+0000 7fa22f700640 1 --2- 172.21.15.105:0/3669109070 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fa20c06c980 0x7fa20c06ee40 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:30.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.943+0000 7fa22f700640 1 --2- 172.21.15.105:0/3669109070 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa22809b780 0x7fa2280b6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:30.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.943+0000 7fa22f700640 1 --2- 172.21.15.105:0/3669109070 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa22809b150 0x7fa228150a80 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:30.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.943+0000 7fa22f700640 1 -- 172.21.15.105:0/3669109070 >> 172.21.15.105:0/3669109070 conn(0x7fa2280d2570 msgr2=0x7fa22806cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:30.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.944+0000 7fa22f700640 1 -- 172.21.15.105:0/3669109070 shutdown_connections 2024-04-15T15:21:30.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:30.944+0000 7fa22f700640 1 -- 172.21.15.105:0/3669109070 wait complete. 2024-04-15T15:21:30.956 INFO:tasks.ceph:Scrubbing osd.6 2024-04-15T15:21:30.956 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-15T15:21:31.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.062+0000 7fe1c4ba8640 1 Processor -- start 2024-04-15T15:21:31.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.062+0000 7fe1c4ba8640 1 -- start start 2024-04-15T15:21:31.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.063+0000 7fe1c4ba8640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe1c015c930 0x7fe1c00b5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:31.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.063+0000 7fe1c4ba8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe1c00cfd80 0x7fe1c00d0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:31.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.063+0000 7fe1c4ba8640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1c00b5150 con 0x7fe1c0072660 2024-04-15T15:21:31.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.063+0000 7fe1c4ba8640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fe1c00b6130 con 0x7fe1c015c930 2024-04-15T15:21:31.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.063+0000 7fe1c4ba8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fe1c006c2c0 con 0x7fe1c00cfd80 2024-04-15T15:21:31.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.063+0000 7fe1bed9d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe1c00cfd80 0x7fe1c00d0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:31.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.063+0000 7fe1be59c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe1c015c930 0x7fe1c00b5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:31.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.063+0000 7fe1b7fff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fe1c0072660 0x7fe1c006b9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.105:43006/0 (socket says 172.21.15.105:43006) 2024-04-15T15:21:31.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.063+0000 7fe1bed9d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe1c00cfd80 0x7fe1c00d0230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.105:55872/0 (socket says 172.21.15.105:55872) 2024-04-15T15:21:31.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.063+0000 7fe1be59c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe1c015c930 0x7fe1c00b5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.105:41204/0 (socket says 172.21.15.105:41204) 2024-04-15T15:21:31.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.063+0000 7fe1b7fff640 1 -- 172.21.15.105:0/2807081918 learned_addr learned my addr 172.21.15.105:0/2807081918 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:21:31.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.063+0000 7fe1be59c640 1 -- 172.21.15.105:0/2807081918 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe1c00cfd80 msgr2=0x7fe1c00d0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:31.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.063+0000 7fe1be59c640 1 --2- 172.21.15.105:0/2807081918 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe1c00cfd80 0x7fe1c00d0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:31.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.063+0000 7fe1be59c640 1 -- 172.21.15.105:0/2807081918 >> v1:172.21.15.154:6789/0 conn(0x7fe1c0072660 legacy=0x7fe1c006b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:31.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.063+0000 7fe1be59c640 1 -- 172.21.15.105:0/2807081918 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1c006c5a0 con 0x7fe1c015c930 2024-04-15T15:21:31.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.063+0000 7fe1bdd9b640 1 -- 172.21.15.105:0/2807081918 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3530804648 0 0) 0x7fe1c00b5150 con 0x7fe1c0072660 2024-04-15T15:21:31.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.064+0000 7fe1be59c640 1 --2- 172.21.15.105:0/2807081918 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe1c015c930 0x7fe1c00b5bf0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe1b00097a0 tx=0x7fe1b002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b993ee19d79ced10 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:31.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.064+0000 7fe1bdd9b640 1 -- 172.21.15.105:0/2807081918 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b0035020 con 0x7fe1c015c930 2024-04-15T15:21:31.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.064+0000 7fe1bdd9b640 1 -- 172.21.15.105:0/2807081918 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1b0031830 con 0x7fe1c015c930 2024-04-15T15:21:31.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.064+0000 7fe1bdd9b640 1 -- 172.21.15.105:0/2807081918 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b0031c00 con 0x7fe1c015c930 2024-04-15T15:21:31.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.065+0000 7fe1c4ba8640 1 -- 172.21.15.105:0/2807081918 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe1c015c930 msgr2=0x7fe1c00b5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:31.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.065+0000 7fe1c4ba8640 1 --2- 172.21.15.105:0/2807081918 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe1c015c930 0x7fe1c00b5bf0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe1b00097a0 tx=0x7fe1b002b8d0 comp rx=0 tx=0).stop 2024-04-15T15:21:31.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.065+0000 7fe1c4ba8640 1 -- 172.21.15.105:0/2807081918 shutdown_connections 2024-04-15T15:21:31.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.065+0000 7fe1c4ba8640 1 --2- 172.21.15.105:0/2807081918 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe1c00cfd80 0x7fe1c00d0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:31.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.065+0000 7fe1c4ba8640 1 --2- 172.21.15.105:0/2807081918 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe1c015c930 0x7fe1c00b5bf0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:31.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.065+0000 7fe1c4ba8640 1 -- 172.21.15.105:0/2807081918 >> 172.21.15.105:0/2807081918 conn(0x7fe1c00d0c60 msgr2=0x7fe1c00d1070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:31.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.065+0000 7fe1c4ba8640 1 -- 172.21.15.105:0/2807081918 shutdown_connections 2024-04-15T15:21:31.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.065+0000 7fe1c4ba8640 1 -- 172.21.15.105:0/2807081918 wait complete. 2024-04-15T15:21:31.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.066+0000 7fe1c4ba8640 1 Processor -- start 2024-04-15T15:21:31.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.066+0000 7fe1c4ba8640 1 -- start start 2024-04-15T15:21:31.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.067+0000 7fe1c4ba8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1c0072660 0x7fe1c01494b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:31.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.067+0000 7fe1c4ba8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe1c00cfd80 0x7fe1c01499f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:31.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.067+0000 7fe1c4ba8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe1c015c930 0x7fe1c01f6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:31.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.067+0000 7fe1c4ba8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c00add80 con 0x7fe1c0072660 2024-04-15T15:21:31.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.067+0000 7fe1c4ba8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c00b56d0 con 0x7fe1c015c930 2024-04-15T15:21:31.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.067+0000 7fe1c4ba8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fe1c00adf00 con 0x7fe1c00cfd80 2024-04-15T15:21:31.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.067+0000 7fe1be59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1c0072660 0x7fe1c01494b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:31.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.067+0000 7fe1be59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1c0072660 0x7fe1c01494b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51034/0 (socket says 172.21.15.105:51034) 2024-04-15T15:21:31.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.067+0000 7fe1be59c640 1 -- 172.21.15.105:0/4108442606 learned_addr learned my addr 172.21.15.105:0/4108442606 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:31.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.067+0000 7fe1b7fff640 1 --2- 172.21.15.105:0/4108442606 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe1c00cfd80 0x7fe1c01499f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:31.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.067+0000 7fe1bed9d640 1 --2- 172.21.15.105:0/4108442606 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe1c015c930 0x7fe1c01f6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:31.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.067+0000 7fe1be59c640 1 -- 172.21.15.105:0/4108442606 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe1c00cfd80 msgr2=0x7fe1c01499f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:31.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.067+0000 7fe1be59c640 1 --2- 172.21.15.105:0/4108442606 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe1c00cfd80 0x7fe1c01499f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:31.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.067+0000 7fe1be59c640 1 -- 172.21.15.105:0/4108442606 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe1c015c930 msgr2=0x7fe1c01f6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:31.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.067+0000 7fe1be59c640 1 --2- 172.21.15.105:0/4108442606 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe1c015c930 0x7fe1c01f6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:31.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.067+0000 7fe1be59c640 1 -- 172.21.15.105:0/4108442606 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1c015f5b0 con 0x7fe1c0072660 2024-04-15T15:21:31.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.068+0000 7fe1b7fff640 1 --2- 172.21.15.105:0/4108442606 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe1c00cfd80 0x7fe1c01499f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:21:31.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.068+0000 7fe1bed9d640 1 --2- 172.21.15.105:0/4108442606 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe1c015c930 0x7fe1c01f6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:21:31.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.068+0000 7fe1be59c640 1 --2- 172.21.15.105:0/4108442606 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1c0072660 0x7fe1c01494b0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fe1b00097a0 tx=0x7fe1b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:31.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.068+0000 7fe1b6ffd640 1 -- 172.21.15.105:0/4108442606 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b0035040 con 0x7fe1c0072660 2024-04-15T15:21:31.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.068+0000 7fe1b6ffd640 1 -- 172.21.15.105:0/4108442606 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1b0004230 con 0x7fe1c0072660 2024-04-15T15:21:31.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.068+0000 7fe1c4ba8640 1 -- 172.21.15.105:0/4108442606 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1c015f890 con 0x7fe1c0072660 2024-04-15T15:21:31.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.068+0000 7fe1b6ffd640 1 -- 172.21.15.105:0/4108442606 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b003a640 con 0x7fe1c0072660 2024-04-15T15:21:31.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.068+0000 7fe1c4ba8640 1 -- 172.21.15.105:0/4108442606 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1c0147490 con 0x7fe1c0072660 2024-04-15T15:21:31.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.069+0000 7fe1c4ba8640 1 -- 172.21.15.105:0/4108442606 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe1c00b5110 con 0x7fe1c0072660 2024-04-15T15:21:31.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.069+0000 7fe1b6ffd640 1 -- 172.21.15.105:0/4108442606 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fe1b0038020 con 0x7fe1c0072660 2024-04-15T15:21:31.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.070+0000 7fe1b6ffd640 1 --2- 172.21.15.105:0/4108442606 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fe1a006c980 0x7fe1a006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:31.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.070+0000 7fe1b6ffd640 1 -- 172.21.15.105:0/4108442606 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5029+0+0 (secure 0 0 0) 0x7fe1b00ad620 con 0x7fe1c0072660 2024-04-15T15:21:31.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.070+0000 7fe1b7fff640 1 --2- 172.21.15.105:0/4108442606 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fe1a006c980 0x7fe1a006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:31.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.070+0000 7fe1b6ffd640 1 --2- 172.21.15.105:0/4108442606 >> [v2:172.21.15.154:6824/1550829005,v1:172.21.15.154:6825/1550829005] conn(0x7fe1a00728e0 0x7fe1a0074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:31.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.070+0000 7fe1b6ffd640 1 -- 172.21.15.105:0/4108442606 --> [v2:172.21.15.154:6824/1550829005,v1:172.21.15.154:6825/1550829005] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe1b003ee50 con 0x7fe1a00728e0 2024-04-15T15:21:31.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.070+0000 7fe1b6ffd640 1 -- 172.21.15.105:0/4108442606 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fe1b00b4050 con 0x7fe1c0072660 2024-04-15T15:21:31.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.070+0000 7fe1b7fff640 1 --2- 172.21.15.105:0/4108442606 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fe1a006c980 0x7fe1a006ee40 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe1c01f62e0 tx=0x7fe1a8007c10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:31.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.070+0000 7fe1bed9d640 1 --2- 172.21.15.105:0/4108442606 >> [v2:172.21.15.154:6824/1550829005,v1:172.21.15.154:6825/1550829005] conn(0x7fe1a00728e0 0x7fe1a0074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:31.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.071+0000 7fe1bed9d640 1 --2- 172.21.15.105:0/4108442606 >> [v2:172.21.15.154:6824/1550829005,v1:172.21.15.154:6825/1550829005] conn(0x7fe1a00728e0 0x7fe1a0074d00 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:31.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.071+0000 7fe1b6ffd640 1 -- 172.21.15.105:0/4108442606 <== osd.6 v2:172.21.15.154:6824/1550829005 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe1b003ee50 con 0x7fe1a00728e0 2024-04-15T15:21:31.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.084+0000 7fe1c4ba8640 1 -- 172.21.15.105:0/4108442606 --> [v2:172.21.15.154:6824/1550829005,v1:172.21.15.154:6825/1550829005] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fe1c00694a0 con 0x7fe1a00728e0 2024-04-15T15:21:31.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.085+0000 7fe1b6ffd640 1 -- 172.21.15.105:0/4108442606 <== osd.6 v2:172.21.15.154:6824/1550829005 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fe1c00694a0 con 0x7fe1a00728e0 2024-04-15T15:21:31.087 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-04-15T15:21:31.087 INFO:teuthology.orchestra.run.smithi105.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-15T15:21:31.087 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-04-15T15:21:31.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.085+0000 7fe1c4ba8640 1 -- 172.21.15.105:0/4108442606 >> [v2:172.21.15.154:6824/1550829005,v1:172.21.15.154:6825/1550829005] conn(0x7fe1a00728e0 msgr2=0x7fe1a0074d00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:31.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.085+0000 7fe1c4ba8640 1 --2- 172.21.15.105:0/4108442606 >> [v2:172.21.15.154:6824/1550829005,v1:172.21.15.154:6825/1550829005] conn(0x7fe1a00728e0 0x7fe1a0074d00 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:31.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.085+0000 7fe1c4ba8640 1 -- 172.21.15.105:0/4108442606 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fe1a006c980 msgr2=0x7fe1a006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:31.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.085+0000 7fe1c4ba8640 1 --2- 172.21.15.105:0/4108442606 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7fe1a006c980 0x7fe1a006ee40 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe1c01f62e0 tx=0x7fe1a8007c10 comp rx=0 tx=0).stop 2024-04-15T15:21:31.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.085+0000 7fe1c4ba8640 1 -- 172.21.15.105:0/4108442606 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1c0072660 msgr2=0x7fe1c01494b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:31.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.085+0000 7fe1c4ba8640 1 --2- 172.21.15.105:0/4108442606 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1c0072660 0x7fe1c01494b0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fe1b00097a0 tx=0x7fe1b00046c0 comp rx=0 tx=0).stop 2024-04-15T15:21:31.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.085+0000 7fe1bed9d640 1 -- 172.21.15.105:0/4108442606 reap_dead start 2024-04-15T15:21:31.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.085+0000 7fe1c4ba8640 1 -- 172.21.15.105:0/4108442606 shutdown_connections 2024-04-15T15:21:31.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.085+0000 7fe1c4ba8640 1 -- 172.21.15.105:0/4108442606 >> 172.21.15.105:0/4108442606 conn(0x7fe1c00d0c60 msgr2=0x7fe1c00b08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:31.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.086+0000 7fe1c4ba8640 1 -- 172.21.15.105:0/4108442606 shutdown_connections 2024-04-15T15:21:31.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.086+0000 7fe1c4ba8640 1 -- 172.21.15.105:0/4108442606 wait complete. 2024-04-15T15:21:31.098 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-15T15:21:31.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.200+0000 7f3607ece640 1 Processor -- start 2024-04-15T15:21:31.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.200+0000 7f3607ece640 1 -- start start 2024-04-15T15:21:31.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.201+0000 7f3607ece640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f360015ecf0 0x7f360006e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:31.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.201+0000 7f3607ece640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f36000d0f10 0x7f36000ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:31.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.201+0000 7f3607ece640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f360006d2a0 con 0x7f36000cd190 2024-04-15T15:21:31.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.201+0000 7f3607ece640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f360006d580 con 0x7f36000d0f10 2024-04-15T15:21:31.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.201+0000 7f3607ece640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f360006d860 con 0x7f360015ecf0 2024-04-15T15:21:31.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.201+0000 7f3605469640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f36000d0f10 0x7f36000ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:31.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.201+0000 7f360646b640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f36000cd190 0x7f360009aa40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:43974/0 (socket says 172.21.15.105:43974) 2024-04-15T15:21:31.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.201+0000 7f360646b640 1 -- 172.21.15.105:0/439132426 learned_addr learned my addr 172.21.15.105:0/439132426 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:21:31.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.201+0000 7f3605c6a640 1 --2- 172.21.15.105:0/439132426 >> v2:172.21.15.154:3300/0 conn(0x7f360015ecf0 0x7f360006e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:31.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.202+0000 7f3605469640 1 -- 172.21.15.105:0/439132426 >> v2:172.21.15.154:3300/0 conn(0x7f360015ecf0 msgr2=0x7f360006e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:31.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.202+0000 7f3605469640 1 --2- 172.21.15.105:0/439132426 >> v2:172.21.15.154:3300/0 conn(0x7f360015ecf0 0x7f360006e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:31.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.202+0000 7f3605469640 1 -- 172.21.15.105:0/439132426 >> v1:172.21.15.105:6789/0 conn(0x7f36000cd190 legacy=0x7f360009aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:31.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.202+0000 7f3605469640 1 -- 172.21.15.105:0/439132426 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f360006db40 con 0x7f36000d0f10 2024-04-15T15:21:31.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.202+0000 7f3604c68640 1 -- 172.21.15.105:0/439132426 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 226702290 0 0) 0x7f360006d2a0 con 0x7f36000cd190 2024-04-15T15:21:31.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.202+0000 7f3605469640 1 --2- 172.21.15.105:0/439132426 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36000d0f10 0x7f36000ccc50 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f3600075520 tx=0x7f35f002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9b78c829bf97927 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:31.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.203+0000 7f3604c68640 1 -- 172.21.15.105:0/439132426 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f0035020 con 0x7f36000d0f10 2024-04-15T15:21:31.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.203+0000 7f3604c68640 1 -- 172.21.15.105:0/439132426 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35f0031830 con 0x7f36000d0f10 2024-04-15T15:21:31.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.203+0000 7f3604c68640 1 -- 172.21.15.105:0/439132426 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f0031c00 con 0x7f36000d0f10 2024-04-15T15:21:31.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.203+0000 7f3607ece640 1 -- 172.21.15.105:0/439132426 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36000d0f10 msgr2=0x7f36000ccc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:31.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.203+0000 7f3607ece640 1 --2- 172.21.15.105:0/439132426 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36000d0f10 0x7f36000ccc50 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f3600075520 tx=0x7f35f002b8d0 comp rx=0 tx=0).stop 2024-04-15T15:21:31.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.203+0000 7f3607ece640 1 -- 172.21.15.105:0/439132426 shutdown_connections 2024-04-15T15:21:31.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.203+0000 7f3607ece640 1 --2- 172.21.15.105:0/439132426 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36000d0f10 0x7f36000ccc50 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:31.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.203+0000 7f3607ece640 1 --2- 172.21.15.105:0/439132426 >> v2:172.21.15.154:3300/0 conn(0x7f360015ecf0 0x7f360006e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:31.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.203+0000 7f3607ece640 1 -- 172.21.15.105:0/439132426 >> 172.21.15.105:0/439132426 conn(0x7f36000d2600 msgr2=0x7f36000d2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:31.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.203+0000 7f3607ece640 1 -- 172.21.15.105:0/439132426 shutdown_connections 2024-04-15T15:21:31.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.203+0000 7f3607ece640 1 -- 172.21.15.105:0/439132426 wait complete. 2024-04-15T15:21:31.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.204+0000 7f3607ece640 1 Processor -- start 2024-04-15T15:21:31.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.204+0000 7f3607ece640 1 -- start start 2024-04-15T15:21:31.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.205+0000 7f3607ece640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36000cd190 0x7f36000c96d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:31.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.205+0000 7f3607ece640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36000d0f10 0x7f3600153980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:31.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.205+0000 7f3607ece640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f360015ecf0 0x7f36001f46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:31.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.205+0000 7f3607ece640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f360015f0d0 con 0x7f36000d0f10 2024-04-15T15:21:31.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.205+0000 7f3607ece640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f36000d1370 con 0x7f36000cd190 2024-04-15T15:21:31.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.205+0000 7f3607ece640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f360015f250 con 0x7f360015ecf0 2024-04-15T15:21:31.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.205+0000 7f3605469640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36000d0f10 0x7f3600153980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:31.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.205+0000 7f3605469640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36000d0f10 0x7f3600153980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51042/0 (socket says 172.21.15.105:51042) 2024-04-15T15:21:31.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.205+0000 7f3605469640 1 -- 172.21.15.105:0/1027380380 learned_addr learned my addr 172.21.15.105:0/1027380380 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:31.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.206+0000 7f360646b640 1 --2- 172.21.15.105:0/1027380380 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f360015ecf0 0x7f36001f46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:31.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.206+0000 7f3605c6a640 1 --2- 172.21.15.105:0/1027380380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36000cd190 0x7f36000c96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:31.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.206+0000 7f3605469640 1 -- 172.21.15.105:0/1027380380 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f360015ecf0 msgr2=0x7f36001f46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:31.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.206+0000 7f3605469640 1 --2- 172.21.15.105:0/1027380380 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f360015ecf0 0x7f36001f46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:31.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.206+0000 7f3605469640 1 -- 172.21.15.105:0/1027380380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36000cd190 msgr2=0x7f36000c96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:31.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.206+0000 7f3605469640 1 --2- 172.21.15.105:0/1027380380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36000cd190 0x7f36000c96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:31.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.206+0000 7f3605469640 1 -- 172.21.15.105:0/1027380380 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36000c9d60 con 0x7f36000d0f10 2024-04-15T15:21:31.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.206+0000 7f3605469640 1 --2- 172.21.15.105:0/1027380380 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36000d0f10 0x7f3600153980 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f35f002bee0 tx=0x7f35f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:31.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.206+0000 7f35f67fc640 1 -- 172.21.15.105:0/1027380380 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f0041070 con 0x7f36000d0f10 2024-04-15T15:21:31.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.206+0000 7f35f67fc640 1 -- 172.21.15.105:0/1027380380 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35f003f070 con 0x7f36000d0f10 2024-04-15T15:21:31.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.206+0000 7f3607ece640 1 -- 172.21.15.105:0/1027380380 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36000b6ae0 con 0x7f36000d0f10 2024-04-15T15:21:31.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.206+0000 7f35f67fc640 1 -- 172.21.15.105:0/1027380380 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f0035040 con 0x7f36000d0f10 2024-04-15T15:21:31.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.206+0000 7f3607ece640 1 -- 172.21.15.105:0/1027380380 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36000b70a0 con 0x7f36000d0f10 2024-04-15T15:21:31.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.207+0000 7f3607ece640 1 -- 172.21.15.105:0/1027380380 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3600068a40 con 0x7f36000d0f10 2024-04-15T15:21:31.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.207+0000 7f35f67fc640 1 -- 172.21.15.105:0/1027380380 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f35f0038020 con 0x7f36000d0f10 2024-04-15T15:21:31.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.208+0000 7f35f67fc640 1 --2- 172.21.15.105:0/1027380380 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f35e406c980 0x7f35e406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:31.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.208+0000 7f3605c6a640 1 --2- 172.21.15.105:0/1027380380 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f35e406c980 0x7f35e406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:31.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.208+0000 7f35f67fc640 1 -- 172.21.15.105:0/1027380380 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5029+0+0 (secure 0 0 0) 0x7f35f00ace00 con 0x7f36000d0f10 2024-04-15T15:21:31.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.208+0000 7f3605c6a640 1 --2- 172.21.15.105:0/1027380380 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f35e406c980 0x7f35e406ee40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f35fc009ad0 tx=0x7f35fc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:31.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.212+0000 7f35f67fc640 1 -- 172.21.15.105:0/1027380380 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f35f0031800 con 0x7f36000d0f10 2024-04-15T15:21:31.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.390+0000 7f3607ece640 1 -- 172.21.15.105:0/1027380380 --> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f360009b610 con 0x7f35e406c980 2024-04-15T15:21:31.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.390+0000 7f35f67fc640 1 -- 172.21.15.105:0/1027380380 <== mgr.4107 v2:172.21.15.105:6832/3506549322 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f360009b610 con 0x7f35e406c980 2024-04-15T15:21:31.392 INFO:teuthology.orchestra.run.smithi105.stderr:instructed osd(s) 6 to deep-scrub 2024-04-15T15:21:31.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.392+0000 7f3607ece640 1 -- 172.21.15.105:0/1027380380 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f35e406c980 msgr2=0x7f35e406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:31.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.392+0000 7f3607ece640 1 --2- 172.21.15.105:0/1027380380 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f35e406c980 0x7f35e406ee40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f35fc009ad0 tx=0x7f35fc008040 comp rx=0 tx=0).stop 2024-04-15T15:21:31.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.392+0000 7f3607ece640 1 -- 172.21.15.105:0/1027380380 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36000d0f10 msgr2=0x7f3600153980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:31.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.392+0000 7f3607ece640 1 --2- 172.21.15.105:0/1027380380 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36000d0f10 0x7f3600153980 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f35f002bee0 tx=0x7f35f00046c0 comp rx=0 tx=0).stop 2024-04-15T15:21:31.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.392+0000 7f3607ece640 1 -- 172.21.15.105:0/1027380380 shutdown_connections 2024-04-15T15:21:31.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.392+0000 7f3607ece640 1 --2- 172.21.15.105:0/1027380380 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f360015ecf0 0x7f36001f46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:31.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.392+0000 7f3607ece640 1 --2- 172.21.15.105:0/1027380380 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f35e406c980 0x7f35e406ee40 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:31.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.392+0000 7f3607ece640 1 --2- 172.21.15.105:0/1027380380 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36000d0f10 0x7f3600153980 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:31.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.392+0000 7f3607ece640 1 --2- 172.21.15.105:0/1027380380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36000cd190 0x7f36000c96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:31.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.392+0000 7f3607ece640 1 -- 172.21.15.105:0/1027380380 >> 172.21.15.105:0/1027380380 conn(0x7f36000d2600 msgr2=0x7f36000d17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:31.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.392+0000 7f3607ece640 1 -- 172.21.15.105:0/1027380380 shutdown_connections 2024-04-15T15:21:31.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.393+0000 7f3607ece640 1 -- 172.21.15.105:0/1027380380 wait complete. 2024-04-15T15:21:31.405 INFO:tasks.ceph:Scrubbing osd.7 2024-04-15T15:21:31.405 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-15T15:21:31.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.509+0000 7f5e949d4640 1 Processor -- start 2024-04-15T15:21:31.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.509+0000 7f5e949d4640 1 -- start start 2024-04-15T15:21:31.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.510+0000 7f5e949d4640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5e90146160 0x7f5e900b22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:31.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.510+0000 7f5e949d4640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e900b2810 0x7f5e900ccec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:31.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.510+0000 7f5e949d4640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e9015cf10 con 0x7f5e900cd400 2024-04-15T15:21:31.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.510+0000 7f5e949d4640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f5e90146540 con 0x7f5e90146160 2024-04-15T15:21:31.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.510+0000 7f5e949d4640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f5e9006c820 con 0x7f5e900b2810 2024-04-15T15:21:31.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.510+0000 7f5e8e59c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5e90146160 0x7f5e900b22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:31.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.510+0000 7f5e8ed9d640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f5e900cd400 0x7f5e9015dfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:43988/0 (socket says 172.21.15.105:43988) 2024-04-15T15:21:31.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.510+0000 7f5e8ed9d640 1 -- 172.21.15.105:0/709101380 learned_addr learned my addr 172.21.15.105:0/709101380 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:21:31.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.510+0000 7f5e85d9b640 1 --2- 172.21.15.105:0/709101380 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e900b2810 0x7f5e900ccec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:31.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.510+0000 7f5e8e59c640 1 -- 172.21.15.105:0/709101380 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e900b2810 msgr2=0x7f5e900ccec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:31.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.510+0000 7f5e8e59c640 1 --2- 172.21.15.105:0/709101380 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e900b2810 0x7f5e900ccec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:31.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.510+0000 7f5e8e59c640 1 -- 172.21.15.105:0/709101380 >> v1:172.21.15.105:6789/0 conn(0x7f5e900cd400 legacy=0x7f5e9015dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:31.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.510+0000 7f5e8e59c640 1 -- 172.21.15.105:0/709101380 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e9015d1b0 con 0x7f5e90146160 2024-04-15T15:21:31.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.510+0000 7f5e8dd9b640 1 -- 172.21.15.105:0/709101380 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 776122148 0 0) 0x7f5e9015cf10 con 0x7f5e900cd400 2024-04-15T15:21:31.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.511+0000 7f5e8e59c640 1 --2- 172.21.15.105:0/709101380 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e90146160 0x7f5e900b22d0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f5e800097d0 tx=0x7f5e8002b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a2d72aa12d1e8b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:31.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.511+0000 7f5e8dd9b640 1 -- 172.21.15.105:0/709101380 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e80035020 con 0x7f5e90146160 2024-04-15T15:21:31.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.511+0000 7f5e8dd9b640 1 -- 172.21.15.105:0/709101380 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e80031800 con 0x7f5e90146160 2024-04-15T15:21:31.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.511+0000 7f5e8dd9b640 1 -- 172.21.15.105:0/709101380 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e80031bd0 con 0x7f5e90146160 2024-04-15T15:21:31.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.511+0000 7f5e949d4640 1 -- 172.21.15.105:0/709101380 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e90146160 msgr2=0x7f5e900b22d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:31.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.511+0000 7f5e949d4640 1 --2- 172.21.15.105:0/709101380 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e90146160 0x7f5e900b22d0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f5e800097d0 tx=0x7f5e8002b8a0 comp rx=0 tx=0).stop 2024-04-15T15:21:31.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.512+0000 7f5e949d4640 1 -- 172.21.15.105:0/709101380 shutdown_connections 2024-04-15T15:21:31.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.512+0000 7f5e949d4640 1 --2- 172.21.15.105:0/709101380 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e900b2810 0x7f5e900ccec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:31.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.512+0000 7f5e949d4640 1 --2- 172.21.15.105:0/709101380 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e90146160 0x7f5e900b22d0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:31.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.512+0000 7f5e949d4640 1 -- 172.21.15.105:0/709101380 >> 172.21.15.105:0/709101380 conn(0x7f5e9009aa40 msgr2=0x7f5e900b3140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:31.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.512+0000 7f5e949d4640 1 -- 172.21.15.105:0/709101380 shutdown_connections 2024-04-15T15:21:31.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.512+0000 7f5e949d4640 1 -- 172.21.15.105:0/709101380 wait complete. 2024-04-15T15:21:31.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.513+0000 7f5e949d4640 1 Processor -- start 2024-04-15T15:21:31.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.513+0000 7f5e949d4640 1 -- start start 2024-04-15T15:21:31.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.514+0000 7f5e949d4640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e900b2810 0x7f5e90154e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:31.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.514+0000 7f5e949d4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e900cd400 0x7f5e900b67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:31.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.514+0000 7f5e949d4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e90146160 0x7f5e900b6d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:31.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.514+0000 7f5e949d4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5e900cec00 con 0x7f5e90146160 2024-04-15T15:21:31.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.514+0000 7f5e949d4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5e9015d770 con 0x7f5e900cd400 2024-04-15T15:21:31.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.514+0000 7f5e949d4640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f5e9015dc90 con 0x7f5e900b2810 2024-04-15T15:21:31.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.514+0000 7f5e8ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e90146160 0x7f5e900b6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:31.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.514+0000 7f5e8ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e90146160 0x7f5e900b6d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51054/0 (socket says 172.21.15.105:51054) 2024-04-15T15:21:31.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.514+0000 7f5e8ed9d640 1 -- 172.21.15.105:0/2561157830 learned_addr learned my addr 172.21.15.105:0/2561157830 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:31.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.514+0000 7f5e8e59c640 1 --2- 172.21.15.105:0/2561157830 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e900b2810 0x7f5e90154e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:31.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.514+0000 7f5e85d9b640 1 --2- 172.21.15.105:0/2561157830 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e900cd400 0x7f5e900b67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:31.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.514+0000 7f5e8ed9d640 1 -- 172.21.15.105:0/2561157830 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e900b2810 msgr2=0x7f5e90154e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:31.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.514+0000 7f5e8ed9d640 1 --2- 172.21.15.105:0/2561157830 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e900b2810 0x7f5e90154e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:31.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.514+0000 7f5e8ed9d640 1 -- 172.21.15.105:0/2561157830 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e900cd400 msgr2=0x7f5e900b67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:31.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.514+0000 7f5e8ed9d640 1 --2- 172.21.15.105:0/2561157830 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e900cd400 0x7f5e900b67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:31.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.514+0000 7f5e8ed9d640 1 -- 172.21.15.105:0/2561157830 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e900b4020 con 0x7f5e90146160 2024-04-15T15:21:31.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.514+0000 7f5e8ed9d640 1 --2- 172.21.15.105:0/2561157830 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e90146160 0x7f5e900b6d10 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f5e8800cb20 tx=0x7f5e8800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:31.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.515+0000 7f5e877fe640 1 -- 172.21.15.105:0/2561157830 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e8800aca0 con 0x7f5e90146160 2024-04-15T15:21:31.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.515+0000 7f5e877fe640 1 -- 172.21.15.105:0/2561157830 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e88012400 con 0x7f5e90146160 2024-04-15T15:21:31.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.515+0000 7f5e877fe640 1 -- 172.21.15.105:0/2561157830 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e8800ae60 con 0x7f5e90146160 2024-04-15T15:21:31.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.515+0000 7f5e949d4640 1 -- 172.21.15.105:0/2561157830 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e900b42c0 con 0x7f5e90146160 2024-04-15T15:21:31.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.515+0000 7f5e949d4640 1 -- 172.21.15.105:0/2561157830 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e900b4880 con 0x7f5e90146160 2024-04-15T15:21:31.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.515+0000 7f5e949d4640 1 -- 172.21.15.105:0/2561157830 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5e90068a40 con 0x7f5e90146160 2024-04-15T15:21:31.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.516+0000 7f5e877fe640 1 -- 172.21.15.105:0/2561157830 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f5e880127f0 con 0x7f5e90146160 2024-04-15T15:21:31.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.517+0000 7f5e877fe640 1 --2- 172.21.15.105:0/2561157830 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f5e7006c980 0x7f5e7006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:31.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.517+0000 7f5e877fe640 1 -- 172.21.15.105:0/2561157830 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5029+0+0 (secure 0 0 0) 0x7f5e8808bed0 con 0x7f5e90146160 2024-04-15T15:21:31.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.517+0000 7f5e8e59c640 1 --2- 172.21.15.105:0/2561157830 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f5e7006c980 0x7f5e7006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:31.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.517+0000 7f5e877fe640 1 --2- 172.21.15.105:0/2561157830 >> [v2:172.21.15.154:6809/3920089903,v1:172.21.15.154:6811/3920089903] conn(0x7f5e700728e0 0x7f5e70074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:31.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.517+0000 7f5e877fe640 1 -- 172.21.15.105:0/2561157830 --> [v2:172.21.15.154:6809/3920089903,v1:172.21.15.154:6811/3920089903] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5e8800de50 con 0x7f5e700728e0 2024-04-15T15:21:31.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.517+0000 7f5e877fe640 1 -- 172.21.15.105:0/2561157830 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f5e8808e620 con 0x7f5e90146160 2024-04-15T15:21:31.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.517+0000 7f5e85d9b640 1 --2- 172.21.15.105:0/2561157830 >> [v2:172.21.15.154:6809/3920089903,v1:172.21.15.154:6811/3920089903] conn(0x7f5e700728e0 0x7f5e70074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:31.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.517+0000 7f5e8e59c640 1 --2- 172.21.15.105:0/2561157830 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f5e7006c980 0x7f5e7006ee40 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5e80036000 tx=0x7f5e80008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:31.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.517+0000 7f5e85d9b640 1 --2- 172.21.15.105:0/2561157830 >> [v2:172.21.15.154:6809/3920089903,v1:172.21.15.154:6811/3920089903] conn(0x7f5e700728e0 0x7f5e70074d00 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:31.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.518+0000 7f5e877fe640 1 -- 172.21.15.105:0/2561157830 <== osd.7 v2:172.21.15.154:6809/3920089903 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5e8800de50 con 0x7f5e700728e0 2024-04-15T15:21:31.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.531+0000 7f5e949d4640 1 -- 172.21.15.105:0/2561157830 --> [v2:172.21.15.154:6809/3920089903,v1:172.21.15.154:6811/3920089903] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f5e900ccfb0 con 0x7f5e700728e0 2024-04-15T15:21:31.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.532+0000 7f5e877fe640 1 -- 172.21.15.105:0/2561157830 <== osd.7 v2:172.21.15.154:6809/3920089903 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f5e900ccfb0 con 0x7f5e700728e0 2024-04-15T15:21:31.535 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-04-15T15:21:31.535 INFO:teuthology.orchestra.run.smithi105.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-15T15:21:31.535 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-04-15T15:21:31.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.532+0000 7f5e949d4640 1 -- 172.21.15.105:0/2561157830 >> [v2:172.21.15.154:6809/3920089903,v1:172.21.15.154:6811/3920089903] conn(0x7f5e700728e0 msgr2=0x7f5e70074d00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:31.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.532+0000 7f5e949d4640 1 --2- 172.21.15.105:0/2561157830 >> [v2:172.21.15.154:6809/3920089903,v1:172.21.15.154:6811/3920089903] conn(0x7f5e700728e0 0x7f5e70074d00 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:31.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.532+0000 7f5e949d4640 1 -- 172.21.15.105:0/2561157830 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f5e7006c980 msgr2=0x7f5e7006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:31.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.532+0000 7f5e949d4640 1 --2- 172.21.15.105:0/2561157830 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f5e7006c980 0x7f5e7006ee40 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5e80036000 tx=0x7f5e80008030 comp rx=0 tx=0).stop 2024-04-15T15:21:31.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.532+0000 7f5e949d4640 1 -- 172.21.15.105:0/2561157830 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e90146160 msgr2=0x7f5e900b6d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:31.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.532+0000 7f5e949d4640 1 --2- 172.21.15.105:0/2561157830 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e90146160 0x7f5e900b6d10 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f5e8800cb20 tx=0x7f5e8800a060 comp rx=0 tx=0).stop 2024-04-15T15:21:31.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.532+0000 7f5e8ed9d640 1 -- 172.21.15.105:0/2561157830 reap_dead start 2024-04-15T15:21:31.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.532+0000 7f5e949d4640 1 -- 172.21.15.105:0/2561157830 shutdown_connections 2024-04-15T15:21:31.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.532+0000 7f5e949d4640 1 -- 172.21.15.105:0/2561157830 >> 172.21.15.105:0/2561157830 conn(0x7f5e9009aa40 msgr2=0x7f5e901449f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:31.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.532+0000 7f5e949d4640 1 -- 172.21.15.105:0/2561157830 shutdown_connections 2024-04-15T15:21:31.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.532+0000 7f5e949d4640 1 -- 172.21.15.105:0/2561157830 wait complete. 2024-04-15T15:21:31.545 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-15T15:21:31.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.645+0000 7f4d7d4b3640 1 Processor -- start 2024-04-15T15:21:31.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.645+0000 7f4d7d4b3640 1 -- start start 2024-04-15T15:21:31.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.646+0000 7f4d7d4b3640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4d7815ec60 0x7f4d780d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:31.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.646+0000 7f4d7d4b3640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d780714f0 con 0x7f4d7809b150 2024-04-15T15:21:31.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.646+0000 7f4d7d4b3640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d780d1940 con 0x7f4d7809b780 2024-04-15T15:21:31.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.646+0000 7f4d7d4b3640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f4d7806cad0 con 0x7f4d7815ec60 2024-04-15T15:21:31.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.646+0000 7f4d777fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f4d7809b150 0x7f4d780cd840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:44000/0 (socket says 172.21.15.105:44000) 2024-04-15T15:21:31.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.646+0000 7f4d777fe640 1 -- 172.21.15.105:0/267589897 learned_addr learned my addr 172.21.15.105:0/267589897 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:21:31.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.646+0000 7f4d76ffd640 1 --2- 172.21.15.105:0/267589897 >> v2:172.21.15.154:3300/0 conn(0x7f4d7815ec60 0x7f4d780d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:31.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.646+0000 7f4d767fc640 1 -- 172.21.15.105:0/267589897 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2037964802 0 0) 0x7f4d780714f0 con 0x7f4d7809b150 2024-04-15T15:21:31.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.647+0000 7f4d767fc640 1 -- 172.21.15.105:0/267589897 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4d64003520 con 0x7f4d7809b150 2024-04-15T15:21:31.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.647+0000 7f4d767fc640 1 -- 172.21.15.105:0/267589897 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2663163842 0 0) 0x7f4d780d1940 con 0x7f4d7809b780 2024-04-15T15:21:31.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.647+0000 7f4d767fc640 1 -- 172.21.15.105:0/267589897 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4d780714f0 con 0x7f4d7809b780 2024-04-15T15:21:31.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.647+0000 7f4d76ffd640 1 -- 172.21.15.105:0/267589897 >> v1:172.21.15.154:6789/0 conn(0x7f4d7809b780 legacy=0x7f4d7809aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:31.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.647+0000 7f4d76ffd640 1 -- 172.21.15.105:0/267589897 >> v1:172.21.15.105:6789/0 conn(0x7f4d7809b150 legacy=0x7f4d780cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:31.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.647+0000 7f4d76ffd640 1 -- 172.21.15.105:0/267589897 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d780ccd60 con 0x7f4d7815ec60 2024-04-15T15:21:31.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.648+0000 7f4d76ffd640 1 --2- 172.21.15.105:0/267589897 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d7815ec60 0x7f4d780d1210 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4d680097a0 tx=0x7f4d6802b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3652d152950e745e server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:31.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.648+0000 7f4d767fc640 1 -- 172.21.15.105:0/267589897 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d68034070 con 0x7f4d7815ec60 2024-04-15T15:21:31.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.648+0000 7f4d767fc640 1 -- 172.21.15.105:0/267589897 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d68030610 con 0x7f4d7815ec60 2024-04-15T15:21:31.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.648+0000 7f4d767fc640 1 -- 172.21.15.105:0/267589897 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d680309c0 con 0x7f4d7815ec60 2024-04-15T15:21:31.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.648+0000 7f4d7d4b3640 1 -- 172.21.15.105:0/267589897 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d7815ec60 msgr2=0x7f4d780d1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:31.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.648+0000 7f4d7d4b3640 1 --2- 172.21.15.105:0/267589897 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d7815ec60 0x7f4d780d1210 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4d680097a0 tx=0x7f4d6802b960 comp rx=0 tx=0).stop 2024-04-15T15:21:31.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.648+0000 7f4d7d4b3640 1 -- 172.21.15.105:0/267589897 shutdown_connections 2024-04-15T15:21:31.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.648+0000 7f4d7d4b3640 1 --2- 172.21.15.105:0/267589897 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d7815ec60 0x7f4d780d1210 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:31.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.648+0000 7f4d7d4b3640 1 -- 172.21.15.105:0/267589897 >> 172.21.15.105:0/267589897 conn(0x7f4d780d2570 msgr2=0x7f4d780d2980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:31.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.648+0000 7f4d7d4b3640 1 -- 172.21.15.105:0/267589897 shutdown_connections 2024-04-15T15:21:31.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.648+0000 7f4d7d4b3640 1 -- 172.21.15.105:0/267589897 wait complete. 2024-04-15T15:21:31.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.649+0000 7f4d7d4b3640 1 Processor -- start 2024-04-15T15:21:31.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.650+0000 7f4d7d4b3640 1 -- start start 2024-04-15T15:21:31.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.650+0000 7f4d7d4b3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d7809b150 0x7f4d78152a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:31.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.650+0000 7f4d7d4b3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d7809b780 0x7f4d780b7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:31.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.650+0000 7f4d7d4b3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4d780ca000 0x7f4d780cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:31.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.650+0000 7f4d7d4b3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4d780cdf50 con 0x7f4d7809b150 2024-04-15T15:21:31.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.650+0000 7f4d7d4b3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4d780cd3b0 con 0x7f4d7809b780 2024-04-15T15:21:31.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.650+0000 7f4d7d4b3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f4d780ce0d0 con 0x7f4d780ca000 2024-04-15T15:21:31.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.650+0000 7f4d76ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d7809b150 0x7f4d78152a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:31.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.650+0000 7f4d76ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d7809b150 0x7f4d78152a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51056/0 (socket says 172.21.15.105:51056) 2024-04-15T15:21:31.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.650+0000 7f4d76ffd640 1 -- 172.21.15.105:0/1632831141 learned_addr learned my addr 172.21.15.105:0/1632831141 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:31.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.651+0000 7f4d6e7fc640 1 --2- 172.21.15.105:0/1632831141 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d7809b780 0x7f4d780b7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:31.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.651+0000 7f4d777fe640 1 --2- 172.21.15.105:0/1632831141 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4d780ca000 0x7f4d780cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:31.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.651+0000 7f4d76ffd640 1 -- 172.21.15.105:0/1632831141 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4d780ca000 msgr2=0x7f4d780cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:31.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.651+0000 7f4d76ffd640 1 --2- 172.21.15.105:0/1632831141 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4d780ca000 0x7f4d780cb7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:31.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.651+0000 7f4d76ffd640 1 -- 172.21.15.105:0/1632831141 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d7809b780 msgr2=0x7f4d780b7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:31.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.651+0000 7f4d76ffd640 1 --2- 172.21.15.105:0/1632831141 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d7809b780 0x7f4d780b7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:31.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.651+0000 7f4d76ffd640 1 -- 172.21.15.105:0/1632831141 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d78152f50 con 0x7f4d7809b150 2024-04-15T15:21:31.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.651+0000 7f4d76ffd640 1 --2- 172.21.15.105:0/1632831141 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d7809b150 0x7f4d78152a10 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f4d68000c00 tx=0x7f4d680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:31.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.651+0000 7f4d6ffff640 1 -- 172.21.15.105:0/1632831141 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d68034040 con 0x7f4d7809b150 2024-04-15T15:21:31.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.651+0000 7f4d6ffff640 1 -- 172.21.15.105:0/1632831141 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d6803aaa0 con 0x7f4d7809b150 2024-04-15T15:21:31.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.651+0000 7f4d7d4b3640 1 -- 172.21.15.105:0/1632831141 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d780ca4e0 con 0x7f4d7809b150 2024-04-15T15:21:31.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.651+0000 7f4d6ffff640 1 -- 172.21.15.105:0/1632831141 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d68006e10 con 0x7f4d7809b150 2024-04-15T15:21:31.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.651+0000 7f4d7d4b3640 1 -- 172.21.15.105:0/1632831141 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d780caaa0 con 0x7f4d7809b150 2024-04-15T15:21:31.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.652+0000 7f4d7d4b3640 1 -- 172.21.15.105:0/1632831141 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d78068a40 con 0x7f4d7809b150 2024-04-15T15:21:31.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.653+0000 7f4d6ffff640 1 -- 172.21.15.105:0/1632831141 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f4d68037070 con 0x7f4d7809b150 2024-04-15T15:21:31.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.653+0000 7f4d6ffff640 1 --2- 172.21.15.105:0/1632831141 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f4d5806c980 0x7f4d5806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:31.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.653+0000 7f4d6e7fc640 1 --2- 172.21.15.105:0/1632831141 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f4d5806c980 0x7f4d5806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:31.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.653+0000 7f4d6ffff640 1 -- 172.21.15.105:0/1632831141 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5029+0+0 (secure 0 0 0) 0x7f4d680ad660 con 0x7f4d7809b150 2024-04-15T15:21:31.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.654+0000 7f4d6e7fc640 1 --2- 172.21.15.105:0/1632831141 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f4d5806c980 0x7f4d5806ee40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4d60007fd0 tx=0x7f4d60009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:31.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.658+0000 7f4d6ffff640 1 -- 172.21.15.105:0/1632831141 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4d78152f50 con 0x7f4d7809b150 2024-04-15T15:21:31.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.836+0000 7f4d7d4b3640 1 -- 172.21.15.105:0/1632831141 --> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f4d7806f070 con 0x7f4d5806c980 2024-04-15T15:21:31.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.837+0000 7f4d6ffff640 1 -- 172.21.15.105:0/1632831141 <== mgr.4107 v2:172.21.15.105:6832/3506549322 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f4d7806f070 con 0x7f4d5806c980 2024-04-15T15:21:31.840 INFO:teuthology.orchestra.run.smithi105.stderr:instructed osd(s) 7 to deep-scrub 2024-04-15T15:21:31.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.839+0000 7f4d7d4b3640 1 -- 172.21.15.105:0/1632831141 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f4d5806c980 msgr2=0x7f4d5806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:31.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.839+0000 7f4d7d4b3640 1 --2- 172.21.15.105:0/1632831141 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f4d5806c980 0x7f4d5806ee40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4d60007fd0 tx=0x7f4d60009040 comp rx=0 tx=0).stop 2024-04-15T15:21:31.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.839+0000 7f4d7d4b3640 1 -- 172.21.15.105:0/1632831141 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d7809b150 msgr2=0x7f4d78152a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:31.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.839+0000 7f4d7d4b3640 1 --2- 172.21.15.105:0/1632831141 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d7809b150 0x7f4d78152a10 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f4d68000c00 tx=0x7f4d680046c0 comp rx=0 tx=0).stop 2024-04-15T15:21:31.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.839+0000 7f4d7d4b3640 1 -- 172.21.15.105:0/1632831141 shutdown_connections 2024-04-15T15:21:31.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.839+0000 7f4d7d4b3640 1 --2- 172.21.15.105:0/1632831141 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4d780ca000 0x7f4d780cb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:31.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.839+0000 7f4d7d4b3640 1 --2- 172.21.15.105:0/1632831141 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f4d5806c980 0x7f4d5806ee40 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:31.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.839+0000 7f4d7d4b3640 1 --2- 172.21.15.105:0/1632831141 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d7809b780 0x7f4d780b7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:31.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.839+0000 7f4d7d4b3640 1 --2- 172.21.15.105:0/1632831141 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d7809b150 0x7f4d78152a10 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:31.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.839+0000 7f4d7d4b3640 1 -- 172.21.15.105:0/1632831141 >> 172.21.15.105:0/1632831141 conn(0x7f4d780d2570 msgr2=0x7f4d780d06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:31.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.839+0000 7f4d7d4b3640 1 -- 172.21.15.105:0/1632831141 shutdown_connections 2024-04-15T15:21:31.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.839+0000 7f4d7d4b3640 1 -- 172.21.15.105:0/1632831141 wait complete. 2024-04-15T15:21:31.853 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-15T15:21:31.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.952+0000 7f069f1e8640 1 Processor -- start 2024-04-15T15:21:31.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.953+0000 7f069f1e8640 1 -- start start 2024-04-15T15:21:31.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.953+0000 7f069f1e8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f069815eb60 0x7f06980d11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:31.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.953+0000 7f069f1e8640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06980d1b30 con 0x7f0698162d40 2024-04-15T15:21:31.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.953+0000 7f069f1e8640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f069815ef40 con 0x7f06980d1710 2024-04-15T15:21:31.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.953+0000 7f069f1e8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f0698163540 con 0x7f069815eb60 2024-04-15T15:21:31.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.954+0000 7f069d785640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f0698162d40 0x7f06980d3450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:44004/0 (socket says 172.21.15.105:44004) 2024-04-15T15:21:31.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.954+0000 7f069d785640 1 -- 172.21.15.105:0/2067668136 learned_addr learned my addr 172.21.15.105:0/2067668136 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:21:31.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.954+0000 7f069cf84640 1 --2- 172.21.15.105:0/2067668136 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f069815eb60 0x7f06980d11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:31.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.954+0000 7f06977fe640 1 -- 172.21.15.105:0/2067668136 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3755275602 0 0) 0x7f06980d1b30 con 0x7f0698162d40 2024-04-15T15:21:31.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.954+0000 7f06977fe640 1 -- 172.21.15.105:0/2067668136 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0684003520 con 0x7f0698162d40 2024-04-15T15:21:31.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.954+0000 7f06977fe640 1 -- 172.21.15.105:0/2067668136 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2109867172 0 0) 0x7f0684003520 con 0x7f0698162d40 2024-04-15T15:21:31.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.954+0000 7f06977fe640 1 -- 172.21.15.105:0/2067668136 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f069815eb60 msgr2=0x7f06980d11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:31.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.954+0000 7f06977fe640 1 --2- 172.21.15.105:0/2067668136 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f069815eb60 0x7f06980d11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:31.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.954+0000 7f06977fe640 1 -- 172.21.15.105:0/2067668136 >> v1:172.21.15.154:6789/0 conn(0x7f06980d1710 legacy=0x7f0698162630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:31.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.954+0000 7f06977fe640 1 -- 172.21.15.105:0/2067668136 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0698163260 con 0x7f0698162d40 2024-04-15T15:21:31.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.954+0000 7f06977fe640 1 -- 172.21.15.105:0/2067668136 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3607114035 0 0) 0x7f0690001670 con 0x7f0698162d40 2024-04-15T15:21:31.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.954+0000 7f06977fe640 1 -- 172.21.15.105:0/2067668136 >> v1:172.21.15.105:6789/0 conn(0x7f0698162d40 legacy=0x7f06980d3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:31.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.954+0000 7f06977fe640 1 --2- 172.21.15.105:0/2067668136 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06840039e0 0x7f0684003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:31.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.954+0000 7f06977fe640 1 --2- 172.21.15.105:0/2067668136 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f069815eb60 0x7f0684004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:31.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.954+0000 7f06977fe640 1 --2- 172.21.15.105:0/2067668136 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0684004d40 0x7f06840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:31.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.954+0000 7f06977fe640 1 -- 172.21.15.105:0/2067668136 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0698163540 con 0x7f06840039e0 2024-04-15T15:21:31.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.954+0000 7f06977fe640 1 -- 172.21.15.105:0/2067668136 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f068400ba70 con 0x7f0684004d40 2024-04-15T15:21:31.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.954+0000 7f06977fe640 1 -- 172.21.15.105:0/2067668136 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f068400bc70 con 0x7f069815eb60 2024-04-15T15:21:31.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.954+0000 7f069cf84640 1 --2- 172.21.15.105:0/2067668136 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06840039e0 0x7f0684003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:31.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.954+0000 7f069d785640 1 --2- 172.21.15.105:0/2067668136 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0684004d40 0x7f06840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:31.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.954+0000 7f0697fff640 1 --2- 172.21.15.105:0/2067668136 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f069815eb60 0x7f0684004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:31.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.954+0000 7f069cf84640 1 -- 172.21.15.105:0/2067668136 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f069815eb60 msgr2=0x7f0684004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:31.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.954+0000 7f069cf84640 1 --2- 172.21.15.105:0/2067668136 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f069815eb60 0x7f0684004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:31.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.954+0000 7f069cf84640 1 -- 172.21.15.105:0/2067668136 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0684004d40 msgr2=0x7f06840051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:31.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.954+0000 7f069cf84640 1 --2- 172.21.15.105:0/2067668136 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0684004d40 0x7f06840051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:31.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.954+0000 7f069cf84640 1 -- 172.21.15.105:0/2067668136 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0684003520 con 0x7f06840039e0 2024-04-15T15:21:31.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.954+0000 7f069d785640 1 --2- 172.21.15.105:0/2067668136 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0684004d40 0x7f06840051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:21:31.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.955+0000 7f069cf84640 1 --2- 172.21.15.105:0/2067668136 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06840039e0 0x7f0684003e00 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f0688009f90 tx=0x7f0684005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee78624a891c4fc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:31.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.955+0000 7f06977fe640 1 -- 172.21.15.105:0/2067668136 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0688039070 con 0x7f06840039e0 2024-04-15T15:21:31.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.955+0000 7f06977fe640 1 -- 172.21.15.105:0/2067668136 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f068802cbd0 con 0x7f06840039e0 2024-04-15T15:21:31.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.955+0000 7f06977fe640 1 -- 172.21.15.105:0/2067668136 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0688009c10 con 0x7f06840039e0 2024-04-15T15:21:31.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.956+0000 7f069f1e8640 1 -- 172.21.15.105:0/2067668136 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06840039e0 msgr2=0x7f0684003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:31.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.956+0000 7f069f1e8640 1 --2- 172.21.15.105:0/2067668136 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06840039e0 0x7f0684003e00 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f0688009f90 tx=0x7f0684005830 comp rx=0 tx=0).stop 2024-04-15T15:21:31.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.956+0000 7f069d785640 1 -- 172.21.15.105:0/2067668136 reap_dead start 2024-04-15T15:21:31.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.956+0000 7f069f1e8640 1 -- 172.21.15.105:0/2067668136 shutdown_connections 2024-04-15T15:21:31.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.956+0000 7f069f1e8640 1 -- 172.21.15.105:0/2067668136 >> 172.21.15.105:0/2067668136 conn(0x7f0698067720 msgr2=0x7f0698067b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:31.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.956+0000 7f069f1e8640 1 -- 172.21.15.105:0/2067668136 shutdown_connections 2024-04-15T15:21:31.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.956+0000 7f069f1e8640 1 -- 172.21.15.105:0/2067668136 wait complete. 2024-04-15T15:21:31.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.957+0000 7f069f1e8640 1 Processor -- start 2024-04-15T15:21:31.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.957+0000 7f069f1e8640 1 -- start start 2024-04-15T15:21:31.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.958+0000 7f069f1e8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f06840039e0 0x7f069815bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:31.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.958+0000 7f069f1e8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f069814fa60 0x7f069814cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:31.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.958+0000 7f069f1e8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f069815f110 0x7f069815ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:31.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.958+0000 7f069f1e8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f06980d2c50 con 0x7f069815f110 2024-04-15T15:21:31.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.958+0000 7f069f1e8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f06980d2fc0 con 0x7f069814fa60 2024-04-15T15:21:31.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.958+0000 7f069f1e8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f06980b78f0 con 0x7f06840039e0 2024-04-15T15:21:31.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.958+0000 7f069d785640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f069815f110 0x7f069815ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:31.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.958+0000 7f069d785640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f069815f110 0x7f069815ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51080/0 (socket says 172.21.15.105:51080) 2024-04-15T15:21:31.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.958+0000 7f069d785640 1 -- 172.21.15.105:0/1681533906 learned_addr learned my addr 172.21.15.105:0/1681533906 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:31.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.958+0000 7f0697fff640 1 --2- 172.21.15.105:0/1681533906 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f069814fa60 0x7f069814cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:31.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.958+0000 7f069cf84640 1 --2- 172.21.15.105:0/1681533906 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f06840039e0 0x7f069815bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:31.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.958+0000 7f069d785640 1 -- 172.21.15.105:0/1681533906 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f06840039e0 msgr2=0x7f069815bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:31.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.958+0000 7f069d785640 1 --2- 172.21.15.105:0/1681533906 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f06840039e0 0x7f069815bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:31.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.958+0000 7f069d785640 1 -- 172.21.15.105:0/1681533906 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f069814fa60 msgr2=0x7f069814cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:31.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.958+0000 7f069d785640 1 --2- 172.21.15.105:0/1681533906 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f069814fa60 0x7f069814cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:31.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.958+0000 7f069d785640 1 -- 172.21.15.105:0/1681533906 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f069814efa0 con 0x7f069815f110 2024-04-15T15:21:31.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.958+0000 7f0697fff640 1 --2- 172.21.15.105:0/1681533906 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f069814fa60 0x7f069814cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:21:31.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.959+0000 7f069d785640 1 --2- 172.21.15.105:0/1681533906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f069815f110 0x7f069815ff00 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f0690001cd0 tx=0x7f069000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:31.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.959+0000 7f06957fa640 1 -- 172.21.15.105:0/1681533906 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069000c5b0 con 0x7f069815f110 2024-04-15T15:21:31.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.959+0000 7f06957fa640 1 -- 172.21.15.105:0/1681533906 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0690018070 con 0x7f069815f110 2024-04-15T15:21:31.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.959+0000 7f069f1e8640 1 -- 172.21.15.105:0/1681533906 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f069815f660 con 0x7f069815f110 2024-04-15T15:21:31.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.959+0000 7f06957fa640 1 -- 172.21.15.105:0/1681533906 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069000c990 con 0x7f069815f110 2024-04-15T15:21:31.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.959+0000 7f069f1e8640 1 -- 172.21.15.105:0/1681533906 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f069814f540 con 0x7f069815f110 2024-04-15T15:21:31.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.960+0000 7f069f1e8640 1 -- 172.21.15.105:0/1681533906 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06980cad10 con 0x7f069815f110 2024-04-15T15:21:31.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.960+0000 7f06957fa640 1 -- 172.21.15.105:0/1681533906 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f069000cb30 con 0x7f069815f110 2024-04-15T15:21:31.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.961+0000 7f06957fa640 1 --2- 172.21.15.105:0/1681533906 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f067806c980 0x7f067806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:31.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.961+0000 7f06957fa640 1 -- 172.21.15.105:0/1681533906 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5029+0+0 (secure 0 0 0) 0x7f069008b500 con 0x7f069815f110 2024-04-15T15:21:31.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.961+0000 7f069cf84640 1 --2- 172.21.15.105:0/1681533906 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f067806c980 0x7f067806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:31.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.961+0000 7f069cf84640 1 --2- 172.21.15.105:0/1681533906 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f067806c980 0x7f067806ee40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0688036000 tx=0x7f068802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:31.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:31.965+0000 7f06957fa640 1 -- 172.21.15.105:0/1681533906 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f06980cad10 con 0x7f069815f110 2024-04-15T15:21:32.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:32.138+0000 7f069f1e8640 1 -- 172.21.15.105:0/1681533906 --> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0698068a40 con 0x7f067806c980 2024-04-15T15:21:32.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:32.139+0000 7f06957fa640 1 -- 172.21.15.105:0/1681533906 <== mgr.4107 v2:172.21.15.105:6832/3506549322 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13422 (secure 0 0 0) 0x7f0698068a40 con 0x7f067806c980 2024-04-15T15:21:32.142 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:21:32.143 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-04-15T15:21:32.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:32.141+0000 7f069f1e8640 1 -- 172.21.15.105:0/1681533906 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f067806c980 msgr2=0x7f067806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:32.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:32.141+0000 7f069f1e8640 1 --2- 172.21.15.105:0/1681533906 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f067806c980 0x7f067806ee40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0688036000 tx=0x7f068802e000 comp rx=0 tx=0).stop 2024-04-15T15:21:32.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:32.141+0000 7f069f1e8640 1 -- 172.21.15.105:0/1681533906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f069815f110 msgr2=0x7f069815ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:32.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:32.141+0000 7f069f1e8640 1 --2- 172.21.15.105:0/1681533906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f069815f110 0x7f069815ff00 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f0690001cd0 tx=0x7f069000ec00 comp rx=0 tx=0).stop 2024-04-15T15:21:32.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:32.141+0000 7f069f1e8640 1 -- 172.21.15.105:0/1681533906 shutdown_connections 2024-04-15T15:21:32.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:32.141+0000 7f069f1e8640 1 --2- 172.21.15.105:0/1681533906 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f067806c980 0x7f067806ee40 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:32.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:32.141+0000 7f069f1e8640 1 --2- 172.21.15.105:0/1681533906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f069815f110 0x7f069815ff00 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:32.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:32.141+0000 7f069f1e8640 1 --2- 172.21.15.105:0/1681533906 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f069814fa60 0x7f069814cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:32.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:32.141+0000 7f069f1e8640 1 --2- 172.21.15.105:0/1681533906 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f06840039e0 0x7f069815bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:32.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:32.141+0000 7f069f1e8640 1 -- 172.21.15.105:0/1681533906 >> 172.21.15.105:0/1681533906 conn(0x7f0698067720 msgr2=0x7f06980c9740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:32.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:32.142+0000 7f069f1e8640 1 -- 172.21.15.105:0/1681533906 shutdown_connections 2024-04-15T15:21:32.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:32.142+0000 7f069f1e8640 1 -- 172.21.15.105:0/1681533906 wait complete. 2024-04-15T15:21:32.155 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":100,"stamp":"2024-04-15T15:21:31.758014+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":220792,"kb_used_data":4616,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753928,"statfs":{"total":773094113280,"available":772868022272,"internally_reserved":0,"allocated":4726784,"data_stored":2588024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-133,"num_objects":-1,"num_object_clones":0,"num_object_copies":-2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-1,"num_write_kb":-1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.786235"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":206,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-15T15:21:24.295342+0000","last_change":"2024-04-15T15:19:14.878269+0000","last_active":"2024-04-15T15:21:24.295342+0000","last_peered":"2024-04-15T15:21:24.295342+0000","last_clean":"2024-04-15T15:21:24.295342+0000","last_became_active":"2024-04-15T15:19:14.877367+0000","last_became_peered":"2024-04-15T15:19:14.877367+0000","last_unstale":"2024-04-15T15:21:24.295342+0000","last_undegraded":"2024-04-15T15:21:24.295342+0000","last_fullsized":"2024-04-15T15:21:24.295342+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:13.827511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:13.827511+0000","last_clean_scrub_stamp":"2024-04-15T15:19:13.827511+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:32:55.902561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574878,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343808,"statfs":{"total":96636764160,"available":96608059392,"internally_reserved":0,"allocated":1036288,"data_stored":766279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27456,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":442368,"data_stored":175911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27452,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":442368,"data_stored":175911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574878,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27452,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":442368,"data_stored":175911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27456,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":442368,"data_stored":175911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27456,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":442368,"data_stored":175911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27452,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":442368,"data_stored":175911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574879,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343804,"statfs":{"total":96636764160,"available":96608055296,"internally_reserved":0,"allocated":1036288,"data_stored":766279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-15T15:21:32.157 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-15T15:19:13.827511+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=15, tm_hour=15, tm_min=19, tm_sec=13, tm_wday=0, tm_yday=106, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=15, tm_hour=15, tm_min=21, tm_sec=27, tm_wday=0, tm_yday=106, tm_isdst=0) 2024-04-15T15:21:32.158 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-15T15:21:52.158 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-15T15:21:52.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.272+0000 7f4690531640 1 Processor -- start 2024-04-15T15:21:52.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.272+0000 7f4690531640 1 -- start start 2024-04-15T15:21:52.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.273+0000 7f4690531640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f468815eb60 0x7f46880d11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:52.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.273+0000 7f4690531640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f46880d1710 0x7f4688162630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:52.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.273+0000 7f4690531640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4688162b70 0x7f46880b6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:52.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.273+0000 7f4690531640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f46880d1b60 con 0x7f468815eb60 2024-04-15T15:21:52.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.273+0000 7f4690531640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f4688163540 con 0x7f46880d1710 2024-04-15T15:21:52.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.273+0000 7f4690531640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f46880d2fc0 con 0x7f4688162b70 2024-04-15T15:21:52.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.274+0000 7f468e2cd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f468815eb60 0x7f46880d11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:52.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.274+0000 7f468e2cd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f468815eb60 0x7f46880d11d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38690/0 (socket says 172.21.15.105:38690) 2024-04-15T15:21:52.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.274+0000 7f468e2cd640 1 -- 172.21.15.105:0/3751318891 learned_addr learned my addr 172.21.15.105:0/3751318891 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:52.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.274+0000 7f468eace640 1 --2- 172.21.15.105:0/3751318891 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4688162b70 0x7f46880b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:52.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.274+0000 7f468e2cd640 1 -- 172.21.15.105:0/3751318891 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4688162b70 msgr2=0x7f46880b6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:52.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.274+0000 7f468e2cd640 1 --2- 172.21.15.105:0/3751318891 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4688162b70 0x7f46880b6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:52.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.274+0000 7f468dacc640 1 --2- 172.21.15.105:0/3751318891 >> v2:172.21.15.154:3300/0 conn(0x7f46880d1710 0x7f4688162630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:52.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.274+0000 7f468e2cd640 1 -- 172.21.15.105:0/3751318891 >> v2:172.21.15.154:3300/0 conn(0x7f46880d1710 msgr2=0x7f4688162630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:52.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.274+0000 7f468e2cd640 1 --2- 172.21.15.105:0/3751318891 >> v2:172.21.15.154:3300/0 conn(0x7f46880d1710 0x7f4688162630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:52.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.274+0000 7f468e2cd640 1 -- 172.21.15.105:0/3751318891 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f468815ef40 con 0x7f468815eb60 2024-04-15T15:21:52.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.274+0000 7f468eace640 1 --2- 172.21.15.105:0/3751318891 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4688162b70 0x7f46880b6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:21:52.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.274+0000 7f468dacc640 1 --2- 172.21.15.105:0/3751318891 >> v2:172.21.15.154:3300/0 conn(0x7f46880d1710 0x7f4688162630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:21:52.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.275+0000 7f468e2cd640 1 --2- 172.21.15.105:0/3751318891 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f468815eb60 0x7f46880d11d0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f46880b7680 tx=0x7f467802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f246cf5e44a252d server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:52.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.275+0000 7f468d2cb640 1 -- 172.21.15.105:0/3751318891 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f467802ca70 con 0x7f468815eb60 2024-04-15T15:21:52.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.275+0000 7f468d2cb640 1 -- 172.21.15.105:0/3751318891 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f467802cc10 con 0x7f468815eb60 2024-04-15T15:21:52.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.275+0000 7f468d2cb640 1 -- 172.21.15.105:0/3751318891 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46780343f0 con 0x7f468815eb60 2024-04-15T15:21:52.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.275+0000 7f4690531640 1 -- 172.21.15.105:0/3751318891 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f468815eb60 msgr2=0x7f46880d11d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:52.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.275+0000 7f4690531640 1 --2- 172.21.15.105:0/3751318891 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f468815eb60 0x7f46880d11d0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f46880b7680 tx=0x7f467802bed0 comp rx=0 tx=0).stop 2024-04-15T15:21:52.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.275+0000 7f4690531640 1 -- 172.21.15.105:0/3751318891 shutdown_connections 2024-04-15T15:21:52.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.275+0000 7f4690531640 1 --2- 172.21.15.105:0/3751318891 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4688162b70 0x7f46880b6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:52.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.275+0000 7f4690531640 1 --2- 172.21.15.105:0/3751318891 >> v2:172.21.15.154:3300/0 conn(0x7f46880d1710 0x7f4688162630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:52.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.275+0000 7f4690531640 1 --2- 172.21.15.105:0/3751318891 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f468815eb60 0x7f46880d11d0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:52.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.275+0000 7f4690531640 1 -- 172.21.15.105:0/3751318891 >> 172.21.15.105:0/3751318891 conn(0x7f4688067720 msgr2=0x7f4688067b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:52.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.275+0000 7f4690531640 1 -- 172.21.15.105:0/3751318891 shutdown_connections 2024-04-15T15:21:52.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.275+0000 7f4690531640 1 -- 172.21.15.105:0/3751318891 wait complete. 2024-04-15T15:21:52.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.277+0000 7f4690531640 1 Processor -- start 2024-04-15T15:21:52.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.277+0000 7f4690531640 1 -- start start 2024-04-15T15:21:52.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.277+0000 7f4690531640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46880d1710 0x7f46880b50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:52.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.277+0000 7f4690531640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f468815eb60 0x7f4688154660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:52.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.277+0000 7f4690531640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4688162b70 0x7f46880ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:52.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.277+0000 7f4690531640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f46880d2970 con 0x7f468815eb60 2024-04-15T15:21:52.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.277+0000 7f4690531640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4688163260 con 0x7f46880d1710 2024-04-15T15:21:52.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.277+0000 7f4690531640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f46880d2c50 con 0x7f4688162b70 2024-04-15T15:21:52.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.277+0000 7f468dacc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f468815eb60 0x7f4688154660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:52.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.277+0000 7f468dacc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f468815eb60 0x7f4688154660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38702/0 (socket says 172.21.15.105:38702) 2024-04-15T15:21:52.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.278+0000 7f468dacc640 1 -- 172.21.15.105:0/4201100582 learned_addr learned my addr 172.21.15.105:0/4201100582 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:52.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.278+0000 7f468eace640 1 --2- 172.21.15.105:0/4201100582 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4688162b70 0x7f46880ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:52.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.278+0000 7f468e2cd640 1 --2- 172.21.15.105:0/4201100582 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46880d1710 0x7f46880b50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:52.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.278+0000 7f468dacc640 1 -- 172.21.15.105:0/4201100582 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4688162b70 msgr2=0x7f46880ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:52.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.278+0000 7f468dacc640 1 --2- 172.21.15.105:0/4201100582 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4688162b70 0x7f46880ccfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:52.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.278+0000 7f468dacc640 1 -- 172.21.15.105:0/4201100582 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46880d1710 msgr2=0x7f46880b50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:52.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.278+0000 7f468dacc640 1 --2- 172.21.15.105:0/4201100582 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46880d1710 0x7f46880b50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:52.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.278+0000 7f468dacc640 1 -- 172.21.15.105:0/4201100582 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46881603b0 con 0x7f468815eb60 2024-04-15T15:21:52.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.278+0000 7f468e2cd640 1 --2- 172.21.15.105:0/4201100582 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46880d1710 0x7f46880b50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:21:52.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.278+0000 7f468eace640 1 --2- 172.21.15.105:0/4201100582 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4688162b70 0x7f46880ccfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:21:52.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.278+0000 7f468dacc640 1 --2- 172.21.15.105:0/4201100582 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f468815eb60 0x7f4688154660 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f467c00c7f0 tx=0x7f467c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:52.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.278+0000 7f4686ffd640 1 -- 172.21.15.105:0/4201100582 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f467c015070 con 0x7f468815eb60 2024-04-15T15:21:52.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.278+0000 7f4686ffd640 1 -- 172.21.15.105:0/4201100582 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f467c007820 con 0x7f468815eb60 2024-04-15T15:21:52.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.278+0000 7f4686ffd640 1 -- 172.21.15.105:0/4201100582 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f467c007c10 con 0x7f468815eb60 2024-04-15T15:21:52.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.278+0000 7f4690531640 1 -- 172.21.15.105:0/4201100582 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4688160690 con 0x7f468815eb60 2024-04-15T15:21:52.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.278+0000 7f4690531640 1 -- 172.21.15.105:0/4201100582 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4688160c50 con 0x7f468815eb60 2024-04-15T15:21:52.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.279+0000 7f4690531640 1 -- 172.21.15.105:0/4201100582 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4688068a40 con 0x7f468815eb60 2024-04-15T15:21:52.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.280+0000 7f4686ffd640 1 -- 172.21.15.105:0/4201100582 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f467c018070 con 0x7f468815eb60 2024-04-15T15:21:52.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.280+0000 7f4686ffd640 1 --2- 172.21.15.105:0/4201100582 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f467406c980 0x7f467406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:52.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.280+0000 7f4686ffd640 1 -- 172.21.15.105:0/4201100582 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5029+0+0 (secure 0 0 0) 0x7f467c08bb10 con 0x7f468815eb60 2024-04-15T15:21:52.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.280+0000 7f468e2cd640 1 --2- 172.21.15.105:0/4201100582 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f467406c980 0x7f467406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:52.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.281+0000 7f468e2cd640 1 --2- 172.21.15.105:0/4201100582 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f467406c980 0x7f467406ee40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4678035000 tx=0x7f4678030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:52.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.285+0000 7f4686ffd640 1 -- 172.21.15.105:0/4201100582 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f467c05a0f0 con 0x7f468815eb60 2024-04-15T15:21:52.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.458+0000 7f4690531640 1 -- 172.21.15.105:0/4201100582 --> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4688160160 con 0x7f467406c980 2024-04-15T15:21:52.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.458+0000 7f4686ffd640 1 -- 172.21.15.105:0/4201100582 <== mgr.4107 v2:172.21.15.105:6832/3506549322 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13359 (secure 0 0 0) 0x7f4688160160 con 0x7f467406c980 2024-04-15T15:21:52.461 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:21:52.461 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-04-15T15:21:52.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.460+0000 7f4690531640 1 -- 172.21.15.105:0/4201100582 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f467406c980 msgr2=0x7f467406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:52.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.460+0000 7f4690531640 1 --2- 172.21.15.105:0/4201100582 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f467406c980 0x7f467406ee40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4678035000 tx=0x7f4678030000 comp rx=0 tx=0).stop 2024-04-15T15:21:52.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.460+0000 7f4690531640 1 -- 172.21.15.105:0/4201100582 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f468815eb60 msgr2=0x7f4688154660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:52.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.460+0000 7f4690531640 1 --2- 172.21.15.105:0/4201100582 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f468815eb60 0x7f4688154660 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f467c00c7f0 tx=0x7f467c00ccb0 comp rx=0 tx=0).stop 2024-04-15T15:21:52.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.460+0000 7f4690531640 1 -- 172.21.15.105:0/4201100582 shutdown_connections 2024-04-15T15:21:52.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.460+0000 7f4690531640 1 --2- 172.21.15.105:0/4201100582 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4688162b70 0x7f46880ccfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:52.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.460+0000 7f4690531640 1 --2- 172.21.15.105:0/4201100582 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f467406c980 0x7f467406ee40 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:52.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.460+0000 7f4690531640 1 --2- 172.21.15.105:0/4201100582 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f468815eb60 0x7f4688154660 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:52.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.460+0000 7f4690531640 1 --2- 172.21.15.105:0/4201100582 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46880d1710 0x7f46880b50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:52.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.461+0000 7f4690531640 1 -- 172.21.15.105:0/4201100582 >> 172.21.15.105:0/4201100582 conn(0x7f4688067720 msgr2=0x7f46880c82d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:52.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.461+0000 7f4690531640 1 -- 172.21.15.105:0/4201100582 shutdown_connections 2024-04-15T15:21:52.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.461+0000 7f4690531640 1 -- 172.21.15.105:0/4201100582 wait complete. 2024-04-15T15:21:52.473 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":110,"stamp":"2024-04-15T15:21:51.762353+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":220780,"kb_used_data":4616,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753940,"statfs":{"total":773094113280,"available":772868034560,"internally_reserved":0,"allocated":4726784,"data_stored":2588024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002820"},"pg_stats":[{"pgid":"1.0","version":"38'119","reported_seq":217,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-15T15:21:32.138461+0000","last_change":"2024-04-15T15:21:32.138499+0000","last_active":"2024-04-15T15:21:32.138461+0000","last_peered":"2024-04-15T15:21:32.138461+0000","last_clean":"2024-04-15T15:21:32.138461+0000","last_became_active":"2024-04-15T15:19:14.877367+0000","last_became_peered":"2024-04-15T15:19:14.877367+0000","last_unstale":"2024-04-15T15:21:32.138461+0000","last_undegraded":"2024-04-15T15:21:32.138461+0000","last_fullsized":"2024-04-15T15:21:32.138461+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"38'119","last_scrub_stamp":"2024-04-15T15:21:32.138372+0000","last_deep_scrub":"38'119","last_deep_scrub_stamp":"2024-04-15T15:21:32.138372+0000","last_clean_scrub_stamp":"2024-04-15T15:21:32.138372+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:07:12.080362+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574882,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343812,"statfs":{"total":96636764160,"available":96608063488,"internally_reserved":0,"allocated":1036288,"data_stored":766279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574884,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27456,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":442368,"data_stored":175911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574883,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27452,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":442368,"data_stored":175911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574882,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27452,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":442368,"data_stored":175911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574883,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27456,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":442368,"data_stored":175911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574884,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27456,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":442368,"data_stored":175911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574883,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27452,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":442368,"data_stored":175911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574883,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343812,"statfs":{"total":96636764160,"available":96608063488,"internally_reserved":0,"allocated":1036288,"data_stored":766279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-15T15:21:52.474 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-15T15:21:52.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.574+0000 7f56d9b9d640 1 Processor -- start 2024-04-15T15:21:52.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.575+0000 7f56d9b9d640 1 -- start start 2024-04-15T15:21:52.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.575+0000 7f56d9b9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f56d40ba480 0x7f56d40d1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:52.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.575+0000 7f56d9b9d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f56d40d1650 0x7f56d40b7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:52.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.575+0000 7f56d9b9d640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56d40d3400 con 0x7f56d415e8d0 2024-04-15T15:21:52.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.575+0000 7f56d9b9d640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f56d40d31c0 con 0x7f56d40ba480 2024-04-15T15:21:52.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.575+0000 7f56d9b9d640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f56d40d1aa0 con 0x7f56d40d1650 2024-04-15T15:21:52.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.575+0000 7f56caffd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f56d40ba480 0x7f56d40d1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:52.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.576+0000 7f56d37fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f56d415e8d0 0x7f56d40d2ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:57332/0 (socket says 172.21.15.105:57332) 2024-04-15T15:21:52.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.576+0000 7f56d37fe640 1 -- 172.21.15.105:0/110893837 learned_addr learned my addr 172.21.15.105:0/110893837 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-15T15:21:52.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.576+0000 7f56d3fff640 1 --2- 172.21.15.105:0/110893837 >> v2:172.21.15.154:3300/0 conn(0x7f56d40d1650 0x7f56d40b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:52.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.576+0000 7f56d2ffd640 1 -- 172.21.15.105:0/110893837 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 9564911 0 0) 0x7f56d40d3400 con 0x7f56d415e8d0 2024-04-15T15:21:52.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.576+0000 7f56d2ffd640 1 -- 172.21.15.105:0/110893837 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f56c0003520 con 0x7f56d415e8d0 2024-04-15T15:21:52.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.576+0000 7f56caffd640 1 -- 172.21.15.105:0/110893837 >> v2:172.21.15.154:3300/0 conn(0x7f56d40d1650 msgr2=0x7f56d40b7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:52.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.576+0000 7f56caffd640 1 --2- 172.21.15.105:0/110893837 >> v2:172.21.15.154:3300/0 conn(0x7f56d40d1650 0x7f56d40b7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:52.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.576+0000 7f56caffd640 1 -- 172.21.15.105:0/110893837 >> v1:172.21.15.105:6789/0 conn(0x7f56d415e8d0 legacy=0x7f56d40d2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:52.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.576+0000 7f56caffd640 1 -- 172.21.15.105:0/110893837 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56d40d36e0 con 0x7f56d40ba480 2024-04-15T15:21:52.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.577+0000 7f56caffd640 1 --2- 172.21.15.105:0/110893837 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56d40ba480 0x7f56d40d1110 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f56bc0097d0 tx=0x7f56bc02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93a04051ad3eaf07 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:52.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.577+0000 7f56d2ffd640 1 -- 172.21.15.105:0/110893837 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56bc034070 con 0x7f56d40ba480 2024-04-15T15:21:52.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.577+0000 7f56d2ffd640 1 -- 172.21.15.105:0/110893837 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56bc030610 con 0x7f56d40ba480 2024-04-15T15:21:52.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.577+0000 7f56d2ffd640 1 -- 172.21.15.105:0/110893837 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56bc0309e0 con 0x7f56d40ba480 2024-04-15T15:21:52.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.577+0000 7f56d9b9d640 1 -- 172.21.15.105:0/110893837 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56d40ba480 msgr2=0x7f56d40d1110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:52.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.577+0000 7f56d9b9d640 1 --2- 172.21.15.105:0/110893837 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56d40ba480 0x7f56d40d1110 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f56bc0097d0 tx=0x7f56bc02b960 comp rx=0 tx=0).stop 2024-04-15T15:21:52.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.577+0000 7f56d9b9d640 1 -- 172.21.15.105:0/110893837 shutdown_connections 2024-04-15T15:21:52.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.577+0000 7f56d9b9d640 1 --2- 172.21.15.105:0/110893837 >> v2:172.21.15.154:3300/0 conn(0x7f56d40d1650 0x7f56d40b7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:52.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.577+0000 7f56d9b9d640 1 --2- 172.21.15.105:0/110893837 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56d40ba480 0x7f56d40d1110 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:52.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.577+0000 7f56d9b9d640 1 -- 172.21.15.105:0/110893837 >> 172.21.15.105:0/110893837 conn(0x7f56d406d460 msgr2=0x7f56d4067720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:52.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.577+0000 7f56d9b9d640 1 -- 172.21.15.105:0/110893837 shutdown_connections 2024-04-15T15:21:52.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.577+0000 7f56d9b9d640 1 -- 172.21.15.105:0/110893837 wait complete. 2024-04-15T15:21:52.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.579+0000 7f56d9b9d640 1 Processor -- start 2024-04-15T15:21:52.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.579+0000 7f56d9b9d640 1 -- start start 2024-04-15T15:21:52.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.579+0000 7f56d9b9d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56d40ba480 0x7f56d4149bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:52.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.579+0000 7f56d9b9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56d40d1650 0x7f56d415a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:52.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.579+0000 7f56d9b9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56d415e8d0 0x7f56d415ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:52.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.579+0000 7f56d9b9d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f56d40d44a0 con 0x7f56d415e8d0 2024-04-15T15:21:52.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.579+0000 7f56d9b9d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f56d40d3f80 con 0x7f56d40d1650 2024-04-15T15:21:52.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.579+0000 7f56d9b9d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f56d415ecb0 con 0x7f56d40ba480 2024-04-15T15:21:52.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.580+0000 7f56d3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56d415e8d0 0x7f56d415ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:52.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.580+0000 7f56d3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56d415e8d0 0x7f56d415ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38710/0 (socket says 172.21.15.105:38710) 2024-04-15T15:21:52.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.580+0000 7f56d3fff640 1 -- 172.21.15.105:0/3560102456 learned_addr learned my addr 172.21.15.105:0/3560102456 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:52.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.580+0000 7f56d37fe640 1 --2- 172.21.15.105:0/3560102456 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56d40ba480 0x7f56d4149bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:52.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.580+0000 7f56caffd640 1 --2- 172.21.15.105:0/3560102456 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56d40d1650 0x7f56d415a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:52.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.580+0000 7f56d3fff640 1 -- 172.21.15.105:0/3560102456 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56d40ba480 msgr2=0x7f56d4149bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:52.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.580+0000 7f56d3fff640 1 --2- 172.21.15.105:0/3560102456 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56d40ba480 0x7f56d4149bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:52.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.580+0000 7f56d3fff640 1 -- 172.21.15.105:0/3560102456 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56d40d1650 msgr2=0x7f56d415a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:52.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.580+0000 7f56d3fff640 1 --2- 172.21.15.105:0/3560102456 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56d40d1650 0x7f56d415a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:52.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.580+0000 7f56d3fff640 1 -- 172.21.15.105:0/3560102456 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56d415ff30 con 0x7f56d415e8d0 2024-04-15T15:21:52.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.580+0000 7f56caffd640 1 --2- 172.21.15.105:0/3560102456 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56d40d1650 0x7f56d415a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:21:52.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.580+0000 7f56d3fff640 1 --2- 172.21.15.105:0/3560102456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56d415e8d0 0x7f56d415ac70 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f56cc009dd0 tx=0x7f56cc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:52.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.580+0000 7f56d0ff9640 1 -- 172.21.15.105:0/3560102456 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56cc018020 con 0x7f56d415e8d0 2024-04-15T15:21:52.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.580+0000 7f56d0ff9640 1 -- 172.21.15.105:0/3560102456 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56cc014480 con 0x7f56d415e8d0 2024-04-15T15:21:52.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.580+0000 7f56d9b9d640 1 -- 172.21.15.105:0/3560102456 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56d4160210 con 0x7f56d415e8d0 2024-04-15T15:21:52.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.580+0000 7f56d0ff9640 1 -- 172.21.15.105:0/3560102456 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56cc014870 con 0x7f56d415e8d0 2024-04-15T15:21:52.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.580+0000 7f56d9b9d640 1 -- 172.21.15.105:0/3560102456 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56d41607d0 con 0x7f56d415e8d0 2024-04-15T15:21:52.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.581+0000 7f56d9b9d640 1 -- 172.21.15.105:0/3560102456 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56d4068ab0 con 0x7f56d415e8d0 2024-04-15T15:21:52.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.582+0000 7f56d0ff9640 1 -- 172.21.15.105:0/3560102456 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f56cc005d10 con 0x7f56d415e8d0 2024-04-15T15:21:52.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.582+0000 7f56d0ff9640 1 --2- 172.21.15.105:0/3560102456 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f56b406c9b0 0x7f56b406ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:52.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.582+0000 7f56d37fe640 1 --2- 172.21.15.105:0/3560102456 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f56b406c9b0 0x7f56b406ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:52.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.582+0000 7f56d0ff9640 1 -- 172.21.15.105:0/3560102456 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5029+0+0 (secure 0 0 0) 0x7f56cc08c490 con 0x7f56d415e8d0 2024-04-15T15:21:52.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.583+0000 7f56d37fe640 1 --2- 172.21.15.105:0/3560102456 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f56b406c9b0 0x7f56b406ee70 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f56c4005880 tx=0x7f56c4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:52.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.586+0000 7f56d0ff9640 1 -- 172.21.15.105:0/3560102456 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f56cc022070 con 0x7f56d415e8d0 2024-04-15T15:21:52.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.764+0000 7f56d9b9d640 1 -- 172.21.15.105:0/3560102456 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f56d40cfb00 con 0x7f56d415e8d0 2024-04-15T15:21:52.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.767+0000 7f56d0ff9640 1 -- 172.21.15.105:0/3560102456 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v16)=0 v16) ==== 123+0+0 (secure 0 0 0) 0x7f56cc010040 con 0x7f56d415e8d0 2024-04-15T15:21:52.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.769+0000 7f56d0ff9640 1 -- 172.21.15.105:0/3560102456 <== mon.0 v2:172.21.15.105:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f56cc05aa60 con 0x7f56d415e8d0 2024-04-15T15:21:52.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.770+0000 7f56d9b9d640 1 -- 172.21.15.105:0/3560102456 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f56b406c9b0 msgr2=0x7f56b406ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:52.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.770+0000 7f56d9b9d640 1 --2- 172.21.15.105:0/3560102456 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f56b406c9b0 0x7f56b406ee70 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f56c4005880 tx=0x7f56c4009040 comp rx=0 tx=0).stop 2024-04-15T15:21:52.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.770+0000 7f56d9b9d640 1 -- 172.21.15.105:0/3560102456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56d415e8d0 msgr2=0x7f56d415ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:52.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.770+0000 7f56d9b9d640 1 --2- 172.21.15.105:0/3560102456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56d415e8d0 0x7f56d415ac70 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f56cc009dd0 tx=0x7f56cc00c490 comp rx=0 tx=0).stop 2024-04-15T15:21:52.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.770+0000 7f56d9b9d640 1 -- 172.21.15.105:0/3560102456 shutdown_connections 2024-04-15T15:21:52.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.770+0000 7f56d9b9d640 1 --2- 172.21.15.105:0/3560102456 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f56b406c9b0 0x7f56b406ee70 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:52.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.770+0000 7f56d9b9d640 1 --2- 172.21.15.105:0/3560102456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56d415e8d0 0x7f56d415ac70 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:52.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.770+0000 7f56d9b9d640 1 --2- 172.21.15.105:0/3560102456 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56d40d1650 0x7f56d415a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:52.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.770+0000 7f56d9b9d640 1 --2- 172.21.15.105:0/3560102456 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56d40ba480 0x7f56d4149bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:52.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.770+0000 7f56d9b9d640 1 -- 172.21.15.105:0/3560102456 >> 172.21.15.105:0/3560102456 conn(0x7f56d406d460 msgr2=0x7f56d40b9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:52.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.770+0000 7f56d9b9d640 1 -- 172.21.15.105:0/3560102456 shutdown_connections 2024-04-15T15:21:52.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.771+0000 7f56d9b9d640 1 -- 172.21.15.105:0/3560102456 wait complete. 2024-04-15T15:21:52.784 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-15T15:21:52.784 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-15T15:21:52.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.885+0000 7f1c87469640 1 Processor -- start 2024-04-15T15:21:52.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.886+0000 7f1c87469640 1 -- start start 2024-04-15T15:21:52.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.886+0000 7f1c87469640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1c8013e680 0x7f1c800cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:52.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.886+0000 7f1c87469640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1c8006f6a0 0x7f1c8015f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:52.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.886+0000 7f1c87469640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c8015bc90 con 0x7f1c8013ec60 2024-04-15T15:21:52.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.886+0000 7f1c87469640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f1c8015bf70 con 0x7f1c8006f6a0 2024-04-15T15:21:52.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.886+0000 7f1c87469640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f1c8015c250 con 0x7f1c8013e680 2024-04-15T15:21:52.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.887+0000 7f1c85a06640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1c8006f6a0 0x7f1c8015f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:52.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.887+0000 7f1c85205640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1c8013e680 0x7f1c800cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:52.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.887+0000 7f1c85a06640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1c8006f6a0 0x7f1c8015f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.105:49956/0 (socket says 172.21.15.105:49956) 2024-04-15T15:21:52.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.887+0000 7f1c85a06640 1 -- 172.21.15.105:0/2093552649 learned_addr learned my addr 172.21.15.105:0/2093552649 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:52.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.887+0000 7f1c84a04640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f1c8013ec60 0x7f1c8006ef90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.105:44128/0 (socket says 172.21.15.105:44128) 2024-04-15T15:21:52.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.887+0000 7f1c85a06640 1 -- 172.21.15.105:0/2093552649 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1c8013e680 msgr2=0x7f1c800cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:52.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.887+0000 7f1c85a06640 1 --2- 172.21.15.105:0/2093552649 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1c8013e680 0x7f1c800cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:52.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.887+0000 7f1c85a06640 1 -- 172.21.15.105:0/2093552649 >> v1:172.21.15.154:6789/0 conn(0x7f1c8013ec60 legacy=0x7f1c8006ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:52.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.887+0000 7f1c85a06640 1 -- 172.21.15.105:0/2093552649 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c8015c530 con 0x7f1c8006f6a0 2024-04-15T15:21:52.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.887+0000 7f1c85205640 1 --2- 172.21.15.105:0/2093552649 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1c8013e680 0x7f1c800cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:21:52.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.888+0000 7f1c85a06640 1 --2- 172.21.15.105:0/2093552649 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c8006f6a0 0x7f1c8015f1e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1c78009e50 tx=0x7f1c7802bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2dfa3a78569081 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:52.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.888+0000 7f1c73fff640 1 -- 172.21.15.105:0/2093552649 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7802cb00 con 0x7f1c8006f6a0 2024-04-15T15:21:52.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.888+0000 7f1c73fff640 1 -- 172.21.15.105:0/2093552649 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f1c7802cca0 con 0x7f1c8006f6a0 2024-04-15T15:21:52.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.888+0000 7f1c73fff640 1 -- 172.21.15.105:0/2093552649 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c78034630 con 0x7f1c8006f6a0 2024-04-15T15:21:52.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.889+0000 7f1c87469640 1 -- 172.21.15.105:0/2093552649 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c8006f6a0 msgr2=0x7f1c8015f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:52.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.889+0000 7f1c87469640 1 --2- 172.21.15.105:0/2093552649 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c8006f6a0 0x7f1c8015f1e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1c78009e50 tx=0x7f1c7802bf60 comp rx=0 tx=0).stop 2024-04-15T15:21:52.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.889+0000 7f1c87469640 1 -- 172.21.15.105:0/2093552649 shutdown_connections 2024-04-15T15:21:52.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.889+0000 7f1c87469640 1 --2- 172.21.15.105:0/2093552649 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c8006f6a0 0x7f1c8015f1e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:52.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.889+0000 7f1c87469640 1 --2- 172.21.15.105:0/2093552649 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1c8013e680 0x7f1c800cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:52.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.889+0000 7f1c87469640 1 -- 172.21.15.105:0/2093552649 >> 172.21.15.105:0/2093552649 conn(0x7f1c80067d50 msgr2=0x7f1c8006b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:52.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.889+0000 7f1c87469640 1 -- 172.21.15.105:0/2093552649 shutdown_connections 2024-04-15T15:21:52.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.889+0000 7f1c87469640 1 -- 172.21.15.105:0/2093552649 wait complete. 2024-04-15T15:21:52.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.890+0000 7f1c87469640 1 Processor -- start 2024-04-15T15:21:52.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.890+0000 7f1c87469640 1 -- start start 2024-04-15T15:21:52.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.891+0000 7f1c87469640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c8006f6a0 0x7f1c8006ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:52.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.891+0000 7f1c87469640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1c8013e680 0x7f1c8013dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:52.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.891+0000 7f1c87469640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c8013ec60 0x7f1c80144200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:52.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.891+0000 7f1c87469640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1c800d0230 con 0x7f1c8013ec60 2024-04-15T15:21:52.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.891+0000 7f1c87469640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1c800d00b0 con 0x7f1c8006f6a0 2024-04-15T15:21:52.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.891+0000 7f1c87469640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f1c800cf590 con 0x7f1c8013e680 2024-04-15T15:21:52.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.891+0000 7f1c85a06640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c8013ec60 0x7f1c80144200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:52.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.891+0000 7f1c85a06640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c8013ec60 0x7f1c80144200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38712/0 (socket says 172.21.15.105:38712) 2024-04-15T15:21:52.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.891+0000 7f1c85a06640 1 -- 172.21.15.105:0/2843739008 learned_addr learned my addr 172.21.15.105:0/2843739008 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-15T15:21:52.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.891+0000 7f1c85205640 1 --2- 172.21.15.105:0/2843739008 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c8006f6a0 0x7f1c8006ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:52.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.891+0000 7f1c84a04640 1 --2- 172.21.15.105:0/2843739008 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1c8013e680 0x7f1c8013dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:52.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.891+0000 7f1c85a06640 1 -- 172.21.15.105:0/2843739008 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1c8013e680 msgr2=0x7f1c8013dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:52.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.891+0000 7f1c85a06640 1 --2- 172.21.15.105:0/2843739008 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1c8013e680 0x7f1c8013dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:52.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.891+0000 7f1c85a06640 1 -- 172.21.15.105:0/2843739008 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c8006f6a0 msgr2=0x7f1c8006ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:52.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.891+0000 7f1c85a06640 1 --2- 172.21.15.105:0/2843739008 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c8006f6a0 0x7f1c8006ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:52.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.891+0000 7f1c85a06640 1 -- 172.21.15.105:0/2843739008 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c80143160 con 0x7f1c8013ec60 2024-04-15T15:21:52.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.891+0000 7f1c85a06640 1 --2- 172.21.15.105:0/2843739008 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c8013ec60 0x7f1c80144200 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f1c78036040 tx=0x7f1c78004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:52.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.891+0000 7f1c71ffb640 1 -- 172.21.15.105:0/2843739008 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c78043020 con 0x7f1c8013ec60 2024-04-15T15:21:52.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.891+0000 7f1c71ffb640 1 -- 172.21.15.105:0/2843739008 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f1c7802ce50 con 0x7f1c8013ec60 2024-04-15T15:21:52.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.891+0000 7f1c87469640 1 -- 172.21.15.105:0/2843739008 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c80143440 con 0x7f1c8013ec60 2024-04-15T15:21:52.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.892+0000 7f1c71ffb640 1 -- 172.21.15.105:0/2843739008 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c78009510 con 0x7f1c8013ec60 2024-04-15T15:21:52.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.892+0000 7f1c87469640 1 -- 172.21.15.105:0/2843739008 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c80143ce0 con 0x7f1c8013ec60 2024-04-15T15:21:52.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.893+0000 7f1c87469640 1 -- 172.21.15.105:0/2843739008 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c80068a40 con 0x7f1c8013ec60 2024-04-15T15:21:52.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.893+0000 7f1c71ffb640 1 -- 172.21.15.105:0/2843739008 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f1c78046020 con 0x7f1c8013ec60 2024-04-15T15:21:52.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.894+0000 7f1c71ffb640 1 --2- 172.21.15.105:0/2843739008 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f1c6406c980 0x7f1c6406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:52.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.894+0000 7f1c71ffb640 1 -- 172.21.15.105:0/2843739008 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5029+0+0 (secure 0 0 0) 0x7f1c780ae120 con 0x7f1c8013ec60 2024-04-15T15:21:52.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.894+0000 7f1c85205640 1 --2- 172.21.15.105:0/2843739008 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f1c6406c980 0x7f1c6406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:52.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.894+0000 7f1c85205640 1 --2- 172.21.15.105:0/2843739008 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f1c6406c980 0x7f1c6406ee40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1c800af800 tx=0x7f1c74008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:52.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:52.898+0000 7f1c71ffb640 1 -- 172.21.15.105:0/2843739008 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1c7804d030 con 0x7f1c8013ec60 2024-04-15T15:21:53.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:53.112+0000 7f1c87469640 1 -- 172.21.15.105:0/2843739008 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1c800b3790 con 0x7f1c8013ec60 2024-04-15T15:21:53.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:53.113+0000 7f1c71ffb640 1 -- 172.21.15.105:0/2843739008 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+3871 (secure 0 0 0) 0x7f1c7807c700 con 0x7f1c8013ec60 2024-04-15T15:21:53.115 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-15T15:21:53.115 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":22,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4778,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/4848508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":4848508},{"type":"v1","addr":"172.21.15.154:6835","nonce":4848508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5005,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/1127070021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1127070021},{"type":"v1","addr":"172.21.15.154:6833","nonce":1127070021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5193,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6836/3825669416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3825669416},{"type":"v1","addr":"172.21.15.105:6836","nonce":3825669416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":5958,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2255273484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":2255273484},{"type":"v1","addr":"172.21.15.105:6837","nonce":2255273484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[]} 2024-04-15T15:21:53.115 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 22 2024-04-15T15:21:53.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:53.115+0000 7f1c87469640 1 -- 172.21.15.105:0/2843739008 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f1c6406c980 msgr2=0x7f1c6406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:53.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:53.115+0000 7f1c87469640 1 --2- 172.21.15.105:0/2843739008 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f1c6406c980 0x7f1c6406ee40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1c800af800 tx=0x7f1c74008040 comp rx=0 tx=0).stop 2024-04-15T15:21:53.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:53.115+0000 7f1c87469640 1 -- 172.21.15.105:0/2843739008 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c8013ec60 msgr2=0x7f1c80144200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:53.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:53.115+0000 7f1c87469640 1 --2- 172.21.15.105:0/2843739008 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c8013ec60 0x7f1c80144200 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f1c78036040 tx=0x7f1c78004a40 comp rx=0 tx=0).stop 2024-04-15T15:21:53.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:53.115+0000 7f1c87469640 1 -- 172.21.15.105:0/2843739008 shutdown_connections 2024-04-15T15:21:53.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:53.115+0000 7f1c87469640 1 --2- 172.21.15.105:0/2843739008 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1c8013e680 0x7f1c8013dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:53.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:53.115+0000 7f1c87469640 1 --2- 172.21.15.105:0/2843739008 >> [v2:172.21.15.105:6832/3506549322,v1:172.21.15.105:6833/3506549322] conn(0x7f1c6406c980 0x7f1c6406ee40 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:53.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:53.115+0000 7f1c87469640 1 --2- 172.21.15.105:0/2843739008 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c8013ec60 0x7f1c80144200 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:53.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:53.115+0000 7f1c87469640 1 --2- 172.21.15.105:0/2843739008 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c8006f6a0 0x7f1c8006ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:53.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:53.115+0000 7f1c87469640 1 -- 172.21.15.105:0/2843739008 >> 172.21.15.105:0/2843739008 conn(0x7f1c80067d50 msgr2=0x7f1c800b21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:53.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:53.115+0000 7f1c87469640 1 -- 172.21.15.105:0/2843739008 shutdown_connections 2024-04-15T15:21:53.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-15T15:21:53.115+0000 7f1c87469640 1 -- 172.21.15.105:0/2843739008 wait complete. 2024-04-15T15:21:53.128 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-15T15:21:53.128 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-15T15:21:53.128 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-15T15:21:53.128 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:21:53.137 INFO:tasks.ceph.mds.a:Stopped 2024-04-15T15:21:53.137 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-15T15:21:53.138 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:21:53.187 INFO:tasks.ceph.mds.c:Stopped 2024-04-15T15:21:53.187 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-15T15:21:53.187 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:21:53.200 INFO:tasks.ceph.mds.b:Stopped 2024-04-15T15:21:53.200 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-15T15:21:53.200 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:21:53.253 INFO:tasks.ceph.mds.d:Stopped 2024-04-15T15:21:53.253 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-15T15:21:53.253 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-15T15:21:53.253 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:21:53.289 INFO:tasks.ceph.osd.0:Stopped 2024-04-15T15:21:53.289 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-15T15:21:53.289 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:21:53.365 INFO:tasks.ceph.osd.1:Stopped 2024-04-15T15:21:53.365 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-15T15:21:53.365 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:21:53.434 INFO:tasks.ceph.osd.2:Stopped 2024-04-15T15:21:53.434 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-15T15:21:53.434 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:21:53.499 INFO:tasks.ceph.osd.3:Stopped 2024-04-15T15:21:53.499 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-15T15:21:53.499 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:21:53.532 INFO:tasks.ceph.osd.4:Stopped 2024-04-15T15:21:53.532 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-15T15:21:53.532 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:21:53.603 INFO:tasks.ceph.osd.5:Stopped 2024-04-15T15:21:53.603 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-15T15:21:53.603 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:21:53.668 INFO:tasks.ceph.osd.6:Stopped 2024-04-15T15:21:53.668 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-15T15:21:53.668 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:21:53.734 INFO:tasks.ceph.osd.7:Stopped 2024-04-15T15:21:53.734 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-15T15:21:53.735 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-15T15:21:53.735 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:21:53.762 INFO:tasks.ceph.mgr.y:Stopped 2024-04-15T15:21:53.762 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-15T15:21:53.762 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:21:53.802 INFO:tasks.ceph.mgr.x:Stopped 2024-04-15T15:21:53.802 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-15T15:21:53.802 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-15T15:21:53.802 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:21:53.813 INFO:tasks.ceph.mon.a:Stopped 2024-04-15T15:21:53.814 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-15T15:21:53.814 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:21:53.854 INFO:tasks.ceph.mon.b:Stopped 2024-04-15T15:21:53.855 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-15T15:21:53.855 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:21:53.907 INFO:tasks.ceph.mon.c:Stopped 2024-04-15T15:21:53.907 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-15T15:21:53.908 DEBUG:teuthology.orchestra.run.smithi105:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow request' | head -n 1 2024-04-15T15:21:53.943 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi105.front.sepia.ceph.com 2024-04-15T15:21:53.944 DEBUG:teuthology.orchestra.run.smithi105:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-15T15:21:54.203 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi105.front.sepia.ceph.com 2024-04-15T15:21:54.204 DEBUG:teuthology.orchestra.run.smithi105:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-15T15:21:54.284 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi105.front.sepia.ceph.com 2024-04-15T15:21:54.285 DEBUG:teuthology.orchestra.run.smithi105:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-15T15:21:54.342 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi105.front.sepia.ceph.com 2024-04-15T15:21:54.342 DEBUG:teuthology.orchestra.run.smithi105:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-15T15:21:54.400 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi154.front.sepia.ceph.com 2024-04-15T15:21:54.400 DEBUG:teuthology.orchestra.run.smithi154:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-15T15:21:54.563 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi154.front.sepia.ceph.com 2024-04-15T15:21:54.563 DEBUG:teuthology.orchestra.run.smithi154:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-15T15:21:54.627 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi154.front.sepia.ceph.com 2024-04-15T15:21:54.627 DEBUG:teuthology.orchestra.run.smithi154:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-15T15:21:54.684 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi154.front.sepia.ceph.com 2024-04-15T15:21:54.685 DEBUG:teuthology.orchestra.run.smithi154:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-15T15:21:54.742 INFO:tasks.ceph:Archiving mon data... 2024-04-15T15:21:54.744 DEBUG:teuthology.misc:Transferring archived files from smithi105:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/leonidus-2024-04-15_14:54:08-fs-wip-lusov-quiesce-xlock-distro-default-smithi/7657132/data/mon.a.tgz 2024-04-15T15:21:54.744 DEBUG:teuthology.orchestra.run.smithi105:> mktemp 2024-04-15T15:21:54.760 INFO:teuthology.orchestra.run.smithi105.stdout:/tmp/tmp.0R2vBM32qW 2024-04-15T15:21:54.760 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.0R2vBM32qW 2024-04-15T15:21:54.894 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 0666 /tmp/tmp.0R2vBM32qW 2024-04-15T15:21:54.982 DEBUG:teuthology.orchestra.remote:smithi105:/tmp/tmp.0R2vBM32qW is 337KB 2024-04-15T15:21:55.079 DEBUG:teuthology.orchestra.run.smithi105:> rm -fr /tmp/tmp.0R2vBM32qW 2024-04-15T15:21:55.094 DEBUG:teuthology.misc:Transferring archived files from smithi154:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/leonidus-2024-04-15_14:54:08-fs-wip-lusov-quiesce-xlock-distro-default-smithi/7657132/data/mon.b.tgz 2024-04-15T15:21:55.094 DEBUG:teuthology.orchestra.run.smithi154:> mktemp 2024-04-15T15:21:55.111 INFO:teuthology.orchestra.run.smithi154.stdout:/tmp/tmp.y5TYATZ7Gd 2024-04-15T15:21:55.111 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.y5TYATZ7Gd 2024-04-15T15:21:55.272 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 0666 /tmp/tmp.y5TYATZ7Gd 2024-04-15T15:21:55.359 DEBUG:teuthology.orchestra.remote:smithi154:/tmp/tmp.y5TYATZ7Gd is 457KB 2024-04-15T15:21:55.457 DEBUG:teuthology.orchestra.run.smithi154:> rm -fr /tmp/tmp.y5TYATZ7Gd 2024-04-15T15:21:55.472 DEBUG:teuthology.misc:Transferring archived files from smithi154:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/leonidus-2024-04-15_14:54:08-fs-wip-lusov-quiesce-xlock-distro-default-smithi/7657132/data/mon.c.tgz 2024-04-15T15:21:55.472 DEBUG:teuthology.orchestra.run.smithi154:> mktemp 2024-04-15T15:21:55.527 INFO:teuthology.orchestra.run.smithi154.stdout:/tmp/tmp.miV5JbZE5f 2024-04-15T15:21:55.528 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.miV5JbZE5f 2024-04-15T15:21:55.676 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 0666 /tmp/tmp.miV5JbZE5f 2024-04-15T15:21:55.763 DEBUG:teuthology.orchestra.remote:smithi154:/tmp/tmp.miV5JbZE5f is 457KB 2024-04-15T15:21:55.859 DEBUG:teuthology.orchestra.run.smithi154:> rm -fr /tmp/tmp.miV5JbZE5f 2024-04-15T15:21:55.874 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-15T15:21:55.875 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-15T15:21:55.877 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-15T15:21:55.915 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-15T15:21:55.948 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-15T15:21:55.963 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-15T15:21:55.999 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-15T15:21:55.999 INFO:tasks.ceph:Archiving crash dumps... 2024-04-15T15:21:56.001 DEBUG:teuthology.misc:Transferring archived files from smithi105:/var/lib/ceph/crash to /home/teuthworker/archive/leonidus-2024-04-15_14:54:08-fs-wip-lusov-quiesce-xlock-distro-default-smithi/7657132/remote/smithi105/crash 2024-04-15T15:21:56.001 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-15T15:21:56.033 DEBUG:teuthology.misc:Transferring archived files from smithi154:/var/lib/ceph/crash to /home/teuthworker/archive/leonidus-2024-04-15_14:54:08-fs-wip-lusov-quiesce-xlock-distro-default-smithi/7657132/remote/smithi154/crash 2024-04-15T15:21:56.034 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-15T15:21:56.065 INFO:tasks.ceph:Shutting down logrotate 2024-04-15T15:21:56.066 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-15T15:21:56.078 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-15T15:21:56.186 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-15T15:21:56.215 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-15T15:21:56.243 INFO:tasks.ceph:Compressing logs... 2024-04-15T15:21:56.244 DEBUG:teuthology.orchestra.run.smithi105:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-15T15:21:56.258 DEBUG:teuthology.orchestra.run.smithi154:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-15T15:21:56.284 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49297.log 2024-04-15T15:21:56.284 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57462.log 2024-04-15T15:21:56.284 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50425.log 2024-04-15T15:21:56.345 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-15T15:21:56.346 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.45956.log 2024-04-15T15:21:56.346 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37429.log 2024-04-15T15:21:56.346 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.3.45956.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-15T15:21:56.346 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.37429.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-15T15:21:56.346 INFO:teuthology.orchestra.run.smithi154.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37429.log.gz 2024-04-15T15:21:56.347 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-15T15:21:56.347 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37269.log 2024-04-15T15:21:56.347 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-mds.b.log: /var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37509.log 2024-04-15T15:21:56.347 INFO:teuthology.orchestra.run.smithi154.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.3.45956.log.gz 2024-04-15T15:21:56.347 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.37269.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-15T15:21:56.347 INFO:teuthology.orchestra.run.smithi154.stderr:gzip 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37269.log.gz 2024-04-15T15:21:56.347 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.37509.log: -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-15T15:21:56.348 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-osd.7.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37509.log.gz 2024-04-15T15:21:56.348 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37549.log 2024-04-15T15:21:56.348 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.44378.log 2024-04-15T15:21:56.348 INFO:teuthology.orchestra.run.smithi154.stderr: 93.8%/var/log/ceph/ceph-client.admin.37549.log: -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-15T15:21:56.348 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.45615.log 2024-04-15T15:21:56.348 INFO:teuthology.orchestra.run.smithi154.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37549.log.gz 2024-04-15T15:21:56.348 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.3.44378.log: -- /var/log/ceph/ceph-client.1.43921.log 2024-04-15T15:21:56.349 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.1.45615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37349.log 2024-04-15T15:21:56.349 INFO:teuthology.orchestra.run.smithi154.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.3.44378.log.gz/var/log/ceph/ceph-client.1.43921.log: 2024-04-15T15:21:56.349 INFO:teuthology.orchestra.run.smithi154.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.1.43921.log.gz 2024-04-15T15:21:56.349 INFO:teuthology.orchestra.run.smithi154.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.1.45615.log.gz 2024-04-15T15:21:56.349 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-15T15:21:56.349 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.37349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37389.log 2024-04-15T15:21:56.349 INFO:teuthology.orchestra.run.smithi154.stderr: 89.3%/var/log/ceph/ceph-mgr.x.log: -- replaced with /var/log/ceph/ceph-client.admin.37349.log.gz 2024-04-15T15:21:56.349 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37309.log 2024-04-15T15:21:56.349 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-15T15:21:56.350 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.37389.log: /var/log/ceph/ceph-client.admin.37309.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.37389.log.gz 2024-04-15T15:21:56.350 INFO:teuthology.orchestra.run.smithi154.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37309.log.gz 2024-04-15T15:21:56.350 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-15T15:21:56.350 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.45121.log 2024-04-15T15:21:56.350 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-mon.b.log: 91.5% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-15T15:21:56.350 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-15T15:21:56.350 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.1.45121.log: 92.9% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-15T15:21:56.350 INFO:teuthology.orchestra.run.smithi154.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.1.45121.log.gz 2024-04-15T15:21:56.351 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37469.log 2024-04-15T15:21:56.351 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-15T15:21:56.351 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.37469.log: 92.2% 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37469.log.gz 2024-04-15T15:21:56.351 INFO:teuthology.orchestra.run.smithi154.stderr: -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-15T15:21:56.351 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-mon.c.log: 91.3% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-15T15:21:56.399 INFO:teuthology.orchestra.run.smithi154.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-15T15:21:56.430 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.57462.log: gzip -5 --verbose -- 89.4% /var/log/ceph/ceph-client.admin.55669.log 2024-04-15T15:21:56.430 INFO:teuthology.orchestra.run.smithi105.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49297.log.gz 2024-04-15T15:21:56.430 INFO:teuthology.orchestra.run.smithi105.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57462.log.gz 2024-04-15T15:21:56.430 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47029.log 2024-04-15T15:21:56.431 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.55669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49465.log 2024-04-15T15:21:56.431 INFO:teuthology.orchestra.run.smithi105.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50425.log.gz 2024-04-15T15:21:56.431 INFO:teuthology.orchestra.run.smithi105.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.55669.log.gz 2024-04-15T15:21:56.431 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47029.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-15T15:21:56.431 INFO:teuthology.orchestra.run.smithi105.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47029.log.gz 2024-04-15T15:21:56.431 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51378.log 2024-04-15T15:21:56.431 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56083.log 89.1% 2024-04-15T15:21:56.431 INFO:teuthology.orchestra.run.smithi105.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49465.log.gz 2024-04-15T15:21:56.432 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47601.log 2024-04-15T15:21:56.432 INFO:teuthology.orchestra.run.smithi105.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51378.log.gz 2024-04-15T15:21:56.432 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.56083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48116.log 2024-04-15T15:21:56.432 INFO:teuthology.orchestra.run.smithi105.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56083.log.gz 2024-04-15T15:21:56.432 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55485.log 2024-04-15T15:21:56.432 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48116.log: gzip -5 --verbose 89.2% -- -- replaced with /var/log/ceph/ceph-client.admin.47601.log.gz /var/log/ceph/ceph-client.0.53766.log 2024-04-15T15:21:56.432 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-04-15T15:21:56.432 INFO:teuthology.orchestra.run.smithi105.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48116.log.gz 2024-04-15T15:21:56.432 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.55485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56690.log 2024-04-15T15:21:56.433 INFO:teuthology.orchestra.run.smithi105.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55485.log.gz 2024-04-15T15:21:56.433 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50983.log 2024-04-15T15:21:56.433 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.0.53766.log: gzip -5 --verbose/var/log/ceph/ceph-client.admin.56690.log: -- /var/log/ceph/ceph-client.admin.50131.log 2024-04-15T15:21:56.433 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43607.log 2024-04-15T15:21:56.433 INFO:teuthology.orchestra.run.smithi105.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56690.log.gz 2024-04-15T15:21:56.433 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50131.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50983.log.gz 2024-04-15T15:21:56.433 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53931.log 2024-04-15T15:21:56.433 INFO:teuthology.orchestra.run.smithi105.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50131.log.gz 2024-04-15T15:21:56.434 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.43607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48786.log 2024-04-15T15:21:56.434 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.53931.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43607.log.gz 2024-04-15T15:21:56.434 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55999.log 2024-04-15T15:21:56.434 INFO:teuthology.orchestra.run.smithi105.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53931.log.gz 2024-04-15T15:21:56.434 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54962.log 2024-04-15T15:21:56.434 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.55999.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48786.log.gz 2024-04-15T15:21:56.434 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45351.log 2024-04-15T15:21:56.434 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.54962.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55999.log.gz 2024-04-15T15:21:56.435 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55155.log 2024-04-15T15:21:56.435 INFO:teuthology.orchestra.run.smithi105.stderr: 88.1%/var/log/ceph/ceph-client.admin.45351.log: -- replaced with /var/log/ceph/ceph-client.admin.54962.log.gz 2024-04-15T15:21:56.435 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57420.log 2024-04-15T15:21:56.435 INFO:teuthology.orchestra.run.smithi105.stderr: 91.8% -- replaced with /var/log/ceph/ceph-client.0.53766.log.gz 2024-04-15T15:21:56.435 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.55155.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45351.log.gz 2024-04-15T15:21:56.435 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57756.log 2024-04-15T15:21:56.435 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.57420.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55155.log.gz 2024-04-15T15:21:56.435 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49381.log 2024-04-15T15:21:56.435 INFO:teuthology.orchestra.run.smithi105.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57420.log.gz 2024-04-15T15:21:56.436 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.57756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55323.log 2024-04-15T15:21:56.436 INFO:teuthology.orchestra.run.smithi105.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57756.log.gz 2024-04-15T15:21:56.436 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51714.log 2024-04-15T15:21:56.436 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.55323.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49381.log.gz 2024-04-15T15:21:56.436 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51672.log 2024-04-15T15:21:56.436 INFO:teuthology.orchestra.run.smithi105.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55323.log.gz 2024-04-15T15:21:56.436 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57714.log 2024-04-15T15:21:56.436 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51672.log: gzip -5 --verbose 88.9% -- -- replaced with /var/log/ceph/ceph-client.admin.51714.log.gz /var/log/ceph/ceph-client.admin.52957.log 2024-04-15T15:21:56.436 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-04-15T15:21:56.437 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.57714.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51672.log.gz 2024-04-15T15:21:56.437 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53413.log 2024-04-15T15:21:56.437 INFO:teuthology.orchestra.run.smithi105.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57714.log.gz 2024-04-15T15:21:56.437 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56988.log 2024-04-15T15:21:56.437 INFO:teuthology.orchestra.run.smithi105.stderr: 88.4%/var/log/ceph/ceph-client.admin.53413.log: -- replaced with /var/log/ceph/ceph-client.admin.52957.log.gz 2024-04-15T15:21:56.437 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55753.log 2024-04-15T15:21:56.437 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.56988.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53413.log.gz 2024-04-15T15:21:56.437 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44049.log 2024-04-15T15:21:56.438 INFO:teuthology.orchestra.run.smithi105.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56988.log.gz 2024-04-15T15:21:56.438 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.55753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54804.log 2024-04-15T15:21:56.438 INFO:teuthology.orchestra.run.smithi105.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55753.log.gz 2024-04-15T15:21:56.438 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56167.log 2024-04-15T15:21:56.438 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.54804.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44049.log.gz 2024-04-15T15:21:56.438 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47433.log 2024-04-15T15:21:56.438 INFO:teuthology.orchestra.run.smithi105.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54804.log.gz 2024-04-15T15:21:56.438 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.56167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49795.log 2024-04-15T15:21:56.439 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47433.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56167.log.gz 2024-04-15T15:21:56.439 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52789.log 2024-04-15T15:21:56.439 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49795.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47433.log.gz 2024-04-15T15:21:56.439 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51420.log 2024-04-15T15:21:56.439 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52789.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49795.log.gz 2024-04-15T15:21:56.439 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55443.log 2024-04-15T15:21:56.439 INFO:teuthology.orchestra.run.smithi105.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.52789.log.gz 2024-04-15T15:21:56.439 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43565.log 2024-04-15T15:21:56.440 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.55443.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51420.log.gz 2024-04-15T15:21:56.440 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43755.log 2024-04-15T15:21:56.440 INFO:teuthology.orchestra.run.smithi105.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55443.log.gz 2024-04-15T15:21:56.440 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.43565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57114.log 2024-04-15T15:21:56.440 INFO:teuthology.orchestra.run.smithi105.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43565.log.gz 2024-04-15T15:21:56.440 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.43755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55795.log 2024-04-15T15:21:56.440 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.57114.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.43755.log.gz 2024-04-15T15:21:56.440 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48372.log 2024-04-15T15:21:56.441 INFO:teuthology.orchestra.run.smithi105.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.57114.log.gz 2024-04-15T15:21:56.441 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.55795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56209.log 2024-04-15T15:21:56.441 INFO:teuthology.orchestra.run.smithi105.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55795.log.gz 2024-04-15T15:21:56.441 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50341.log 2024-04-15T15:21:56.441 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.56209.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48372.log.gz 2024-04-15T15:21:56.441 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44351.log 2024-04-15T15:21:56.441 INFO:teuthology.orchestra.run.smithi105.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56209.log.gz 2024-04-15T15:21:56.441 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56816.log 2024-04-15T15:21:56.442 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44351.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50341.log.gz 2024-04-15T15:21:56.442 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51153.log 2024-04-15T15:21:56.442 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.56816.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44351.log.gz 2024-04-15T15:21:56.442 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48618.log 2024-04-15T15:21:56.442 INFO:teuthology.orchestra.run.smithi105.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.56816.log.gz 2024-04-15T15:21:56.442 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56041.log 2024-04-15T15:21:56.442 INFO:teuthology.orchestra.run.smithi105.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51153.log.gz 2024-04-15T15:21:56.443 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49071.log 2024-04-15T15:21:56.443 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.56041.log: gzip 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48618.log.gz -5 2024-04-15T15:21:56.443 INFO:teuthology.orchestra.run.smithi105.stderr: --verbose -- /var/log/ceph/ceph-client.admin.53329.log 2024-04-15T15:21:56.443 INFO:teuthology.orchestra.run.smithi105.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56041.log.gz 2024-04-15T15:21:56.443 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51197.log 2024-04-15T15:21:56.443 INFO:teuthology.orchestra.run.smithi105.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49071.log.gz 2024-04-15T15:21:56.443 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.53329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50467.log 2024-04-15T15:21:56.443 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57504.log 2024-04-15T15:21:56.444 INFO:teuthology.orchestra.run.smithi105.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53329.log.gz 2024-04-15T15:21:56.444 INFO:teuthology.orchestra.run.smithi105.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51197.log.gz 2024-04-15T15:21:56.444 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43797.log 2024-04-15T15:21:56.444 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48227.log 2024-04-15T15:21:56.444 INFO:teuthology.orchestra.run.smithi105.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50467.log.gz 2024-04-15T15:21:56.444 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.57504.log: /var/log/ceph/ceph-client.admin.43797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51290.log 2024-04-15T15:21:56.444 INFO:teuthology.orchestra.run.smithi105.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57504.log.gz 2024-04-15T15:21:56.444 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48227.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43797.log.gz 2024-04-15T15:21:56.445 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48177.log 2024-04-15T15:21:56.445 INFO:teuthology.orchestra.run.smithi105.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48227.log.gz 2024-04-15T15:21:56.445 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49963.log 2024-04-15T15:21:56.445 INFO:teuthology.orchestra.run.smithi105.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51290.log.gz 2024-04-15T15:21:56.445 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55711.log 2024-04-15T15:21:56.445 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48177.log: /var/log/ceph/ceph-client.admin.49963.log: 87.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.48177.log.gz -5 2024-04-15T15:21:56.445 INFO:teuthology.orchestra.run.smithi105.stderr: --verbose -- /var/log/ceph/ceph-client.admin.57167.log 2024-04-15T15:21:56.445 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.55711.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49963.log.gz 2024-04-15T15:21:56.446 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43923.log 2024-04-15T15:21:56.446 INFO:teuthology.orchestra.run.smithi105.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.55711.log.gz 2024-04-15T15:21:56.446 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.57167.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.31292.log 2024-04-15T15:21:56.446 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.43923.log: gzip -5 88.8% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.57167.log.gz 2024-04-15T15:21:56.446 INFO:teuthology.orchestra.run.smithi105.stderr: /var/log/ceph/ceph-client.admin.51504.log 2024-04-15T15:21:56.446 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph.tmp-client.admin.31292.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43923.log.gz 2024-04-15T15:21:56.446 INFO:teuthology.orchestra.run.smithi105.stderr: 0.0%gzip -- replaced with /var/log/ceph/ceph.tmp-client.admin.31292.log.gz 2024-04-15T15:21:56.446 INFO:teuthology.orchestra.run.smithi105.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.45225.log 2024-04-15T15:21:56.447 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52647.log 2024-04-15T15:21:56.447 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45729.log 2024-04-15T15:21:56.447 INFO:teuthology.orchestra.run.smithi105.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51504.log.gz 2024-04-15T15:21:56.447 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52647.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45225.log.gz 2024-04-15T15:21:56.447 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47937.log 2024-04-15T15:21:56.447 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45729.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52647.log.gz 2024-04-15T15:21:56.447 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51462.log 2024-04-15T15:21:56.447 INFO:teuthology.orchestra.run.smithi105.stderr: 88.8%/var/log/ceph/ceph-client.admin.47937.log: -- replaced with /var/log/ceph/ceph-client.admin.45729.log.gz 2024-04-15T15:21:56.448 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45603.log 2024-04-15T15:21:56.448 INFO:teuthology.orchestra.run.smithi105.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47937.log.gz 2024-04-15T15:21:56.448 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57336.log 2024-04-15T15:21:56.448 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51462.log: /var/log/ceph/ceph-client.admin.45603.log: gzip -5 --verbose -- 88.8% /var/log/ceph/ceph-client.admin.48498.log -- replaced with /var/log/ceph/ceph-client.admin.51462.log.gz 2024-04-15T15:21:56.448 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-04-15T15:21:56.448 INFO:teuthology.orchestra.run.smithi105.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45603.log.gz 2024-04-15T15:21:56.448 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.57336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44693.log 2024-04-15T15:21:56.448 INFO:teuthology.orchestra.run.smithi105.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57336.log.gz 2024-04-15T15:21:56.449 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50005.log 2024-04-15T15:21:56.449 INFO:teuthology.orchestra.run.smithi105.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48498.log.gz 2024-04-15T15:21:56.449 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47685.log 2024-04-15T15:21:56.449 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50005.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44693.log.gz 2024-04-15T15:21:56.449 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43177.log 2024-04-15T15:21:56.449 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47685.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50005.log.gz 2024-04-15T15:21:56.449 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57922.log 2024-04-15T15:21:56.449 INFO:teuthology.orchestra.run.smithi105.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47685.log.gz 2024-04-15T15:21:56.450 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.43177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57630.log 2024-04-15T15:21:56.450 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.57922.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43177.log.gz 2024-04-15T15:21:56.450 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44175.log 2024-04-15T15:21:56.450 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.57630.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57922.log.gz 2024-04-15T15:21:56.450 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.54191.log 2024-04-15T15:21:56.450 INFO:teuthology.orchestra.run.smithi105.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.57630.log.gz 2024-04-15T15:21:56.450 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44353.log 2024-04-15T15:21:56.450 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.2.54191.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44175.log.gz 2024-04-15T15:21:56.450 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50881.log 2024-04-15T15:21:56.451 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53506.log 2024-04-15T15:21:56.451 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50881.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44353.log.gz 2024-04-15T15:21:56.451 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46367.log 2024-04-15T15:21:56.451 INFO:teuthology.orchestra.run.smithi105.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50881.log.gz 2024-04-15T15:21:56.451 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45435.log 2024-04-15T15:21:56.451 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.53506.log: /var/log/ceph/ceph-client.0.46367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37562.log 2024-04-15T15:21:56.451 INFO:teuthology.orchestra.run.smithi105.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53506.log.gz 2024-04-15T15:21:56.451 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45435.log: 91.3% -- replaced with /var/log/ceph/ceph-client.2.54191.log.gz 2024-04-15T15:21:56.452 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51906.log 2024-04-15T15:21:56.452 INFO:teuthology.orchestra.run.smithi105.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45435.log.gz 2024-04-15T15:21:56.452 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50257.log 2024-04-15T15:21:56.452 INFO:teuthology.orchestra.run.smithi105.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.0.46367.log.gz 2024-04-15T15:21:56.452 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.37562.log: /var/log/ceph/ceph-client.admin.51906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57546.log 2024-04-15T15:21:56.452 INFO:teuthology.orchestra.run.smithi105.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37562.log.gz 2024-04-15T15:21:56.452 INFO:teuthology.orchestra.run.smithi105.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.51906.log.gz 2024-04-15T15:21:56.452 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57294.log 2024-04-15T15:21:56.453 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50257.log: /var/log/ceph/ceph-client.admin.57546.log: gzip 88.5% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.50257.log.gz 2024-04-15T15:21:56.453 INFO:teuthology.orchestra.run.smithi105.stderr: -- /var/log/ceph/ceph-client.admin.50797.log 2024-04-15T15:21:56.453 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.57294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47307.log 2024-04-15T15:21:56.453 INFO:teuthology.orchestra.run.smithi105.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.57546.log.gz 2024-04-15T15:21:56.453 INFO:teuthology.orchestra.run.smithi105.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57294.log.gz 2024-04-15T15:21:56.453 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51630.log 2024-04-15T15:21:56.453 INFO:teuthology.orchestra.run.smithi105.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50797.log.gz 2024-04-15T15:21:56.453 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-15T15:21:56.454 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47307.log: /var/log/ceph/ceph-client.admin.51630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57252.log 2024-04-15T15:21:56.454 INFO:teuthology.orchestra.run.smithi105.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47307.log.gz 2024-04-15T15:21:56.454 INFO:teuthology.orchestra.run.smithi105.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51630.log.gz 2024-04-15T15:21:56.454 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48871.log 2024-04-15T15:21:56.454 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.57252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45855.log 2024-04-15T15:21:56.454 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48871.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57252.log.gz 2024-04-15T15:21:56.454 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55569.log 2024-04-15T15:21:56.454 INFO:teuthology.orchestra.run.smithi105.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48871.log.gz 2024-04-15T15:21:56.455 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45028.log 2024-04-15T15:21:56.455 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45855.log: 89.1% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-15T15:21:56.455 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.55569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45813.log 2024-04-15T15:21:56.455 INFO:teuthology.orchestra.run.smithi105.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.55569.log.gz 2024-04-15T15:21:56.455 INFO:teuthology.orchestra.run.smithi105.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.45855.log.gz 2024-04-15T15:21:56.455 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44352.log 2024-04-15T15:21:56.455 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45028.log: /var/log/ceph/ceph-client.admin.45813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47643.log 2024-04-15T15:21:56.455 INFO:teuthology.orchestra.run.smithi105.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45028.log.gz 2024-04-15T15:21:56.456 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51546.log 2024-04-15T15:21:56.456 INFO:teuthology.orchestra.run.smithi105.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45813.log.gz 2024-04-15T15:21:56.456 INFO:teuthology.orchestra.run.smithi105.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44352.log.gz 2024-04-15T15:21:56.456 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57798.log 2024-04-15T15:21:56.456 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51546.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47643.log.gz 2024-04-15T15:21:56.456 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48744.log 2024-04-15T15:21:56.456 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.57798.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51546.log.gz 2024-04-15T15:21:56.456 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47979.log 2024-04-15T15:21:56.457 INFO:teuthology.orchestra.run.smithi105.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57798.log.gz 2024-04-15T15:21:56.457 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44689.log 2024-04-15T15:21:56.457 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47979.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.48744.log.gz 2024-04-15T15:21:56.457 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44356.log 2024-04-15T15:21:56.457 INFO:teuthology.orchestra.run.smithi105.stderr: 89.3%/var/log/ceph/ceph-client.admin.44689.log: -- replaced with /var/log/ceph/ceph-client.admin.47979.log.gz 2024-04-15T15:21:56.457 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44357.log 2024-04-15T15:21:56.457 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55957.log 2024-04-15T15:21:56.457 INFO:teuthology.orchestra.run.smithi105.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44689.log.gz 2024-04-15T15:21:56.457 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55627.log 2024-04-15T15:21:56.458 INFO:teuthology.orchestra.run.smithi105.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44356.log.gz 2024-04-15T15:21:56.458 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.55957.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44357.log.gz 2024-04-15T15:21:56.458 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44091.log 2024-04-15T15:21:56.458 INFO:teuthology.orchestra.run.smithi105.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55957.log.gz 2024-04-15T15:21:56.458 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.55627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44908.log 2024-04-15T15:21:56.458 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44091.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.55627.log.gz 2024-04-15T15:21:56.458 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50215.log 2024-04-15T15:21:56.458 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44908.log: gzip -5 89.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.44091.log.gz 2024-04-15T15:21:56.459 INFO:teuthology.orchestra.run.smithi105.stderr: /var/log/ceph/ceph-client.admin.47139.log 2024-04-15T15:21:56.459 INFO:teuthology.orchestra.run.smithi105.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44908.log.gz 2024-04-15T15:21:56.459 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47769.log 2024-04-15T15:21:56.459 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47139.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50215.log.gz 2024-04-15T15:21:56.459 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43354.log 2024-04-15T15:21:56.459 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47769.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47139.log.gz 2024-04-15T15:21:56.459 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53161.log 2024-04-15T15:21:56.459 INFO:teuthology.orchestra.run.smithi105.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47769.log.gz 2024-04-15T15:21:56.460 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.43354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55873.log 2024-04-15T15:21:56.460 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.53161.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43354.log.gz 2024-04-15T15:21:56.460 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49585.log 2024-04-15T15:21:56.460 INFO:teuthology.orchestra.run.smithi105.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53161.log.gz 2024-04-15T15:21:56.460 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.55873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49155.log 2024-04-15T15:21:56.460 INFO:teuthology.orchestra.run.smithi105.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55873.log.gz 2024-04-15T15:21:56.460 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45561.log 2024-04-15T15:21:56.460 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56606.log 2024-04-15T15:21:56.460 INFO:teuthology.orchestra.run.smithi105.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49585.log.gz 2024-04-15T15:21:56.461 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45561.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49155.log.gz 2024-04-15T15:21:56.461 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54390.log 2024-04-15T15:21:56.461 INFO:teuthology.orchestra.run.smithi105.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45561.log.gz 2024-04-15T15:21:56.461 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.56606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50923.log 2024-04-15T15:21:56.461 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.54390.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56606.log.gz 2024-04-15T15:21:56.461 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51588.log 2024-04-15T15:21:56.461 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50923.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-15T15:21:56.461 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51588.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.54390.log.gz 2024-04-15T15:21:56.462 INFO:teuthology.orchestra.run.smithi105.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.50923.log.gz 2024-04-15T15:21:56.462 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47265.log 2024-04-15T15:21:56.462 INFO:teuthology.orchestra.run.smithi105.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51588.log.gz 2024-04-15T15:21:56.462 INFO:teuthology.orchestra.run.smithi105.stderr:gzip/var/log/ceph/ceph-osd.1.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.54313.log 2024-04-15T15:21:56.462 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50671.log 2024-04-15T15:21:56.462 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.54313.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47265.log.gz 2024-04-15T15:21:56.462 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50089.log 2024-04-15T15:21:56.462 INFO:teuthology.orchestra.run.smithi105.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54313.log.gz 2024-04-15T15:21:56.463 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45957.log 2024-04-15T15:21:56.463 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50671.log: /var/log/ceph/ceph-client.admin.50089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48540.log 2024-04-15T15:21:56.463 INFO:teuthology.orchestra.run.smithi105.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50671.log.gz 2024-04-15T15:21:56.463 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45957.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50089.log.gz 2024-04-15T15:21:56.463 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52365.log 2024-04-15T15:21:56.463 INFO:teuthology.orchestra.run.smithi105.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45957.log.gz 2024-04-15T15:21:56.463 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57672.log 2024-04-15T15:21:56.463 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48540.log: /var/log/ceph/ceph-client.admin.52365.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48540.log.gz 2024-04-15T15:21:56.464 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53889.log 2024-04-15T15:21:56.464 INFO:teuthology.orchestra.run.smithi105.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52365.log.gz 2024-04-15T15:21:56.464 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.57672.log: /var/log/ceph/ceph-client.admin.45029.log 2024-04-15T15:21:56.464 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.53889.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.57672.log.gz 2024-04-15T15:21:56.464 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43713.log 2024-04-15T15:21:56.464 INFO:teuthology.orchestra.run.smithi105.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53889.log.gz 2024-04-15T15:21:56.464 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55527.log 2024-04-15T15:21:56.464 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45029.log: /var/log/ceph/ceph-client.admin.43713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52731.log 2024-04-15T15:21:56.465 INFO:teuthology.orchestra.run.smithi105.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45029.log.gz 2024-04-15T15:21:56.465 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.55527.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43713.log.gz 2024-04-15T15:21:56.465 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43651.log 2024-04-15T15:21:56.465 INFO:teuthology.orchestra.run.smithi105.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55527.log.gz 2024-04-15T15:21:56.465 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43273.log 2024-04-15T15:21:56.465 INFO:teuthology.orchestra.run.smithi105.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52731.log.gz 2024-04-15T15:21:56.465 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47223.log 2024-04-15T15:21:56.466 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.43651.log: /var/log/ceph/ceph-client.admin.43273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52689.log 2024-04-15T15:21:56.466 INFO:teuthology.orchestra.run.smithi105.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43651.log.gz 2024-04-15T15:21:56.466 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47223.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43273.log.gz 2024-04-15T15:21:56.466 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51798.log 2024-04-15T15:21:56.466 INFO:teuthology.orchestra.run.smithi105.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47223.log.gz 2024-04-15T15:21:56.466 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45477.log 2024-04-15T15:21:56.466 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52689.log: /var/log/ceph/ceph-client.admin.51798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44686.log 2024-04-15T15:21:56.466 INFO:teuthology.orchestra.run.smithi105.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52689.log.gz 2024-04-15T15:21:56.467 INFO:teuthology.orchestra.run.smithi105.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51798.log.gz 2024-04-15T15:21:56.467 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52831.log 2024-04-15T15:21:56.467 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55010.log 2024-04-15T15:21:56.467 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44686.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45477.log.gz 2024-04-15T15:21:56.467 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52831.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44686.log.gz 2024-04-15T15:21:56.467 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48702.log 2024-04-15T15:21:56.468 INFO:teuthology.orchestra.run.smithi105.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.52831.log.gz 2024-04-15T15:21:56.468 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49543.log 2024-04-15T15:21:56.468 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.55010.log: /var/log/ceph/ceph-client.admin.48702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54518.log 2024-04-15T15:21:56.468 INFO:teuthology.orchestra.run.smithi105.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55010.log.gz 2024-04-15T15:21:56.468 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49543.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48702.log.gz 2024-04-15T15:21:56.468 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52449.log 2024-04-15T15:21:56.469 INFO:teuthology.orchestra.run.smithi105.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49543.log.gz 2024-04-15T15:21:56.469 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.54518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53371.log 2024-04-15T15:21:56.469 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52449.log: gzip -5 90.7% --verbose -- replaced with /var/log/ceph/ceph-client.admin.54518.log.gz -- 2024-04-15T15:21:56.469 INFO:teuthology.orchestra.run.smithi105.stderr: /var/log/ceph/ceph-client.admin.56125.log 2024-04-15T15:21:56.469 INFO:teuthology.orchestra.run.smithi105.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52449.log.gz 2024-04-15T15:21:56.469 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50047.log 2024-04-15T15:21:56.470 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.53371.log: /var/log/ceph/ceph-client.admin.56125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55915.log 2024-04-15T15:21:56.470 INFO:teuthology.orchestra.run.smithi105.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53371.log.gz 2024-04-15T15:21:56.470 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50047.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.56125.log.gz 2024-04-15T15:21:56.470 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37665.log 2024-04-15T15:21:56.470 INFO:teuthology.orchestra.run.smithi105.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50047.log.gz/var/log/ceph/ceph-client.admin.55915.log: 2024-04-15T15:21:56.470 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.46865.log 2024-04-15T15:21:56.470 INFO:teuthology.orchestra.run.smithi105.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55915.log.gz 2024-04-15T15:21:56.470 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.37665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56563.log 2024-04-15T15:21:56.471 INFO:teuthology.orchestra.run.smithi105.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37665.log.gz 2024-04-15T15:21:56.471 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.2.46865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56774.log 2024-04-15T15:21:56.471 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56377.log 2024-04-15T15:21:56.471 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.56563.log: 90.8% -- replaced with /var/log/ceph/ceph-client.2.46865.log.gz 2024-04-15T15:21:56.471 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.56774.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-15T15:21:56.471 INFO:teuthology.orchestra.run.smithi105.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56563.log.gz 2024-04-15T15:21:56.471 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52873.log 2024-04-15T15:21:56.471 INFO:teuthology.orchestra.run.smithi105.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56774.log.gz 2024-04-15T15:21:56.471 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.56377.log: /var/log/ceph/ceph-osd.3.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56377.log.gz 2024-04-15T15:21:56.472 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48829.log 2024-04-15T15:21:56.472 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49423.log 2024-04-15T15:21:56.472 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52873.log: /var/log/ceph/ceph-client.admin.48829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53245.log 2024-04-15T15:21:56.472 INFO:teuthology.orchestra.run.smithi105.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52873.log.gz 2024-04-15T15:21:56.472 INFO:teuthology.orchestra.run.smithi105.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48829.log.gz 2024-04-15T15:21:56.472 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52407.log 2024-04-15T15:21:56.472 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.53245.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49423.log.gz 2024-04-15T15:21:56.472 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47517.log 2024-04-15T15:21:56.472 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52407.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53245.log.gz 2024-04-15T15:21:56.473 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51248.log 2024-04-15T15:21:56.473 INFO:teuthology.orchestra.run.smithi105.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52407.log.gz 2024-04-15T15:21:56.473 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48955.log 2024-04-15T15:21:56.473 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56648.log 2024-04-15T15:21:56.473 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51248.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.47517.log.gz 2024-04-15T15:21:56.473 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49239.log 2024-04-15T15:21:56.473 INFO:teuthology.orchestra.run.smithi105.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.51248.log.gz 2024-04-15T15:21:56.473 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.56648.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48955.log.gz 2024-04-15T15:21:56.474 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49339.log 2024-04-15T15:21:56.474 INFO:teuthology.orchestra.run.smithi105.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.56648.log.gz 2024-04-15T15:21:56.474 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44683.log 2024-04-15T15:21:56.474 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50839.log 2024-04-15T15:21:56.474 INFO:teuthology.orchestra.run.smithi105.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49239.log.gz 2024-04-15T15:21:56.474 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49339.log: /var/log/ceph/ceph-client.admin.44683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43085.log 2024-04-15T15:21:56.474 INFO:teuthology.orchestra.run.smithi105.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49339.log.gz 2024-04-15T15:21:56.474 INFO:teuthology.orchestra.run.smithi105.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44683.log.gz 2024-04-15T15:21:56.475 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44909.log 2024-04-15T15:21:56.475 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.43085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48313.log 2024-04-15T15:21:56.475 INFO:teuthology.orchestra.run.smithi105.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50839.log.gz 2024-04-15T15:21:56.475 INFO:teuthology.orchestra.run.smithi105.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43085.log.gz 2024-04-15T15:21:56.475 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51109.log 2024-04-15T15:21:56.475 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48313.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44909.log.gz 2024-04-15T15:21:56.475 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57209.log 2024-04-15T15:21:56.475 INFO:teuthology.orchestra.run.smithi105.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48313.log.gz 2024-04-15T15:21:56.475 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49627.log 2024-04-15T15:21:56.476 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.57209.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51109.log.gz 2024-04-15T15:21:56.476 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45267.log 2024-04-15T15:21:56.476 INFO:teuthology.orchestra.run.smithi105.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57209.log.gz 2024-04-15T15:21:56.476 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54864.log 2024-04-15T15:21:56.476 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49879.log 2024-04-15T15:21:56.476 INFO:teuthology.orchestra.run.smithi105.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49627.log.gz 2024-04-15T15:21:56.476 INFO:teuthology.orchestra.run.smithi105.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45267.log.gz 2024-04-15T15:21:56.476 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.54864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47811.log 2024-04-15T15:21:56.477 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49879.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54864.log.gz 2024-04-15T15:21:56.477 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57072.log 2024-04-15T15:21:56.477 INFO:teuthology.orchestra.run.smithi105.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49879.log.gz 2024-04-15T15:21:56.477 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45393.log 2024-04-15T15:21:56.477 INFO:teuthology.orchestra.run.smithi105.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47811.log.gz 2024-04-15T15:21:56.477 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.57072.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57072.log.gz 2024-04-15T15:21:56.477 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49197.log 2024-04-15T15:21:56.477 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53035.log 2024-04-15T15:21:56.477 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45393.log: /var/log/ceph/ceph-client.admin.49197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43965.log 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45393.log.gz 2024-04-15T15:21:56.478 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-04-15T15:21:56.478 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.53035.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49197.log.gz 2024-04-15T15:21:56.478 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44688.log 2024-04-15T15:21:56.478 INFO:teuthology.orchestra.run.smithi105.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.53035.log.gz 2024-04-15T15:21:56.478 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.43965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51025.log 2024-04-15T15:21:56.478 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44688.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43965.log.gz 2024-04-15T15:21:56.478 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44007.log 2024-04-15T15:21:56.478 INFO:teuthology.orchestra.run.smithi105.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44688.log.gz 2024-04-15T15:21:56.479 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55401.log 2024-04-15T15:21:56.479 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57882.log 2024-04-15T15:21:56.479 INFO:teuthology.orchestra.run.smithi105.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51025.log.gz 2024-04-15T15:21:56.479 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.55401.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44007.log.gz 2024-04-15T15:21:56.479 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-15T15:21:56.479 INFO:teuthology.orchestra.run.smithi105.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55401.log.gz 2024-04-15T15:21:56.479 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.57882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56335.log 2024-04-15T15:21:56.479 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57964.log 2024-04-15T15:21:56.480 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-mgr.y.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.57882.log.gz 2024-04-15T15:21:56.480 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.56335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47475.log 2024-04-15T15:21:56.480 INFO:teuthology.orchestra.run.smithi105.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.56335.log.gz 2024-04-15T15:21:56.480 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.57964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49921.log 2024-04-15T15:21:56.480 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 88.5%/var/log/ceph/ceph-client.admin.47475.log: -- replaced with /var/log/ceph/ceph-client.admin.57964.log.gz 2024-04-15T15:21:56.480 INFO:teuthology.orchestra.run.smithi105.stderr: --verbose -- /var/log/ceph/ceph-client.admin.48414.log 2024-04-15T15:21:56.480 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49921.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47475.log.gz 2024-04-15T15:21:56.480 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57588.log 2024-04-15T15:21:56.481 INFO:teuthology.orchestra.run.smithi105.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49921.log.gz 2024-04-15T15:21:56.481 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54456.log 2024-04-15T15:21:56.481 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.57588.log: 88.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52915.log 2024-04-15T15:21:56.481 INFO:teuthology.orchestra.run.smithi105.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48414.log.gz 2024-04-15T15:21:56.481 INFO:teuthology.orchestra.run.smithi105.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.57588.log.gz 2024-04-15T15:21:56.481 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.54456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50173.log 2024-04-15T15:21:56.481 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52915.log: gzip -5 --verbose 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54456.log.gz 2024-04-15T15:21:56.481 INFO:teuthology.orchestra.run.smithi105.stderr: -- /var/log/ceph/ceph-client.0.52166.log 2024-04-15T15:21:56.482 INFO:teuthology.orchestra.run.smithi105.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52915.log.gz 2024-04-15T15:21:56.482 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50173.log: gzip -5 --verbose -- 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50173.log.gz 2024-04-15T15:21:56.482 INFO:teuthology.orchestra.run.smithi105.stderr: /var/log/ceph/ceph-client.admin.45027.log 2024-04-15T15:21:56.482 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.0.52166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48913.log 2024-04-15T15:21:56.482 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47895.log 2024-04-15T15:21:56.482 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48913.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45027.log.gz 2024-04-15T15:21:56.482 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50509.log 2024-04-15T15:21:56.483 INFO:teuthology.orchestra.run.smithi105.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.0.52166.log.gz 2024-04-15T15:21:56.483 INFO:teuthology.orchestra.run.smithi105.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48913.log.gz 2024-04-15T15:21:56.483 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57030.log 2024-04-15T15:21:56.483 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47895.log: /var/log/ceph/ceph-client.admin.50509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49837.log 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47895.log.gz 2024-04-15T15:21:56.483 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-04-15T15:21:56.483 INFO:teuthology.orchestra.run.smithi105.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50509.log.gz 2024-04-15T15:21:56.483 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.57030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55052.log 2024-04-15T15:21:56.483 INFO:teuthology.orchestra.run.smithi105.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57030.log.gz 2024-04-15T15:21:56.484 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47727.log 2024-04-15T15:21:56.484 INFO:teuthology.orchestra.run.smithi105.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49837.log.gz 2024-04-15T15:21:56.484 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.55052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50713.log 2024-04-15T15:21:56.484 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45022.log 2024-04-15T15:21:56.484 INFO:teuthology.orchestra.run.smithi105.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.55052.log.gz 2024-04-15T15:21:56.484 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50713.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47727.log.gz 2024-04-15T15:21:56.484 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56732.log 2024-04-15T15:21:56.484 INFO:teuthology.orchestra.run.smithi105.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50713.log.gz 2024-04-15T15:21:56.485 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48456.log 2024-04-15T15:21:56.485 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45022.log: /var/log/ceph/ceph-client.admin.56732.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-15T15:21:56.485 INFO:teuthology.orchestra.run.smithi105.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45022.log.gz 2024-04-15T15:21:56.485 INFO:teuthology.orchestra.run.smithi105.stderr: 88.1%/var/log/ceph/ceph-client.admin.48456.log: -- replaced with /var/log/ceph/ceph-client.admin.56732.log.gz 2024-04-15T15:21:56.485 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50299.log 2024-04-15T15:21:56.485 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-osd.0.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48456.log.gz 2024-04-15T15:21:56.485 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44348.log 2024-04-15T15:21:56.485 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56503.log 2024-04-15T15:21:56.485 INFO:teuthology.orchestra.run.smithi105.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50299.log.gz 2024-04-15T15:21:56.486 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43481.log 2024-04-15T15:21:56.486 INFO:teuthology.orchestra.run.smithi105.stderr: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47181.log 2024-04-15T15:21:56.486 INFO:teuthology.orchestra.run.smithi105.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44348.log.gz 2024-04-15T15:21:56.486 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.56503.log: /var/log/ceph/ceph-client.admin.43481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52323.log 2024-04-15T15:21:56.486 INFO:teuthology.orchestra.run.smithi105.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.56503.log.gz 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43481.log.gz 2024-04-15T15:21:56.486 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47181.log: 2024-04-15T15:21:56.486 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47181.log.gz 2024-04-15T15:21:56.486 INFO:teuthology.orchestra.run.smithi105.stderr: --verbose -- /var/log/ceph/ceph-client.admin.56461.log 2024-04-15T15:21:56.487 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47349.log 2024-04-15T15:21:56.487 INFO:teuthology.orchestra.run.smithi105.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52323.log.gz 2024-04-15T15:21:56.487 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.56461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49669.log 2024-04-15T15:21:56.487 INFO:teuthology.orchestra.run.smithi105.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.56461.log.gz 2024-04-15T15:21:56.487 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51840.log 2024-04-15T15:21:56.487 INFO:teuthology.orchestra.run.smithi105.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47349.log.gz 2024-04-15T15:21:56.487 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53077.log 2024-04-15T15:21:56.487 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49669.log: /var/log/ceph/ceph-client.admin.51840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49753.log 2024-04-15T15:21:56.488 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.53077.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49669.log.gz 2024-04-15T15:21:56.488 INFO:teuthology.orchestra.run.smithi105.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.53077.log.gz 2024-04-15T15:21:56.488 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46987.log 2024-04-15T15:21:56.488 INFO:teuthology.orchestra.run.smithi105.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.51840.log.gz 2024-04-15T15:21:56.488 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47853.log 2024-04-15T15:21:56.488 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49753.log: /var/log/ceph/ceph-client.admin.46987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45519.log 2024-04-15T15:21:56.488 INFO:teuthology.orchestra.run.smithi105.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46987.log.gz 2024-04-15T15:21:56.488 INFO:teuthology.orchestra.run.smithi105.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49753.log.gz 2024-04-15T15:21:56.489 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56293.log 2024-04-15T15:21:56.489 INFO:teuthology.orchestra.run.smithi105.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47853.log.gz/var/log/ceph/ceph-client.admin.45519.log: 2024-04-15T15:21:56.489 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-15T15:21:56.489 INFO:teuthology.orchestra.run.smithi105.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45519.log.gz 2024-04-15T15:21:56.489 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.56293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45645.log 2024-04-15T15:21:56.489 INFO:teuthology.orchestra.run.smithi105.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56293.log.gz 2024-04-15T15:21:56.489 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52505.log 2024-04-15T15:21:56.489 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45026.log 2024-04-15T15:21:56.490 INFO:teuthology.orchestra.run.smithi105.stderr: 84.5% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-15T15:21:56.490 INFO:teuthology.orchestra.run.smithi105.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45645.log.gz 2024-04-15T15:21:56.490 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43881.log 2024-04-15T15:21:56.490 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48021.log 2024-04-15T15:21:56.490 INFO:teuthology.orchestra.run.smithi105.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52505.log.gz 2024-04-15T15:21:56.490 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.43881.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45026.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54920.log 2024-04-15T15:21:56.490 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-04-15T15:21:56.491 INFO:teuthology.orchestra.run.smithi105.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43881.log.gz 2024-04-15T15:21:56.491 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-15T15:21:56.491 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48021.log: /var/log/ceph/ceph-client.admin.54920.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48021.log.gz 2024-04-15T15:21:56.491 INFO:teuthology.orchestra.run.smithi105.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54920.log.gz 2024-04-15T15:21:56.491 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57378.log 2024-04-15T15:21:56.491 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49113.log 2024-04-15T15:21:56.491 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-osd.2.log: /var/log/ceph/ceph-client.admin.57378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48660.log 2024-04-15T15:21:56.491 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49113.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.57378.log.gz 2024-04-15T15:21:56.492 INFO:teuthology.orchestra.run.smithi105.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49113.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48997.log 2024-04-15T15:21:56.492 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-04-15T15:21:56.492 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48660.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-15T15:21:56.492 INFO:teuthology.orchestra.run.smithi105.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48660.log.gz 2024-04-15T15:21:56.492 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51067.log 2024-04-15T15:21:56.492 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-mds.c.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48997.log.gz 2024-04-15T15:21:56.492 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45771.log 2024-04-15T15:21:56.492 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51067.log: 92.3% 89.5% -- replaced with /var/log/ceph/ceph-client.admin.51067.log.gz 2024-04-15T15:21:56.493 INFO:teuthology.orchestra.run.smithi105.stderr: -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-15T15:21:56.493 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43396.log 2024-04-15T15:21:56.493 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43439.log 2024-04-15T15:21:56.493 INFO:teuthology.orchestra.run.smithi105.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45771.log.gz 2024-04-15T15:21:56.493 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.43396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56419.log 2024-04-15T15:21:56.493 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.43439.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43396.log.gz 2024-04-15T15:21:56.493 INFO:teuthology.orchestra.run.smithi105.stderr: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53203.log 2024-04-15T15:21:56.493 INFO:teuthology.orchestra.run.smithi105.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43439.log.gz 2024-04-15T15:21:56.494 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43839.log 2024-04-15T15:21:56.494 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.56419.log: /var/log/ceph/ceph-client.admin.53203.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56419.log.gz 2024-04-15T15:21:56.494 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46489.log 2024-04-15T15:21:56.494 INFO:teuthology.orchestra.run.smithi105.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53203.log.gz 2024-04-15T15:21:56.494 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.43839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55239.log 2024-04-15T15:21:56.494 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.46489.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43839.log.gz 2024-04-15T15:21:56.494 INFO:teuthology.orchestra.run.smithi105.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.46489.log.gz 2024-04-15T15:21:56.495 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44355.log 2024-04-15T15:21:56.495 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37310.log 2024-04-15T15:21:56.495 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.55239.log: /var/log/ceph/ceph-client.admin.44355.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.55239.log.gz 2024-04-15T15:21:56.495 INFO:teuthology.orchestra.run.smithi105.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44355.log.gz 2024-04-15T15:21:56.495 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45025.log 2024-04-15T15:21:56.495 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52563.log 2024-04-15T15:21:56.495 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45025.log: /var/log/ceph/ceph-client.admin.37310.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.37310.log.gz 2024-04-15T15:21:56.496 INFO:teuthology.orchestra.run.smithi105.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45025.log.gz 2024-04-15T15:21:56.496 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44349.log 2024-04-15T15:21:56.496 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44681.log 2024-04-15T15:21:56.496 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52563.log: /var/log/ceph/ceph-client.admin.44349.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52563.log.gz 2024-04-15T15:21:56.496 INFO:teuthology.orchestra.run.smithi105.stderr: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44687.log 2024-04-15T15:21:56.496 INFO:teuthology.orchestra.run.smithi105.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44349.log.gz 2024-04-15T15:21:56.496 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47559.log 2024-04-15T15:21:56.496 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44687.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44681.log.gz 2024-04-15T15:21:56.497 INFO:teuthology.orchestra.run.smithi105.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44687.log.gz 2024-04-15T15:21:56.497 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45687.log 2024-04-15T15:21:56.497 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56251.log 2024-04-15T15:21:56.497 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47559.log: /var/log/ceph/ceph-client.admin.45687.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47559.log.gz 2024-04-15T15:21:56.497 INFO:teuthology.orchestra.run.smithi105.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.45687.log.gzgzip 2024-04-15T15:21:56.497 INFO:teuthology.orchestra.run.smithi105.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.44133.log 2024-04-15T15:21:56.497 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50755.log 2024-04-15T15:21:56.497 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44133.log: /var/log/ceph/ceph-client.admin.56251.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56251.log.gz 2024-04-15T15:21:56.498 INFO:teuthology.orchestra.run.smithi105.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44133.log.gz 2024-04-15T15:21:56.498 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53287.log 2024-04-15T15:21:56.498 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49711.log 2024-04-15T15:21:56.498 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50755.log: /var/log/ceph/ceph-client.admin.53287.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50755.log.gz 2024-04-15T15:21:56.498 INFO:teuthology.orchestra.run.smithi105.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53287.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44692.log 2024-04-15T15:21:56.498 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-04-15T15:21:56.498 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51756.log 2024-04-15T15:21:56.498 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44692.log: /var/log/ceph/ceph-client.admin.49711.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49711.log.gz 2024-04-15T15:21:56.499 INFO:teuthology.orchestra.run.smithi105.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44692.log.gz 2024-04-15T15:21:56.499 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48271.log 2024-04-15T15:21:56.499 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53119.log 2024-04-15T15:21:56.499 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51756.log: /var/log/ceph/ceph-client.admin.48271.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51756.log.gz 2024-04-15T15:21:56.499 INFO:teuthology.orchestra.run.smithi105.stderr: 89.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.48271.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.45309.log 2024-04-15T15:21:56.499 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-04-15T15:21:56.499 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47391.log 2024-04-15T15:21:56.499 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.53119.log: /var/log/ceph/ceph-client.admin.45309.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45309.log.gz 2024-04-15T15:21:56.500 INFO:teuthology.orchestra.run.smithi105.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53119.log.gz 2024-04-15T15:21:56.500 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57840.log 2024-04-15T15:21:56.500 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55281.log 2024-04-15T15:21:56.500 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47391.log: /var/log/ceph/ceph-client.admin.57840.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47391.log.gz 2024-04-15T15:21:56.500 INFO:teuthology.orchestra.run.smithi105.stderr: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51332.log 2024-04-15T15:21:56.500 INFO:teuthology.orchestra.run.smithi105.stderr: -- replaced with /var/log/ceph/ceph-client.admin.57840.log.gz 2024-04-15T15:21:56.500 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.55281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43523.log 2024-04-15T15:21:56.500 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51332.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55281.log.gz 2024-04-15T15:21:56.500 INFO:teuthology.orchestra.run.smithi105.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51332.log.gz 2024-04-15T15:21:56.501 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50383.log 2024-04-15T15:21:56.501 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.43523.log: -- /var/log/ceph/ceph-mds.a.log 2024-04-15T15:21:56.501 INFO:teuthology.orchestra.run.smithi105.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43523.log.gz 2024-04-15T15:21:56.501 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55197.log 2024-04-15T15:21:56.501 INFO:teuthology.orchestra.run.smithi105.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50383.log.gz 2024-04-15T15:21:56.501 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50629.log 2024-04-15T15:21:56.501 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.55197.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55197.log.gz 2024-04-15T15:21:56.501 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52605.log 2024-04-15T15:21:56.502 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50629.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50629.log.gz 2024-04-15T15:21:56.502 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52605.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52605.log.gz 2024-04-15T15:21:56.502 INFO:teuthology.orchestra.run.smithi105.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-15T15:21:56.504 INFO:teuthology.orchestra.run.smithi105.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-15T15:21:56.504 INFO:teuthology.orchestra.run.smithi105.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-15T15:21:56.504 INFO:teuthology.orchestra.run.smithi105.stderr: 92.4% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-15T15:21:56.504 INFO:teuthology.orchestra.run.smithi105.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-15T15:21:56.504 INFO:teuthology.orchestra.run.smithi105.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-15T15:21:56.504 INFO:teuthology.orchestra.run.smithi105.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-15T15:21:56.505 INFO:teuthology.orchestra.run.smithi154.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-15T15:21:56.505 INFO:teuthology.orchestra.run.smithi154.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-15T15:21:56.505 INFO:teuthology.orchestra.run.smithi154.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-15T15:21:56.505 INFO:teuthology.orchestra.run.smithi154.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-15T15:21:56.505 INFO:teuthology.orchestra.run.smithi154.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-15T15:21:56.506 INFO:teuthology.orchestra.run.smithi154.stderr: 2024-04-15T15:21:56.506 INFO:teuthology.orchestra.run.smithi154.stderr:real 0m0.156s 2024-04-15T15:21:56.506 INFO:teuthology.orchestra.run.smithi154.stderr:user 0m0.739s 2024-04-15T15:21:56.506 INFO:teuthology.orchestra.run.smithi154.stderr:sys 0m0.058s 2024-04-15T15:21:56.507 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-04-15T15:21:56.507 INFO:teuthology.orchestra.run.smithi105.stderr:real 0m0.234s 2024-04-15T15:21:56.507 INFO:teuthology.orchestra.run.smithi105.stderr:user 0m1.041s 2024-04-15T15:21:56.507 INFO:teuthology.orchestra.run.smithi105.stderr:sys 0m0.210s 2024-04-15T15:21:56.508 INFO:tasks.ceph:Archiving logs... 2024-04-15T15:21:56.508 DEBUG:teuthology.misc:Transferring archived files from smithi105:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-15_14:54:08-fs-wip-lusov-quiesce-xlock-distro-default-smithi/7657132/remote/smithi105/log 2024-04-15T15:21:56.509 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-15T15:21:56.928 DEBUG:teuthology.misc:Transferring archived files from smithi154:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-15_14:54:08-fs-wip-lusov-quiesce-xlock-distro-default-smithi/7657132/remote/smithi154/log 2024-04-15T15:21:56.929 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-15T15:21:57.085 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-15T15:21:57.095 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-15T15:21:57.095 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-15T15:21:57.098 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-15T15:21:57.193 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-15T15:21:57.202 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-15T15:21:57.203 DEBUG:teuthology.orchestra.run.smithi105:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-15T15:21:57.206 DEBUG:teuthology.orchestra.run.smithi154:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-15T15:21:57.218 INFO:teuthology.orchestra.run.smithi105.stderr:bash: line 1: ntpq: command not found 2024-04-15T15:21:57.221 INFO:teuthology.orchestra.run.smithi154.stderr:bash: line 1: ntpq: command not found 2024-04-15T15:21:57.222 INFO:teuthology.orchestra.run.smithi105.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-15T15:21:57.222 INFO:teuthology.orchestra.run.smithi105.stdout:=============================================================================== 2024-04-15T15:21:57.222 INFO:teuthology.orchestra.run.smithi105.stdout:^* hv01.front.sepia.ceph.com 3 6 377 13 +1796us[+1613us] +/- 54ms 2024-04-15T15:21:57.222 INFO:teuthology.orchestra.run.smithi105.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 11 -1549us[-1549us] +/- 53ms 2024-04-15T15:21:57.222 INFO:teuthology.orchestra.run.smithi105.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 11 -474us[ -474us] +/- 78ms 2024-04-15T15:21:57.223 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-15T15:21:57.225 INFO:teuthology.orchestra.run.smithi154.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-15T15:21:57.225 INFO:teuthology.orchestra.run.smithi154.stdout:=============================================================================== 2024-04-15T15:21:57.226 INFO:teuthology.orchestra.run.smithi154.stdout:^* hv01.front.sepia.ceph.com 3 6 377 12 +1826us[+1659us] +/- 54ms 2024-04-15T15:21:57.226 INFO:teuthology.orchestra.run.smithi154.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 11 -1515us[-1515us] +/- 53ms 2024-04-15T15:21:57.226 INFO:teuthology.orchestra.run.smithi154.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 11 -441us[ -441us] +/- 78ms 2024-04-15T15:21:57.226 INFO:teuthology.orchestra.run.smithi154.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-15T15:21:57.226 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-15T15:21:57.236 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-15T15:21:57.237 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-15T15:21:57.245 DEBUG:teuthology.orchestra.run.smithi105:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-15T15:21:57.339 DEBUG:teuthology.orchestra.run.smithi154:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-15T15:21:57.409 DEBUG:teuthology.orchestra.run.smithi105:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-15T15:21:57.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-15T15:21:57.441 DEBUG:teuthology.orchestra.run.smithi154:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-15T15:21:57.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-15T15:21:57.473 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-15T15:21:57.483 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-15T15:21:57.491 INFO:teuthology.task.internal:Duration was 600.643536 seconds 2024-04-15T15:21:57.492 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-15T15:21:57.501 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-15T15:21:57.501 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-15T15:21:57.504 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-15T15:21:57.547 INFO:teuthology.orchestra.run.smithi105.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-15T15:21:57.558 INFO:teuthology.orchestra.run.smithi154.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-15T15:21:57.851 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-15T15:21:57.851 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi105.front.sepia.ceph.com 2024-04-15T15:21:57.852 DEBUG:teuthology.orchestra.run.smithi105:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-15T15:21:57.873 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi154.front.sepia.ceph.com 2024-04-15T15:21:57.873 DEBUG:teuthology.orchestra.run.smithi154:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-15T15:21:57.910 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-15T15:21:57.910 DEBUG:teuthology.orchestra.run.smithi105:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-15T15:21:57.916 DEBUG:teuthology.orchestra.run.smithi154:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-15T15:21:57.980 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-15T15:21:57.980 DEBUG:teuthology.orchestra.run.smithi105:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-15T15:21:57.986 DEBUG:teuthology.orchestra.run.smithi154:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-15T15:21:58.122 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-15T15:21:58.133 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-15T15:21:58.133 DEBUG:teuthology.orchestra.run.smithi105:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-15T15:21:58.162 DEBUG:teuthology.orchestra.run.smithi154:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-15T15:21:58.190 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-15T15:21:58.200 DEBUG:teuthology.orchestra.run.smithi105:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-15T15:21:58.205 DEBUG:teuthology.orchestra.run.smithi154:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-15T15:21:58.229 INFO:teuthology.orchestra.run.smithi105.stdout:kernel.core_pattern = core 2024-04-15T15:21:58.257 INFO:teuthology.orchestra.run.smithi154.stdout:kernel.core_pattern = core 2024-04-15T15:21:58.273 DEBUG:teuthology.orchestra.run.smithi105:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-15T15:21:58.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-15T15:21:58.301 DEBUG:teuthology.orchestra.run.smithi154:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-15T15:21:58.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-15T15:21:58.328 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-15T15:21:58.338 INFO:teuthology.task.internal:Transferring archived files... 2024-04-15T15:21:58.339 DEBUG:teuthology.misc:Transferring archived files from smithi105:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-15_14:54:08-fs-wip-lusov-quiesce-xlock-distro-default-smithi/7657132/remote/smithi105 2024-04-15T15:21:58.339 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-15T15:21:58.380 DEBUG:teuthology.misc:Transferring archived files from smithi154:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-15_14:54:08-fs-wip-lusov-quiesce-xlock-distro-default-smithi/7657132/remote/smithi154 2024-04-15T15:21:58.380 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-15T15:21:58.417 INFO:teuthology.task.internal:Removing archive directory... 2024-04-15T15:21:58.417 DEBUG:teuthology.orchestra.run.smithi105:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-15T15:21:58.419 DEBUG:teuthology.orchestra.run.smithi154:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-15T15:21:58.466 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-15T15:21:58.478 INFO:teuthology.task.internal:Not uploading archives. 2024-04-15T15:21:58.479 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-15T15:21:58.488 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-15T15:21:58.489 DEBUG:teuthology.orchestra.run.smithi105:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-15T15:21:58.491 DEBUG:teuthology.orchestra.run.smithi154:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-15T15:21:58.503 INFO:teuthology.orchestra.run.smithi105.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 15 15:21 /home/ubuntu/cephtest 2024-04-15T15:21:58.521 INFO:teuthology.orchestra.run.smithi154.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 15 15:21 /home/ubuntu/cephtest 2024-04-15T15:21:58.522 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-15T15:21:58.533 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-15T15:21:58.567 INFO:teuthology.nuke:Checking targets against current locks 2024-04-15T15:21:58.593 DEBUG:teuthology.nuke:shortname: smithi105 2024-04-15T15:21:58.594 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-15T15:21:58.614 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi105.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-15_14:54:08-fs-wip-lusov-quiesce-xlock-distro-default-smithi/7657132', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-15 15:02:54.101133', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGtK/GTXibZ8m3DeYPVGTDDt7HFJ9BpvJbR1yk41DfL7oxO7ofgneKZl4qsIl1MlnWxCCYjI3Wgx1TP31Ne8oRM='} 2024-04-15T15:21:58.620 DEBUG:teuthology.nuke:shortname: smithi154 2024-04-15T15:21:58.620 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-15T15:21:58.641 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi154.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-15_14:54:08-fs-wip-lusov-quiesce-xlock-distro-default-smithi/7657132', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-15 15:02:54.103150', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEfwTt4MW0X5hlmNE9lb0QI3waz4vvcx9Y/jhtC4t1Ux/qKV/DMJmxst2RT3YbWTceKKt3owaT80t6MOstAUvfs='} 2024-04-15T15:21:58.662 INFO:teuthology.orchestra.console.smithi105:Power off 2024-04-15T15:21:58.662 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-15T15:21:58.688 DEBUG:teuthology.orchestra.console.smithi105:power off output: Chassis Power Control: Down/Off 2024-04-15T15:21:58.688 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-15T15:21:58.707 INFO:teuthology.orchestra.console.smithi154:Power off 2024-04-15T15:21:58.707 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-15T15:21:58.717 DEBUG:teuthology.orchestra.console.smithi105:check power output: Chassis Power is on 2024-04-15T15:21:58.730 DEBUG:teuthology.orchestra.console.smithi154:power off output: Chassis Power Control: Down/Off 2024-04-15T15:21:58.730 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-15T15:21:58.753 DEBUG:teuthology.orchestra.console.smithi154:check power output: Chassis Power is on 2024-04-15T15:22:02.718 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-15T15:22:02.755 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-15T15:22:02.829 DEBUG:teuthology.orchestra.console.smithi105:check power output: Chassis Power is on 2024-04-15T15:22:02.866 DEBUG:teuthology.orchestra.console.smithi154:check power output: Chassis Power is on 2024-04-15T15:22:06.832 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-15T15:22:06.866 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-15T15:22:06.944 DEBUG:teuthology.orchestra.console.smithi105:check power output: Chassis Power is on 2024-04-15T15:22:06.979 DEBUG:teuthology.orchestra.console.smithi154:check power output: Chassis Power is off 2024-04-15T15:22:07.080 INFO:teuthology.orchestra.console.smithi154:Power off completed 2024-04-15T15:22:10.946 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-15T15:22:11.057 DEBUG:teuthology.orchestra.console.smithi105:check power output: Chassis Power is off 2024-04-15T15:22:11.158 INFO:teuthology.orchestra.console.smithi105:Power off completed 2024-04-15T15:22:11.263 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/quiesce} duration: 600.6435358524323 failure_reason: 'Test failure: test_quiesce_blocked (tasks.cephfs.test_quiesce.TestQuiesce)' flavor: default owner: scheduled_leonidus@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=a5d90336d7714916919c8bcf287d1c5f status: fail success: false 2024-04-15T15:22:11.263 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-15T15:22:11.346 INFO:teuthology.run:FAIL